Article management method, article management device, article management program, and method for manufacturing steel pipe

Abstract

To provide a technique capable of more accurately associating an article with an identification label attached to the article and improving the tracking accuracy of the article.SOLUTION: An imaging unit 4 that acquires image data obtained by imaging the article 2 being conveyed; a label recognition unit 11 ba that detects the identification label 3 in the image data and acquires position information of the detected identification label 3 in the image data as label position information; The article management device includes an article recognition part 11Bb for acquiring position information in the image of the detected article 2 as article 2 position information, and an association part 11C for associating the article 2 with the identification label 3 from the label position information and the article 2 position information acquired from the same image.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a technology for tracking and managing items transported along a conveyor line. The present invention is directed to items having an identification label, such as a one-dimensional barcode, a two-dimensional barcode, or a QR code (registered trademark), attached to their surface. The present invention is a technology suitable for managing items such as steel pipes with a circular cross section. Specifically, the present invention is a technology that is particularly effective when tracking items that are transported while the orientation of the surface to which the identification label is attached changes. A situation in which the orientation of the surface to which the identification label is attached changes during transport occurs, for example, when the item is transported while rolling or vibrating.

[0002] Furthermore, even in a situation where the orientation of the identification labels does not change during transportation, the present invention is also suitable for transporting articles in which the orientations of the surfaces to which the identification labels are attached are random. The application of an identification label in the present invention also includes application to an article by engraving or printing. [Background technology]

[0003] Patent Document 1 describes a method for tracking an item by acquiring identification information and position information from image data of the item.

[0004] Patent Document 2 also describes tracking the distribution of goods by acquiring location information and detection time using a sensor. Patent Document 2 also describes acquiring goods information such as roll diameter by reading the tag (identification label) of the roll using a handy terminal.

[0005] When multiple items with identification labels attached are transferred to another process for further processing, the order of the multiple items may become random. In such cases, the other process may need to track the items to be transported and manage the order of the items being transported. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent Application No. 2022-552726 [Patent Document 2] Patent Application No. 2012-200751 Summary of the Invention [Problem to be solved by the invention]

[0007] Here, when steel materials are considered as articles, steel materials include thick plates, steel sections, steel pipes, and the like. First, let's consider the case where the item is a large steel material such as a thick plate. In this case, when the item is transported by conveying equipment such as a belt conveyor or conveying rollers, the item is transported in batches at intervals without rolling. Next, let's consider the case where the item is a small steel material such as a steel beam. In this case, too, when the item is transported by conveying equipment such as a belt conveyor or conveying rollers, the item is transported continuously without rolling.

[0008] Next, consider the case of an object with a circular cross section, such as a steel pipe. In this case, the object is transported by continuously rolling on a transport skid, for example. In this case, the orientation of the identification label attached to the object changes as the object is transported, and depending on the timing of the image capture, the identification label may not be facing the camera. In other words, when an object is transported while rolling, there may be cases where the identification label attached to the object is not captured in a single image data set capturing an image of the object.

[0009] Furthermore, in a situation where a plurality of articles are continuously conveyed, even if the conveyance situation is captured by a camera, there are cases where a plurality of articles are detected within the angle of view (within the image capture area).

[0010] The method of Patent Document 1 targets thick plates, acquires identification information and position information of identification labels attached to the surfaces of the plates, and tracks the plates. However, in the case of thick plates, each plate is transported without rotating at intervals in a batch. Therefore, as the plate passes through the camera's imaging area, the identification labels attached to the plate can always be imaged along with the plate. Furthermore, even in a situation where multiple thick plates are simultaneously imaged within the field of view, the positional relationship between the transported items is always constant. Therefore, the detection order of the items and the detection order of the labels can easily be associated one-to-one. Therefore, it is easy to link the information on the identification labels to the items, count the quantity, and obtain the transport order. However, the method described in Patent Document 1 cannot be used to track articles such as steel pipes.

[0011] For example, when an object with a circular cross section, such as a steel pipe, is transported while rolling, the orientation of the label attached to the object changes as the object is transported. Therefore, in consecutively captured images, it is conceivable that in one captured image data, the object can be detected but not labeled, and in the next image data, the label attached to the object is detected along with the object. Therefore, when multiple objects are transported consecutively, it is conceivable that the order in which the objects are transported (detection order) and the order in which the labels attached to each object are detected may differ.

[0012] For example, suppose that item A (label A) and item B (label B), both of the same type, are conveyed in this order. Consider a case where item A and item B are present in the imaging area, but only label B is detected in one image data (first image), and only label A is detected in the next image (second image). In this case, the order of conveyance of the items is recognized as A, B, but the order of detection of the labels is B, A. Furthermore, while the first image data (first image) and the next image data (second image) are image data captured from the same imaging area, the position of the item moves as it is conveyed. Therefore, it is conceivable that the position coordinates of label A in the next image data (second image) relative to the imaging area may be downstream in the conveyance direction from the position coordinates of label B in the first image data (first image).

[0013] For this reason, it is not possible to simply associate the order in which the objects are detected with the order in which the labels are detected based on the order in which the labels are detected or the detection coordinates of the labels. In other words, when multiple objects such as steel pipes are transported while continuously rolling, the method of Patent Document 1 makes it difficult to link the identification information with the objects or to obtain the order in which they were transported.

[0014] If the items are tracked by label detection and the transport order of the identified items is reversed, the resulting steel product will have different properties from the original steel product properties obtained from the identification labels, which can be a major problem in steel production.

[0015] Furthermore, in Patent Document 2, the distribution of rolls is tracked using multiple sensors and location information and detection time. Furthermore, a handheld terminal is used to read the roll tag, and product information such as the roll diameter is obtained from the identification label. Here, in cases where there are few items such as rolls to manage and the distribution route is fixed, the method of Patent Document 2 can also be used to manage the distribution. In other words, in the case of the distribution of rolls, for example, it is possible to manage the distribution by placing sensors and handheld terminals along the distribution route and having workers read the tags.

[0016] However, when thousands of items are transported each day along various routes depending on the required specifications of the items, the method of Patent Document 2 requires a huge number of sensors and handheld terminals, which increases costs. Furthermore, there is the problem that the task of reading the tags on each and every item with a handheld terminal places an excessive burden on the workers who perform the task.

[0017] The present invention has been made with the above points in mind, and aims to provide a technology that can more accurately match items with the identification labels attached to them, thereby improving the tracking accuracy of the items. [Means for solving the problem]

[0018] In order to solve the problem, one aspect of the present invention is an item management device that tracks multiple items each having an identification label attached thereto as the items are transported, and includes: an imaging unit that captures an imaging area where the transported items pass through and acquires image data of the items being transported; a label recognition unit that performs image analysis of the image data to detect identification labels in the image data and acquires position information in the image data of the detected identification labels as label position information; an item recognition unit that performs image analysis of the image data to detect items in the image data and acquires position information in the image data of the detected items as item position information; and a matching unit that matches items with identification labels based on the label position information and the item position information acquired from the same image data. [Effects of the Invention]

[0019] According to an aspect of the present invention, each item is associated with an identification label based on the item's position information and the identification label's position information in a single image data. Therefore, even in a situation where multiple items are continuously transported and the orientations of the identification labels change or become different from each other during transport, it is possible to more accurately associate the items with the identification labels affixed to the items.

[0020] As a result, according to this aspect of the present invention, it is possible to improve the tracking accuracy of each of the continuously conveyed articles. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of conveyance of a steel pipe (article) with an identification label according to an embodiment of the present invention. [Figure 2] FIG. 10 is a diagram showing an example of a steel pipe with identification labels attached to the outer surface at two locations around the circumference. [Figure 3] 1 is a diagram illustrating a configuration of an article management device according to an embodiment of the present invention. [Figure 4] FIG. 2 is a diagram illustrating an example of the configuration of a management processing unit main body. [Figure 5] FIG. 10 is a diagram showing an example of a processing flow of the management processing unit main body. [Figure 6] 10A to 10C are diagrams illustrating examples of changes in the content of frame images when the same image area is captured. DETAILED DESCRIPTION OF THE INVENTION

[0022] Next, an embodiment of the present invention will be described with reference to the drawings. In this embodiment, a steel pipe is used as an example of the article to be transported. However, the article to which the present invention is applicable is not limited to a steel pipe. The present invention can also be applied to articles such as a steel plate.

[0023] The present invention is a technology suitable for tracking transported objects in a situation where the orientation of an identification label attached to the object may change during transport, such as when the object is transported while rolling or while being subjected to vibrations during transport, such as in a vibration transport system.

[0024] Furthermore, even if there is no risk that the orientation of the identification label will change during transport, the present invention is also suitable for cases where the orientation of each item during transport is random and the orientation of the identification label of each item may differ from each other. A situation in which the orientations of items being transported differ from each other may occur, for example, during transport at a baggage collection and distribution center or during transport of items such as baggage at an airport. In the following description, the steel pipe will also be referred to as an article.

[0025] (Configuration of the item management device) The item management device of this embodiment is a device that tracks and manages each item when multiple items are transported consecutively. Specifically, the items are managed by determining the order in which the items are transported using identification labels and the number of items transported by image analysis of image data. The steel pipe production line of this embodiment includes a steel pipe forming process, an inspection process, an adjustment process, etc. The inspection process and the adjustment process may be performed two or more times depending on the use of the steel pipe, etc.

[0026] In a steel pipe production line, for example, after forming the steel pipe into a desired shape, the pipe undergoes adjustment processes such as heat treatment, inspection processes such as non-destructive testing and water pressure testing, correction processes such as repairing scratches, and final processes such as printing and packaging before being shipped. The adjustment and inspection processes vary depending on the steel pipe's use. For this reason, not all processes are performed in a consistent line configuration, and each process is located in a different building. Therefore, goods are moved between each process using transportation means such as trucks. Therefore, the order of the steel pipes tends to be random each time they are transported.

[0027] <Identification label 3> The identification label 3 will now be described. The identification label 3 has information such as symbols and numbers for identifying each and every item. One piece of information included in the identification label 3 is an individual identification number for individual identification. The information on the identification label 3 may include at least one piece of information specific to the item, such as the shipping destination and delivery date of the item.

[0028] The information contained in the identification label 3 is configured, for example, as a one-dimensional barcode, a two-dimensional barcode, a data matrix (data code), etc. In this example, the label information is configured as a data matrix.

[0029] An example of a method for applying the identification label 3 to the surface of the item 2 is to print label information on a sticker or tag using a label printer or the like, and then attach the sticker or tag to the item 2. However, label application is not limited to pasting a sticker or the like. The application method may also be a method in which an individual identification number as label information is applied directly to the surface of the item 2. In this case, the label is applied, for example, by automatically or hand-printing or by engraving with paint. The application of the identification label 3 is preferably performed by printing directly on the item 2 itself, taking into account the risk of the label falling off during transportation.

[0030] Furthermore, it is sufficient that one or more identification labels 3 are attached to one article 2. To improve the accuracy of label detection, it is preferable to attach the same identification labels 3 to multiple locations on the outer surface of the article 2. When attaching multiple identification labels 3 to one article 2, it is preferable to attach the identification labels 3 to surfaces facing different directions so that any one label is easily visible during transportation, as shown in Figure 2. For example, multiple identification labels 3 are attached along the rolling direction of the article 2 during transportation (the circumferential direction of the steel pipe).

[0031] <Conveyor line overview> In this embodiment, it is assumed that after a steel pipe 2 (a part to be managed) is molded, an identification label 3 containing information on an individual identification number is attached to the steel pipe 2. Furthermore, it is assumed that the steel pipes 2 to which the identification label 3 has been attached are transported to the next processing step in bundles of several pipes.

[0032] In this case, a bundle number is assigned to each bundle. The individual identification numbers of the steel pipes 2 contained in the bundle are transmitted, for example, from an automatic printer that applies the identification labels 3 when the labels are applied. This allows the steel pipes to be managed in association with the bundle number. This information is also stored in a data management server.

[0033] Then, suppose that the bundle of steel pipes 2 transported by a crane or other means is unloaded upstream on a conveyance line that manages goods, as shown in Figure 1. When the bundle is unloaded, the bundle number is read, and a list of individual identification numbers included in the current bundle number (list of steel pipes 2) is obtained from the data management server.

[0034] That is, in this embodiment, as shown in FIG. 1, a group of multiple items 2 from the same lot is unloaded upstream of the conveyor line (on the right side of FIG. 1). Then, consider a case where the items 2 are conveyed downstream without the order of each item being clear. In FIG. 1, identification labels 3 are represented by "☆". Furthermore, the conveyor 1 that conveys the items 2 at this time may be a conveyor using a device such as a belt conveyor, or may be a device that conveys the items by their own weight while rolling on rails.

[0035] Here, in FIG. 1, reference numeral 4 denotes an imaging unit, and reference numeral 5 is a conceptual diagram showing an example of an image of image data captured by the imaging unit 4.

[0036] <Configuration of the article management device> As shown in FIG. 3, the article management device of this embodiment includes an imaging unit 4 that continuously captures images of articles 2 being transported, an article tracking processing unit 6, and a data management server 12.

[0037] The item tracking processing unit 6 is a processing unit that performs tracking processing of the item 2 based on image data continuously captured by the imaging unit 4. As shown in Fig. 3, the item tracking processing unit 6 of this embodiment includes a video communication unit 8 that communicates with the imaging unit 4, an information communication unit 7 that communicates with the data management server 12, a management processing unit main body 11 that processes data obtained from the video communication unit 8 and the information communication unit 7, an input unit 9, and an output unit 10.

[0038] The communication interface between the imaging unit 4, video communication unit 8, information communication unit 7, and data management server 12 may be, for example, a LAN cable. Preferably, a standard that allows faster communication should be used. Wireless communication may also be used. The video communication unit 8 and the information communication unit 7 do not necessarily need to be separate. They may both be provided as a video and information communication unit that has the functions of 7 and 8. The information communication unit 7 may also have the function of communicating with other external communication devices, for example, information processing devices such as PLCs, and controlling the operation of the equipment.

[0039] The input unit 9 may include an input device that accepts input from a user. The input device may include, for example, a keyboard or physical keys. The input device may also include a touch panel, a touch sensor, or a pointing device such as a mouse. The input device is not limited to these examples and may include various other devices.

[0040] The output unit 10 has a function of visualizing the information obtained from the management processing unit main body 11 to the user through a display device. In addition, the information obtained from the output unit 10 is managed by, for example, storing it in a storage device built into or external to the video processing device, or communicating it to a data management server 12 through the information communication unit 7.

[0041] The data management server 12 is hardware that stores the individual identification number information of the original item 2, which is used as a reference for comparing the results of reading the identification label 3. The data management server 12 only needs to have a database format that makes it easy to retrieve and store data. The data management server 12 may also be a cloud-based server.

[0042] The information stored in the data management server 12 is not limited to the individual identification number of the item 2. In addition, the information may include at least one of the following information on the manufacturing history of the item 2, such as the dimensions of the item 2 and inspection results.

[0043] The arithmetic processing of the management processing unit main body 11 is performed by executing a program. To execute the program in real time, a PC (computer) having specifications such as a CPU, GPU, RAM, and SSD may be used. Also, a storage medium device for recording image streaming (video) may be provided as necessary.

[0044] <Management processing unit main body 11> The management processing unit main body 11 of this embodiment uses a series of image data of an item 2 with an identification label 3 attached thereto during transport to perform item tracking processing. Then, it detects the identification label 3 and the object, and performs identification label reading and item tracking processing. Then, it obtains item management information from the position information of the detected identification label 3, position information of the item 2, read information of the identification label 3, and tracking information of the item 2 between the image data. Specifically, it obtains the transport order using the identification label 3 of the item 2 during transport, the reading of the label information of the identification label 3, and a count of the number of items transported as item management information.

[0045] The item management device of this embodiment includes an imaging unit 4 and the above-described management processing unit main body 11. As shown in Fig. 4, the item management device of this embodiment includes the imaging unit 4, an image processing unit 1B, a correlation unit 11C, an item ID adjustment unit 11D, a conveyance order assignment unit 11E, and an item number calculation unit 11F. It also performs error processing. The image processing unit 11B includes a label recognition unit 11Ba, an item recognition unit 11Bb, and an item tracking unit 11Bc.

[0046] <Imaging unit 4> The imaging unit 4 captures an imaging area, which is a position where the transported article 2 passes, and acquires image data of the article 2 during transport. Specifically, the imaging unit 4 continuously captures images of the imaging area to continuously acquire a series of image data (video). The series of image data is configured as an image streaming consisting of a plurality of image frames.

[0047] The imaging unit 4 is composed of a video capturing device such as a camera. Specifically, the imaging unit 4 may be a general camera such as a GigE camera or a USB camera. Multiple cameras may be connected to one video processing device.

[0048] <Imaging conditions> The imaging unit 4 may be positioned so that it can capture an image of the identification label 3. Preferably, it is installed in a position that allows it to face the identification label 3 facing in a certain direction, such as upward. For example, the imaging unit 4 is installed in a position that allows it to capture the steel pipe 2 being transported from directly above.

[0049] At this time, the field of view (imaging area, angle of view) of the imaging unit 4 is set to a size that allows multiple steel pipes 2 to be imaged. Preferably, the imaging area is set so that a field of view width of at least one rotation (circumferential length) of the steel pipe 2 that rolls during transportation is ensured. The upper limit value is, for example, two rotations (circumferential length) of the steel pipe 2. The imaging resolution, working distance, and lens focal length are then adjusted accordingly. The field of view width for one rotation of the steel pipe 2 is, for example, the circumferential length of the steel pipe 2 with the largest diameter, or that length plus a margin. This makes it possible to recognize the identification label 3 attached to each steel pipe 2 in any of the consecutively captured images.

[0050] The resolution of the imaging unit 4 is selected so that the identification label 3 to be read can be recognized. Then, the FPS according to the conveying speed of the article 2 and the shutter speed that can capture the identification label 3 while suppressing blurring can be selected. The imaging unit 4 may have automatic exposure time adjustment and automatic focus adjustment functions. However, these functions should be set not to be used. In addition, it may be equipped with general camera functions. It may also be configured with lighting equipment to suppress external disturbance factors.

[0051] In addition, the shutter speed for capturing images is adjusted to match the rotation speed of the transported steel pipe 2. If adjusting the shutter speed results in insufficient light, the amount of light can be ensured by adjusting the aperture or installing lighting. Additionally, if focus adjustment is required due to changes in the outer diameter of the steel pipe 2, a mechanism for moving the imaging unit 4 up and down can be provided, or a system can be used that allows remote focus adjustment using a liquid lens.

[0052] Here, the steel pipes 2 that have been broken off upstream on the conveying line roll one by one under their own weight and pass through the imaging area imaged by the imaging unit 4. The imaging unit 4 continuously captures images of this situation at a predetermined shutter cycle.

[0053] The decision to start imaging is made by a start signal (external), such as an ON / OFF signal for the equipment or an output from a position detection device using a laser. The decision to start imaging is also made by a start signal (internal), which is an imaging start signal input from a dedicated user interface screen. The decision to end imaging can also be made by an end signal (external) or end signal (internal).

[0054] <Video processing unit 11A> The image data captured by the imaging unit 4 is sequentially supplied to the image processing unit 11A. The image processing unit 11A supplies the image data to the image processing unit. In this embodiment, the video processing unit 11A acquires video data from the imaging unit 4 in real time after the start of imaging. At this time, the video processing unit 11A divides the acquired image streaming into multiple image frames at an FPS according to the specifications of the imaging unit 4 (video camera) to generate consecutive frame images (image data). Each piece of image data is also called a frame image.

[0055] The video processing unit 11A performs image processing such as binarization on each frame image to improve the reading accuracy of the identification label 3. In addition to binarization, image processing can include general image processing such as brightness adjustment, color adjustment, grayscale conversion, and image manipulation such as rotation and cropping, etc. Furthermore, if the imaging unit 4 is not positioned directly facing the identification label 3, keystone correction processing can be performed.

[0056] The divided frame images (after image processing) are organized in chronological order and stored in a memory or a storage unit such as an SSD, HDD, etc. Then, the video processing unit 11A sequentially supplies the frame images to the image processing unit 11B.

[0057] The image processing unit 11B, the association unit 11C, the item ID adjustment unit 11D, the conveyance sequence assignment unit 11E, and the item number calculation unit 11F described below execute processing for each piece of image data.

[0058] <Image processing unit 11B> The image processing unit 11B includes a label recognition unit 11Ba, an item recognition unit 11Bb, and an item tracking unit 11Bc.

[0059] <Label recognition unit 11Ba> The label recognition unit 11Ba performs image analysis on the image data to detect the identification labels 3 in the image data. Then, the detected identification labels 3 are analyzed to obtain label information of the identification labels 3, and position information of the detected identification labels 3 in the image data (in the imaging area) is obtained as label position information.

[0060] The recognition of the identification labels 3 in the image and the acquisition of position information may be performed using a known image processing method. For example, a learning model for detecting the identification labels 3 is prepared by machine learning using image data showing the identification labels 3. The image data is then input into the learning model to recognize the identification labels 3.

[0061] Furthermore, label information is obtained by reading the recognized identification label 3. At this time, if the recognized identification label 3 cannot be read, an error flag may be added as read information to indicate that the identification label 3 is unreadable.

[0062] Furthermore, the label recognition unit 11Ba acquires position information in the image of the recognized identification label 3. For example, the center position of the recognized identification label 3 is acquired as the position information of the identification label 3.

[0063] The label recognition unit 11Ba of this embodiment detects the identification label 3 in the frame image received from the management processing unit main body 11, and acquires position information of the identification label 3. The acquired position information is stored as label position information on a frame-by-frame basis. This label position information includes the position coordinates of the detected identification label 3. Furthermore, an identification label 3 recognition process is performed on the detected identification label 3, and label information is acquired.

[0064] The position information of the identification label 3 can be obtained by a method using image processing such as edge detection, etc. The position information of the identification label 3 can be obtained by applying a known image processing method.

[0065] Furthermore, the recognition process for the identification label 3 can employ a known acquisition method. For example, if the data matrix of the identification label 3 is a barcode, one method is to use the barcode detection function of the Python dedicated library pylibdmtx, zxing, or opencv. If the data matrix is ​​a two-dimensional barcode, one method is to employ the two-dimensional barcode detection function of the Python dedicated library pyzbar or opencv. Furthermore, a commercially available handheld terminal or barcode reader can be used to recognize the identification label 3.

[0066] Next, analysis is performed by using the above Python library in accordance with the format of the identification label 3 to analyze the individual identification number information in the identification label 3. The analysis results are then stored as the results of reading the identification label, and are then stored together with the position information as the results of reading the identification label.

[0067] If the read result is an unreadable character code such as a binary code, it is converted using a character code conversion function. Here, if the individual identification number is printed directly on the surface of the item 2 as the identification label 3, the following method may be used. That is, instead of the code reading library described above, it is recommended to use a Python library for AIOCR such as TesseractOCR or PaddleOCR. Alternatively, commercially available AIOCR software may be used. In this example, since the attached identification label 3 is a data matrix, pylibmtx is used to detect the position information and read the identification label.

[0068] Furthermore, the label recognition unit 11Ba of this embodiment determines whether the identification label reading result is included in the individual identification numbers obtained from the data management server 12. If the identification label reading result is not included in the obtained individual identification numbers, an error flag is set and a state is created in which a determination can be made using the identification label reading result correction function described below. This error processing may be performed separately.

[0069] Through the above processing, the label recognition unit 11Ba performs the following for all identification labels 3 in one frame image: That is, data linking the position coordinates of the identification label 3 with the reading result of the identification label 3 is obtained for each frame as the analysis result of the identification label 3. The label recognition unit 11Ba executes this processing sequentially for each frame image that is sent sequentially.

[0070] <Article recognition section 11Bb> The image data is the same as that of the label recognition unit 11Ba, and the image data is subjected to image analysis to detect the item 2 in the image data. Then, position information of the detected item 2 in the image data (in the imaging area) is acquired as position information of the item 2.

[0071] The recognition of the article 2 in the image and the acquisition of its position information may be performed using a known image processing method. For example, a learning model for article detection is prepared using machine learning of image data showing the article 2. The image data is then input into the learning model to recognize the article 2. The article recognition unit 11Bb also acquires position information of the recognized steel pipe 2 in the image. For example, the center position of the recognized image is acquired as the position information of the identification label 3.

[0072] Furthermore, the item recognition unit 11Bb sets an item ID for tracking for each recognized item 2, and links the recognized item 2 to the item ID. Here, multiple items 2 may be recognized in one image data. In this case, an item ID is set for each item.

[0073] Here, the item ID is set for an item recognized from image data and is a unique ID for identifying each item being transported. Note that the same item may be captured in two or more image data, but in this example, adjustments are made so that the same item ID is assigned. This item ID is set to track the items being transported and is used to manage the order in which the items are transported, the number of items transported, etc. This item ID may also be set for an item recognized before the identification label 3 is detected (before the item is associated with the identification label 3).

[0074] The article recognition unit 11Bb of this embodiment detects the steel pipe 2 in the acquired frame image using a known object detection program, and then acquires article information about the detected steel pipe 2, including the detection reliability and coordinate information of the bounding box.

[0075] The object detection program uses a pre-trained model using pre-installed AI. The algorithm used can be an object detection algorithm published on Github using a popular framework such as Keras or Pytorch. In this example, from the perspective of detection accuracy and speed, YOLO, which uses the Pytorch framework, was used as the object detection program. The model and weight files used here can be any currently published on Github. However, from the perspective of detection performance for object 2, a newly generated weight file is used by additionally training image data annotated with object 2 from actual video. Due to YOLO specifications, the annotation file is saved in text format, describing the position of object 2 within the image data.

[0076] Next, the coordinates of the center position of the steel pipe 2 are obtained from the coordinate information of the bounding box detected by the object detection program. Based on the acquired central position information, a unique item ID is assigned to the detected steel pipe 2. The item ID is then managed for each steel pipe 2 together with the position coordinates of the steel pipe 2 as analysis information of the item 2.

[0077] In addition, the process of detecting the item 2 in the frame image and recognizing whether the detected item 2 is an identification label 3 or a steel pipe 2 may be configured as a processing unit common to the label recognition unit 11Ba and the item recognition unit 11Bb.

[0078] <Item Tracking Unit 11Bc> The item tracking unit 11Bc executes a process of tracking the position of the item 2 between successive frame images. That is, the item tracking unit 11Bc executes a process of determining whether the item 2 recognized in one image data matches any of the items 2 in the image data prior to the one image data.

[0079] The object tracking process may use a known tracking technique such as MOT (Multiple Object Tracking). Note that the item tracking unit 11Bc may perform the object detection process in the image in the item recognition unit 11Bb. MOT is a technique that continuously assigns a unique ID to the same object (multiple objects) in consecutive frames of a video, and tracks the position of the object between images. MOT detects the position of an object for each image, and if the objects detected between consecutive images are recognized as the same object, it starts tracking.

[0080] In this embodiment, a tracking program using deep learning such as MOT determines whether a steel pipe 2 detected from a frame image currently being processed is a steel pipe 2 that has already been detected in the previous frame. When a tracking program using deep learning such as MOT is used as the processing of the item tracking unit 11Bc, the item detection processing in each frame image corresponds to the object detection processing of the item recognition unit 11Bb.

[0081] In the tracking program constituting the item tracking unit 11Bc of this embodiment, once an item 2 enters the imaging area (frame-in) and is assigned an item ID, it is tracked using the unique item ID until it leaves the imaging area (frame-out).

[0082] The tracking program can be used in conjunction with an algorithm that uses open source software such as BoT-SORT or ByteTrack, which are available on Github. In this embodiment, the following procedure is performed for all bounding boxes of N objects detected in the frame. Specifically, N predicted bounding boxes are calculated, indicating the location of each object in the next frame, using their coordinates, aspect ratio, velocity, and acceleration. After the calculation, N × N IoU combinations are calculated for the N bounding boxes of the frame and the N predicted bounding boxes. A process is added to link the bounding boxes of the frame and the predicted bounding boxes so that the sum of these N × N IoU combinations is maximized. This allows the same item ID to be assigned across different frames. In this way, data in which an item ID and location information are assigned to each steel pipe 2 can be obtained as item tracking results for each frame. In addition, when transporting steel pipes 2, the order of transport of the steel pipes 2 during transport is not changed.

[0083] <Association section 11C> The association unit 11C associates the item 2 with the identification label 3 in the same frame image. It compares the label position information in the same image data with the position information of the item 2. If there is label position information and item position information that are similar in position, it determines that the identification label 3 corresponding to that label position information is attached to the item 2 corresponding to that item position information. Here, for an article 2 whose identification label 3 cannot be recognized, there is no label position information to be associated with it in the current image frame.

[0084] Then, when it is determined that an identification label 3 corresponding to the target item 2 exists in the same image data, the associating unit 11C associates the information of the corresponding identification label 3 with the item ID of the corresponding item 2.

[0085] The information on the corresponding identification label 3 is, for example, label information that can be read from the corresponding identification label 3. Alternatively, an identification label 3 ID that identifies the identification label 3 may be generated, and the identification label 3 ID may be used as the information on the corresponding identification label 3. In this case, the relationship between the identification label 3 ID and the label information that can be read from the identification label 3 may be stored in the server.

[0086] The association unit 11C of this embodiment collates the analysis results of the identification labels 3 obtained from the label recognition unit 11Ba and the item tracking unit 11Bc with the location information data of the item tracking results.

[0087] Then, if there is a combination of the position coordinates of the identification label 3 and the position coordinates of the steel pipe 2 where the difference between the position coordinates is within a threshold, the combination is deemed to have been matched. For the combinations for which matching has been successful, a process is performed to link the item ID and position information in the item tracking result with the individual identification number information, which is the analysis result of the identification label 3, to obtain a matched item tracking result.

[0088] Once an item ID is linked to an individual identification number (label information), it is managed in the storage unit. Then, in the processing of the next frame onwards, processing is added so that the item ID that has already been linked to an individual identification number is not subjected to location information matching.

[0089] Here, the threshold value for the position coordinates may be set to any fixed value. Furthermore, using the dimensional information of the item 2 acquired from the data management server 12, the threshold value is set to the coordinate value in the conveying direction of the calculated center coordinates obtained from the position information of the item 2 ± the item 2 dimension / 2. If the coordinate value in the conveying direction of the position coordinates of the identification label 3 falls within this threshold value, it may be determined that matching is successful. In other words, it is preferable to determine that matching is successful when the difference in the position in the conveying direction among the position information is within a predetermined value of the threshold value.

[0090] <Article ID adjustment section 11D> The item ID adjustment unit 11D executes the process based on the processing of the item tracking unit 11Bc. Specifically, it determines whether the item 2 detected in the current frame image is the same item as an item that existed in another frame image data that is the frame image data prior to the current frame image data. If it is determined that they are the same item, it sets the same item ID for the item 2 detected in the current frame image and for the item 2 that exists in the other frame image data and that it is determined to be the same item.

[0091] Specifically, if it is determined that the item 2 recognized in the current image data is the same as the item 2 recognized in the immediately previous image data, the item recognition unit 11Bb discards the item ID set for the item 2 recognized in the current image data, and then changes the item ID of the item 2 recognized in the current image data to the item ID set for the same item 2 recognized in the immediately previous image data.

[0092] The above-mentioned item tracking unit 11Bc uses deep learning to track the movement of the item 2 between successive image data and determine whether the item 2 between the image data is the same or different. When the sampling time for acquiring image data is short, the amount of movement of the item 2 between the image data is almost fixed. Also, in this embodiment, the transport order of the item 2 is not changed. From this perspective, it may be determined whether the item 2 between the image data is the same or not based on the position information of the item 2 in the current image data and the position information of the item 2 in the previous image data.

[0093] In the object detection program of this embodiment, when a steel pipe 2 is detected in the current frame image, the item ID adjustment unit 11D determines whether the steel pipe 2 is already detected in the previous frame in chronological order. If the steel pipe 2 was detected in the previous frame, the item ID assigned this time is discarded, and the current item ID is changed to the item ID of the same item 2 that existed in the previous frame. On the other hand, if the steel pipe 2 was not detected as a steel pipe 2 that was already detected in the previous frame, the assigned item ID is used as is. In this way, processing is performed such that one unique item ID is assigned to one item 2.

[0094] <Transportation order assignment unit 11E> The transport order assigning unit 11E determines whether the item 2 detected in one image data matches any of the items 2 in the other image data that is the image data immediately before the one image data. Then, based on the processing of the item tracking unit 11Bc, transport order information is assigned to the item IDs of the items 2 that exist in the other image data but do not exist in the one image data.

[0095] Specifically, for an item 2 that existed in the previous image data and has been framed out in the current image data, information on the conveyance order is added to the item ID of that item 2. For example, a counter for setting the order is provided, and the counter is incremented and added each time an item 2 to which conveyance order information is to be added is detected.

[0096] In addition, if two items 2 are framed out of the imaging area at the same time, the order of arrangement in the conveying direction can be determined from the item ID and the position information in the previous image data, and a relatively smaller number can be assigned to the item 2 downstream in the conveying direction.

[0097] In the transport sequence assigning unit 11E of this embodiment, when an item ID is no longer detected as a result of item tracking by the item tracking unit 11Bc, the transport sequence assigning unit 11E acquires in real time the individual identification numbers linked to the item IDs that are no longer detected. The acquired individual identification numbers are acquired as a list of ordered individual identification numbers arranged in chronological order after imaging is completed.

[0098] <Number of items calculation section 11F> The item number calculation unit 11F calculates the number of conveyed items based on the number of item IDs that have been set. The article number calculation unit 11F may have a counter for calculation, and may calculate the number of conveyed articles by incrementing the counter each time an article 2 goes out of the frame of the imaging area.

[0099] The item number calculation unit 11F of this embodiment accumulates the number of item IDs assigned from the start of imaging to the end of imaging, and stops the accumulated value as the number of transported steel pipes 2. It may also be an accumulation of the analysis results of the identification labels 3 or the number of individual identification numbers, excluding duplicates. When the imaging is completed, this cumulative result is obtained as the total number of items 2 transported.

[0100] <Error handling> Here, by increasing the imaging area in the conveyance direction, the identification label 3 of each article 2 can usually be recognized in any of the image data. However, if the identification label 3 can be recognized but cannot be read, the label information will be flagged as an error. In this case, the identification label 3 may be defective, so the worker will check and reattach the correct identification label 3.

[0101] For example, it checks whether an error flag has been assigned as label information linked to each item ID. The individual identification number corresponding to the flagged identification label 3 is not included in the individual identification numbers obtained from the data management server 12. This means that the analysis result of the identification label 3 may be incorrect. In this case, the operator corrects the identification label 3 according to the individual identification number obtained from the data management server 12. Alternatively, the operator identifies the actual item from the acquired item order, reads the identification label 3 on the front side with a commercially available handheld barcode reader or handheld QR code reader to determine the correct individual identification number, and then corrects the identification label 3 from the user interface screen of the video processing device. After that, a list of the corrected, ordered individual identification numbers is obtained.

[0102] The total number of items is also checked against the total number of individual identification numbers obtained from the data management server 12, and if they match, a list of the corrected, ordered individual identification numbers is sent to the data management server 12. After that, each management processing unit main body 11 initializes the data remaining in its memory and waits for the next imaging start decision. If they do not match, the number of actual items is checked and corrected again from the user interface screen of the video processing device.

[0103] <Final Processing> The above process is carried out until the target article 2 passes through the imaging area. This allows the conveyance order of the articles 2 to be known even if the articles 2 are conveyed while rolling, which was previously unknown before the conveyance.

[0104] Furthermore, by linking label information to the item ID corresponding to each item 2, it is also possible to acquire the label information attached to each item 2. This makes it possible to accurately determine, for example, the processing content and transport destination in the next process corresponding to each item 2. It also becomes possible to determine the number of items transported and the number of items per lot.

[0105] (Example of processing by an item management device) Next, an example of a processing flow in the article management device will be described with reference to FIG. First, in step f0, the management processing unit main body 11 acquires from the data management server 12 the individual identification number of each item 2 included in the lot to be transported this time.

[0106] Specifically, the lot information of the item 2 is registered via the input unit 9, and the individual identification number included in the lot is obtained from the data management server 12. The registration method may be manual registration using an input device. Also, if an identification label 3 for identifying the lot is attached, the identification label 3 for the lot may be read by a reader.

[0107] Next, in step f1, the management processing unit main body 11 acquires video data from the imaging unit 4 capturing images of the situation during the transport of the item 2. At this time, the acquired video data may be streaming video, a video file saved in a general video file format such as mp4 or mov, or a video file reconstructed using the acquired images. If necessary, measures such as compression may be added to improve communication speed.

[0108] That is, in this example, each piece of image data is acquired in the form of a moving image file from the imaging unit 4. Each piece of image data is also referred to as a frame.

[0109] Next, in step f2, the management processing unit main body 11 divides the acquired video data into frames in accordance with the FPS of the imaging unit 4. It is also possible to add processing to thin out frames in order to reduce the load on the management processing unit main body 11. However, in that case, the number of frames containing identification labels 3 in the video data will decrease, which may worsen the reading accuracy of the identification labels 3, so care must be taken when performing the thinning processing.

[0110] Next, in step f3, the management processing unit main body 11 performs binarization processing on each frame acquired by dividing the image into frames. The binarization threshold for this processing can be set to a threshold that allows separation of the article 2 and the identification label 3, and then image processing can be performed using this threshold. Other image processing methods include brightness adjustment, color adjustment, grayscale conversion, and other general image processing such as RGB and HSV value adjustment, rotation, and cropping. Processing aimed at improving the reading accuracy of the identification label 3, such as a keystone correction function, may also be added.

[0111] Next, in steps f4 and f6, the management processing unit 11 inputs the same frame to the label recognition unit 11Ba and the object tracking unit. In this case, the management processing unit 11 may be configured to process each process in parallel in order to improve the processing speed. In this processing example, the object tracking unit also performs the processing of the object recognition unit.

[0112] In step f5, the management processing unit main body 11 analyzes the identification label 3 in the frame image using the label recognition unit 11Ba to detect the identification label 3. Then, the content of the identification label 3 and the position coordinates of the identification label 3 are obtained. At this time, the label recognition unit 11Ba may recognize the identification label 3 using a publicly available code analysis function. Furthermore, an analysis method may be selected according to the form of the identification label 3. The position coordinates of the identification label 3 may be, for example, information on the center position of the identification label 3 in the frame. Two or more position coordinates may be obtained for one identification label 3. The identification label 3 may be detected using a learning model for object recognition generated by known machine learning.

[0113] Furthermore, in step f7, the management processing unit main body 11 performs processing in which the object tracking unit detects the articles 2 in the frame image, assigns a unique article ID to each article 2, and acquires the position coordinates of the articles 2. At this time, the object tracking unit may use a method that uses a contour extraction process such as Sobel processing or Canny processing, or a method that uses an AI object detection model, but preferably uses a trained AI model that has undergone machine learning of the image of the article 2 to be actually tracked.

[0114] Next, in step f8, the management processing unit main body 11 performs a matching determination in the association unit 11C to determine whether the relationship between the position coordinates of the identification label 3 and the position coordinates of the item 2 obtained from the label recognition unit 11Ba and the object tracking unit, respectively, falls within a preset threshold.

[0115] At this time, it is not necessary to perform the matching process (f8) again for the item ID to which the identification label 3 is already linked.

[0116] If it is determined that there is an identification label 3 having position coordinates similar to the position coordinates of the detected item 2, the process proceeds to step f9. Then, in step f9, the association unit 11C associates the reading result (label information) of the identification label 3 with the item ID of the corresponding item 2. Then, the process proceeds to step f10.

[0117] On the other hand, if it is determined that there is no identification label 3 with position coordinates similar to those of the detected item 2, the reading result of the identification label 3 is not linked to the item ID of the item 2. Then, the process proceeds to step f10. Note that label information without an identification label 3 may be assigned.

[0118] In step f10, the management processing unit main body 11 compares the item ID of the item 2 detected in the current frame with the item ID of the item 2 detected in the previous frame. If the comparison reveals that there is an item ID that is included in the previous frame but not in the current frame, the process proceeds to step f11; if there is no such item ID, the process proceeds to step f12.

[0119] Here, if an item 2 that is determined to be the same as the item 2 detected in the current frame was present in the previous frame, the item ID of the item 2 detected in the current frame is changed to the item ID set for the same item 2 detected in the previous frame. Along with the change, the information linked to the current item ID is also changed.

[0120] In step f11, a transport order is assigned to the item IDs that are included in the previous frame but not included in the current frame as a result of the comparison. The transport order to be assigned is not particularly limited as long as it is a symbol that can identify the transport order.

[0121] In step f12, the management processing unit main body 11 saves the contents acquired in steps f9 and f11 as the results of the frame. At this time, the results may be saved locally. Preferably, the results are only held in memory to improve processing speed.

[0122] Next, in step f13, the management processing unit main body 11 completes the processing for one frame, and if there are remaining frames, proceeds to step f3. Then, the management processing unit main body 11 repeats the above processing for the remaining frames. On the other hand, if it determines that the processing for all frames has been completed, proceeds to step f14.

[0123] In step f14, the management processing unit main body 11 obtains the transport order of each item 2 from the item IDs linked to the transport order and the identification label reading results obtained after all frames have been processed, and also obtains the number of transported items 2. The number of transported items 2 can be obtained, for example, by counting the number of item IDs.

[0124] Next, in step f15, the management processing unit main body 11 compares the individual identification number obtained from the reading result of the identification label 3 with the original individual identification number obtained in step f0. If the comparison result is not valid, the process proceeds to step f16, where the identification label reading result is corrected.

[0125] In step f16, the management processing unit main body 11 transmits the corrected ordered individual identification number to the data management server 12 and performs data management. At this time, the result may be saved as a backup in a local folder of the video processing device itself.

[0126] (Inventory Management Program) As described above, the processing of the management processing unit main body 11 (see FIG. 4) of this embodiment is realized by a computer executing a predetermined program. Therefore, the item management program of the present invention can be specified to cause a computer to function as an image processing unit 11B, a matching unit 11C, an item ID adjustment unit 11D, a conveying order assignment unit 11E, and an item number calculation unit 11F.

[0127] (Steel pipe manufacturing method) An item management device is applied to a steel pipe production line. As a result, even if the order of transport of steel pipes is random due to the transfer of multiple steel pipes in lots, the item management device of this embodiment can more reliably obtain the order of transported items and information on the identification labels 3 attached to the items. This makes it possible to manufacture steel pipes with greater precision without making mistakes in the processing of each item.

[0128] (Operation etc.) In this embodiment, the next item management process can be performed even in a situation where multiple items 2 are continuously transported and there are times when the identification labels 3 are not always visible. That is, by using information obtained from captured image data, it is possible to read the identification labels of the items 2, count the number of items, and obtain the order of items, while minimizing costs and worker burden.

[0129] Furthermore, unlike the case where a handheld terminal is used, this embodiment does not require human intervention, which makes it possible to reduce labor costs and improve work efficiency. Furthermore, in cases where tracking of each item2 has not been possible in the past due to work efficiency concerns, the ability to track each item will improve operational and quality control levels, leading to improved manufacturing precision.

[0130] An example of the processing of this embodiment will now be described with reference to FIG. For example, as shown in FIG. 6(a), assume that four conveyed steel pipes A to D enter the imaging area in this order. It is assumed that frames 1 to 3 shown in FIG. 6 are captured as a series of three image data (image frames). In the very first frame 1, four steel pipes A to D are imaged, and the article recognition unit 11Bb acquires the positional information of the four steel pipes. Meanwhile, the label recognition unit 11Ba detects only the identification labels 3 of steel pipes B and D out of the four steel pipes A to D, acquires only the coordinates of those labels, and makes it possible to read the information on those identification labels 3.

[0131] At this time, a unique item ID is set for each of the four steel pipes A to D. However, based on information from the item tracking unit 11Bc, for steel pipes that existed in the previous frame, the item ID set for the steel pipe that existed in the frame before that is set. In this example, the four steel pipes A to D are imaged for the first time in frame 1, and item IDs id:1 to id:4 are set.

[0132] 6(a), X(B) and Y(B) are similar as coordinates in the conveying direction. Also, X(D) and Y(D) are similar. Therefore, the association unit 11C recognizes label Q(B) as the identification label 3 for steel pipe B, and label Q(D) as the identification label 3 for steel pipe D.

[0133] Next, frame 2 is image data obtained by capturing an image of the same imaging area Δt seconds after frame 1 is captured. In frame 2, four steel pipes A to D are also imaged, and the item recognition unit 11Bb acquires the position information of the four steel pipes A to D. On the other hand, the label recognition unit 11Ba detects only the identification labels 3 of steel pipes A and C out of the four steel pipes A to D, and can acquire only the coordinates of those labels.

[0134] Here, the steel pipes A to D move in the conveying direction between the images of frame 1 and frame 2, but the item tracking unit 11Bc tracks the steel pipes between the image data (between frames). Then, through the processing of the item ID adjustment unit 11D, since the steel pipes A to D are steel pipes that existed in frame 1, the item IDs that were temporarily set in frame 2 are changed to id:1 to id:4, which are the item IDs that were set in frame 1.

[0135] Furthermore, in frame 3, steel pipe A is framed out, and four steel pipes B to D are imaged, and the item recognition unit 11Bb acquires the position information of the three steel pipes. Meanwhile, the label recognition unit 11Ba detects only the identification labels 3 of steel pipes B and D out of the four steel pipes, and acquires only the coordinates of those labels. In frame 3, steel pipe A is framed out, so the transport order assigning unit 11E assigns a transport order to the item ID of that steel pipe.

[0136] As described above, in this embodiment, even if the detection order of the identification labels 3 differs from the detection order (transport order) of the steel pipes, the steel pipes are reliably associated with the identification labels 3. Furthermore, the label information of the steel pipes can be more reliably acquired.

[0137] (others) The present disclosure may also have the following configuration. (1) Disclosure 1 is an item management device that tracks each item when transporting multiple items with identification labels attached, an imaging unit that acquires image data of an imaging area that is a position where the conveyed article passes; a label recognition unit that performs image analysis on the image data to detect an identification label in the image data and obtains position information of the detected identification label in the image data as label position information; an item recognition unit that performs image analysis on the image data to detect an item in the image data and acquires position information of the detected item in the image data as item position information; an association unit that associates the item with the identification label based on the label position information and the item position information acquired from the same image data; Equipped with Goods management device. (2) Disclosure 2: The item recognition unit sets an item ID for tracking to the detected item, The associating unit associates information on the corresponding identification label with an item ID of the item for which the corresponding identification label exists. Goods management device. (3) Disclosure 3: The label recognition unit acquires label information from the detected identification label; The label information constitutes information on the identification label. Goods management device. (4) Disclosure 4 is that the imaging unit acquires a plurality of consecutive image data by continuously imaging the imaging area, an item tracking unit that determines whether an item detected in one image data matches any of items in other image data that are image data prior to the one image data; an item ID adjustment unit that, when it is determined based on the processing of the item tracking unit that an item detected in one image data is the same item as an item present in other image data, sets the same item ID to the item ID of the item detected in the one image data and the item determined to be the same item; An article management device comprising: (5) Disclosure 5 further relates to the imaging unit, which acquires a plurality of consecutive image data by continuously capturing images of the imaging area; an item tracking unit that determines whether an item detected in one image data matches any of items in other image data that is the image data immediately before the one image data; a transport order assigning unit that assigns transport order information to the item IDs of items that exist in the other image data but do not exist in the one image data based on the processing of the item tracking unit; An article management device comprising: (6) Disclosure 6 includes an item number calculation unit that calculates the number of transported items based on the number of set item IDs. Goods management device. (7) Disclosure 7 is a product management method for tracking each product when transporting multiple products with identification labels attached, capturing an image of an imaging area, which is a position where the conveyed article passes through, and acquiring image data of the image of the article being conveyed; performing image analysis on the image data to detect an identification label in the image data, and acquiring position information of the detected identification label in the image data as label position information; performing image analysis on the image data to detect an item in the image data, and acquiring position information of the detected item in the image data as item position information; Correlating the item with the identification label based on the label position information and the item position information acquired from the same image data. How to manage goods. (8) Disclosure 8 sets an item ID for tracking to the detected item, Link the information of the corresponding identification label to the product ID of the product for which the corresponding identification label exists. How to manage goods. (9) Disclosure 9 acquires label information from the detected identification label, The label information constitutes information for an identification label. How to manage goods. (10) Disclosure 10 acquires a plurality of consecutive image data by continuously capturing images of the imaging area; Determine whether an item detected in one image data matches any of items in other image data that are image data prior to the one image data, and if it is determined that the item detected in the one image data is the same item as an item that existed in the other image data, set the same item ID to the item ID of the item detected in the one image data and the item determined to be the same item. How to manage goods. (11) Disclosure 11 acquires a plurality of consecutive image data by continuously capturing images of the imaging area; Determine whether an item detected in one image data matches any of the items in other image data that is the image data immediately preceding the one image data, and based on the processing of the item tracking unit, assign information on the order of transport to the item IDs of items that exist in the other image data but do not exist in the one image data. How to manage goods. (12) Disclosure 12 calculates the number of transported items based on the number of set item IDs. How to manage goods. (13) Disclosure 13 relates to a case in which multiple identical identification labels are attached to a single item. How to manage goods. (14) Disclosure 14 is that the above-mentioned article has a circular outer shape, The width of the imaging area in the conveying direction is set to be equal to or greater than the circumferential length of the article. How to manage goods. (15) Disclosure 15 is an item management program that causes a computer to function as the item management device of the present disclosure. (16) Disclosure 16 is a method for manufacturing a steel pipe, in which the article is a steel pipe and the method is equipped with the article management device of the present disclosure. [Explanation of symbols]

[0138] 2 Articles (steel pipes) 3 Identification Label 4. Imaging unit 11 Management processing unit main body 11A Video processing section 11B Image processing unit 11Ba Label Recognition Unit 11Bb Article recognition section 11Bc Item Tracking Department 11C Mapping section 11D Article ID adjustment department 11E Transport order assignment unit 11F Item count calculation department

Claims

1. An article management device that tracks each article when transporting a plurality of articles to which an identification label is attached, an imaging unit that acquires image data of an imaging area that is a position where the conveyed article passes; a label recognition unit that performs image analysis on the image data to detect an identification label in the image data and obtains position information of the detected identification label in the image data as label position information; an item recognition unit that performs image analysis on the image data to detect an item in the image data and acquires position information of the detected item in the image data as item position information; an association unit that associates the item with the identification label based on the label position information and the item position information acquired from the same image data; Equipped with Goods management device.

2. The item recognition unit assigns an item ID for tracking to the detected item, the associating unit associates information on the corresponding identification label with an item ID of the item for which the corresponding identification label exists; 2. An article management device according to claim 1.

3. the label recognition unit acquires label information from the detected identification label; The label information constitutes information on the identification label.

3. An article management device according to claim 2.

4. the imaging unit acquires a plurality of consecutive image data by continuously imaging the imaging area; an item tracking unit that determines whether an item detected in one image data matches any of items in other image data that are image data prior to the one image data; an item ID adjustment unit that, when it is determined based on the processing of the item tracking unit that an item detected in one image data is the same item as an item present in other image data, sets the same item ID to the item ID of the item detected in the one image data and the item determined to be the same item; 3. The article management device according to claim 2, further comprising:

5. the imaging unit acquires a plurality of consecutive image data by continuously imaging the imaging area; an item tracking unit that determines whether an item detected in one image data matches any of items in other image data that is the image data immediately before the one image data; a transport order assigning unit that assigns transport order information to the item IDs of items that exist in the other image data but do not exist in the one image data based on the processing of the item tracking unit; 3. The article management device according to claim 2, further comprising:

6. an item number calculation unit that calculates the number of conveyed items based on the number of set item IDs; 3. An article management device according to claim 2.

7. A product management method for tracking each product when transporting a plurality of products to which an identification label is attached, comprising: capturing an image of an imaging area, which is a position where the conveyed article passes through, and acquiring image data of the image of the article being conveyed; performing image analysis on the image data to detect an identification label in the image data, and acquiring position information of the detected identification label in the image data as label position information; performing image analysis on the image data to detect an item in the image data, and acquiring position information of the detected item in the image data as item position information; Correlating the item with the identification label based on the label position information and the item position information acquired from the same image data. How to manage goods.

8. A tracking item ID is set for the detected item, Linking information on the corresponding identification label to the product ID of the product for which the corresponding identification label exists; 8. An article management method according to claim 7.

9. Acquire label information from the detected identification label, The label information constitutes information for an identification label.

9. An article management method according to claim 8.

10. By continuously capturing images of the imaging region, a plurality of continuous image data are obtained; determining whether an item detected in one image data matches an item in other image data that is image data prior to the one image data, and if it is determined that the item detected in the one image data is the same item as an item that existed in the other image data, setting the same item ID for the item detected in the one image data and the item determined to be the same item; 9. An article management method according to claim 8.

11. By continuously capturing images of the imaging region, a plurality of continuous image data are obtained; Determine whether an item detected in one image data matches any of items in other image data that is the image data immediately preceding the one image data, and assign information on the order of conveyance to the item IDs of items that exist in the other image data but do not exist in the one image data.

9. An article management method according to claim 8.

12. The number of items transported is calculated based on the number of item IDs set.

9. An article management method according to claim 8.

13. Multiple identical identification labels are attached to the same item.

9. An article management method according to claim 8.

14. The article has a circular outer shape, The width of the imaging area in the conveying direction is set to be equal to or greater than the circumferential length of the article.

9. An article management method according to claim 8.

15. An article management program that causes a computer to function as the article management device according to any one of claims 1 to 6.

16. The article is a steel pipe, and a manufacturing method of the steel pipe is provided with the article management device according to any one of claims 1 to 6.

Images