Video recording system, video recording method, and program

Abstract

To provide a video recording system, a video recording method, and a program for visually confirming whether or not an article to be sorted is normally sorted.SOLUTION: A video recording system includes: an imaging device 40 for imaging a movie of an article conveyed by an inspection conveyor and a sorting conveyor; a video processing unit 52 for generating a reconstructed movie in which an imaging time is recorded in a frame image constituting the movie imaged by the imaging device 40; an inspection data receiving unit 53 for receiving inspection data output from an inspection device 10; a movie recording unit 54 for cutting out a cut-out movie of a predetermined time from the reconstructed movie; a first article detecting unit 55 for detecting, from the cut-out movie, an article whose determination result is a specific determination result based on the determination time of the specific determination result output from the inspection device 10; a movie editing unit 59 for generating an edited movie in which a mark for identifying the article detected by the first article detecting unit 55 is superimposed on the cut-out movie; and a display part 71 for displaying the edited movie.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a video recording system, a video recording method, and a program for recording the operation of an article inspection system that inspects the quality of articles such as food and medicine. [Background technology]

[0002] Conventionally, food and pharmaceutical manufacturing factories have introduced an item inspection system to inspect items for abnormalities. The item inspection system is equipped with an inspection device that inspects items conveyed sequentially along the production line, and a sorting machine located downstream, and is configured so that items determined to be abnormal by the inspection device (hereinafter also referred to as "defective items") are removed from the production line by the sorting machine.

[0003] In order to prevent defective products from reaching the market, there is a demand for confirmation that the defective products have been reliably removed from the production line. To this end, there is an item inspection system in which a camera is installed to record video when a defective product is detected by an inspection device (for example, video from a few seconds before to a few seconds after the detection of the defective product), and a sorting machine that removes defective products from the production line determines whether the defective product has been removed, thereby confirming the sorting operation (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 3974015 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in addition to defective products, normal products (hereinafter also referred to as "good products") flow simultaneously on the production line, and conventional product inspection systems such as those disclosed in Patent Document 1 can confirm "whether or not products have been sorted," but cannot confirm "whether or not the sorted products are defective." This is because conventional product inspection systems record the sorting operation at the time when defective products are normally transported and reach the sorter, and if the transport of products is disrupted, it is not possible to prove that the products shown in the image or video are defective. Therefore, even if the number of times an abnormality is determined to be the same as the number of rejected products, if the reinspection results of the rejected products are not defective, there is a problem in that it is not possible to clearly confirm that the products rejected by the sorter are defective, even by checking the video.

[0006] In addition, while there has traditionally been a function for checking sorting operations using a light emitter and receiver, this function also does not recognize the judgment results of the items being sorted, so if the transport of items is disrupted, sorting errors cannot be correctly detected, and it is not possible to determine whether the sorting machine has operated on defective items.

[0007] The present invention has been made to solve such conventional problems, and aims to provide a video recording system, a video recording method, and a program that make it possible to visually confirm whether the items to be sorted have been sorted correctly. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems, the video recording system according to the present invention is a video recording system that photographs and records the operation of an item inspection system that includes an inspection device that inspects items transported on an inspection conveyor that constitutes a production line and outputs inspection data including a judgment result as to whether the item is a good or defective item and the judgment time, and a sorting machine that performs a sorting operation to remove, from a sorting conveyor that constitutes the production line, the item for which the judgment result is an abnormal judgment, indicating that the item is a defective item, and a video processing unit that records the shooting time of frame images that make up the video shot by the video shooting device in each of the frame images and generates a reconstructed video by reconstructing the plurality of frame images whose shooting time can be identified in time series; an inspection data receiving unit that receives the inspection data output from the inspection device; a video recording unit that cuts out a cut-out video of a predetermined length from the reconstructed video on condition that a cut-out instruction is output from the inspection data receiving unit in an operation mode in which the plurality of articles are transported on the production line; Abnormality judgment indicating a defective product When Abnormality judgment Based on the determination time, the determination result is Abnormality judgment a first object detection unit that detects the object from the extracted video, and Applicable a video editing unit that generates an edited video in which a mark that identifies an article is superimposed on the extracted video; and a display unit that displays the edited video, and the inspection data receiving unit receives the inspection data output from the inspection device. Abnormality judgment triggered by the instruction to cut out the video, and the predetermined time is Abnormality judgment The time includes the timing when the sorting operation by the sorter is completed and the timing when the output of the sorting operation is completed.

[0009] That is, the video recording system according to the present invention is configured to use a single camera to capture images of multiple items being transported on the inspection conveyor and sorting conveyor from the time an abnormality is determined by the inspection device until the sorting operation of the sorting machine is completed.Furthermore, the video recording system according to the present invention is configured to display on a display unit a video in which a mark identifying an item with a specific determination result is superimposed on multiple items being transported on the inspection conveyor and sorting conveyor.

[0010] As a result, the video recording system of the present invention visually identifies which of multiple items in the video has a specific judgment result, thereby streamlining the user's task of visually confirming an abnormality in the sorting operation of the sorting machine. For example, even if the inspection device fails to detect an abnormality in the sorting operation, the video recording system of the present invention allows the user to visually confirm whether there was actually an abnormality in the sorting operation of the sorting machine, i.e., whether the items to be sorted were sorted correctly. Furthermore, the video recording system of the present invention marks items in the video with a specific judgment result, improving the traceability of items with a specific judgment result and contributing to reducing the risk of defective products being released onto the market. Furthermore, the video recording system of the present invention allows the user to accurately understand the sorting operation of the sorting machine, thereby contributing to improving production line operation and handling.

[0011] Furthermore, the video recording system according to the present invention is configured to, in an operation mode in which one of the articles is transported on the production line, record the following in the reconstructed video: Detecting an outline of one of the articles from a difference image between two consecutive frame images in the time series; A specific detection area on the inspection conveyor In The aforementioned Contour was first detected Applicablea second item detection unit that acquires the photographing time of the frame image; a time difference acquisition unit that acquires the time difference between the determination time output from the inspection device for the one item and the photographing time acquired by the second item detection unit for the one item; and an average time difference calculation unit that calculates an average value of the time differences acquired multiple times by the time difference acquisition unit as an average time difference by transporting the one item multiple times on the production line, Abnormality judgment and calculating an added time, which is a time obtained by adding the average time difference to the judgment time, and determining whether the judgment result is Abnormality judgment The video editing unit may be configured to detect the item, and to superimpose the mark identifying the item detected by the first item detection unit on multiple frame images of the extracted video in which the item was detected by the first item detection unit.

[0012] With this configuration, before starting inspection of multiple items whose pass / fail status is unknown, the video recording system of the present invention calculates the average time difference between the time when the item inspection system makes an abnormality determination and the time when the video recording system detects an item with a specific determination result on the inspection conveyor. This allows the video recording system of the present invention to identify an item with a specific determination result from among multiple items transported on the inspection conveyor and sorting conveyor in a video captured by the camera. Furthermore, by detecting an item with a specific determination result on the inspection conveyor before the sorting machine, the video recording system of the present invention can improve the tracking of an item with a specific determination result in a video from when the inspection device determines that the item is abnormal to when the sorting operation of the sorting machine is completed.

[0013] Furthermore, a video recording method according to the present invention is a video recording method for recording the operation of an item inspection system that includes an inspection device that inspects items transported on an inspection conveyor that constitutes a production line and outputs inspection data including a judgment result as to whether the item is a good item or a defective item and the time of the judgment, and a sorting machine that performs a sorting operation to remove items whose judgment result is an abnormal judgment from a sorting conveyor that constitutes the production line when the judgment result output from the inspection device is an abnormal judgment indicating that the item is a defective item, and the method includes a video recording method for recording the operation of an item inspection system that includes an inspection device that inspects items transported on an inspection conveyor that constitutes a production line and an inspection conveyor that outputs inspection data including a judgment result as to whether the item is a good item or a defective item and the time of the judgment, the video recording method comprising: a video processing step of recording the shooting time of frame images constituting the video shot in the shooting step in each of the frame images and generating a reconstructed video by reconstructing the plurality of frame images whose shooting time can be identified in time series; an inspection data receiving step of receiving the inspection data output from the inspection device by an inspection data receiving unit; a video recording step of cutting out a cut-out video of a predetermined length from the reconstructed video on condition that a cut-out instruction is output from the inspection data receiving unit in an operation mode in which the plurality of articles are transported on the production line; Abnormality judgment indicating a defective product When Abnormality judgment Based on the determination time, the determination result is Abnormality judgment a first object detection step of detecting the object from the extracted video, and Applicable a video editing step of generating an edited video in which a mark for identifying an article is superimposed on the extracted video; and a display step of displaying the edited video, wherein the inspection data receiving unit receives the Abnormality judgment triggered by the instruction to cut out the video, and the predetermined time is Abnormality judgment The time includes the timing when the sorting operation by the sorter is completed and the timing when the output of the sorting operation is completed.

[0014] Furthermore, the video recording method according to the present invention includes, in an operation mode in which one of the articles is transported on the production line, recording the following in the reconstructed video: Detecting an outline of one of the articles from a difference image between two consecutive frame images in the time series; A specific detection area on the inspection conveyor In The aforementioned Contour was first detected Applicable a second item detection step of acquiring the photographing time of a frame image; a time difference acquisition step of acquiring the time difference between the determination time output from the inspection device for the one item and the photographing time acquired by the second item detection step for the one item; and an average time difference calculation step of calculating an average value of the time differences acquired multiple times by the time difference acquisition step as an average time difference by transporting the one item multiple times on the production line, wherein the first item detection step is performed by acquiring the time difference between the determination time output from the inspection device for the one item and the photographing time acquired by the second item detection step. Abnormality judgment and calculating an added time, which is a time obtained by adding the average time difference to the judgment time, and determining whether the judgment result is Abnormality judgment The video editing step may be configured to detect the item, and to superimpose the mark identifying the item detected by the first item detection step on multiple frame images of the extracted video in which the item was detected by the first item detection step.

[0015] Furthermore, the program of the present invention is a program for causing a computer to execute the above-mentioned photographing step, the image processing step, the inspection data receiving step, the video recording step, the first item detection step, the video editing step, and the display step, as well as the above-mentioned second item detection step, the time difference acquisition step, and the average time difference calculation step. [Effects of the Invention]

[0016] The present invention provides a video recording system, a video recording method, and a program that enable visual confirmation of whether or not the items to be sorted have been sorted correctly. [Brief explanation of the drawings]

[0017] [Figure 1]1 is a diagram showing the configuration of an article inspection system according to an embodiment of the present invention and a video recording system that captures and records the operation of the article inspection system. [Figure 2] FIG. 2 is a plan view of the article inspection system shown in FIG. [Figure 3] FIG. 2 is a diagram showing an example of inspection data output from the article inspection system shown in FIG. [Figure 4] 2 is a block diagram showing the configuration of an image processing device provided in the video recording system shown in FIG. 1. FIG. [Figure 5] 2 is a diagram showing a schematic diagram of a change over time when an article is transported on a production line in an edited video generated by the video recording system shown in FIG. 1. FIG. [Figure 6] 1 is a flowchart showing the process of a video recording method using a video recording system according to an embodiment of the present invention. [Figure 7] 7 is a flowchart showing details of the preparation mode process in step S3 in the flowchart of FIG. 6. DETAILED DESCRIPTION OF THE INVENTION

[0018] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of a video recording system, a video recording method, and a program according to the present invention will be described with reference to the accompanying drawings.

[0019] Fig. 1 is a diagram showing the configuration of an article inspection system 1 and a video recording system 2 that captures and records the operation of the article inspection system 1. Fig. 2 is a diagram showing the configuration of the main parts of the article inspection system 1. As shown in Fig. 1, the article inspection system 1 sorts articles W according to their quality state, and includes an inspection device 10, a sorting machine 20, and a defective item storage box 30.

[0020] Examples of the inspection device 10 include an X-ray foreign object detection device (X-ray inspection device), a magnetic field metal detection device, or a weight sorting device using a scale. In this embodiment, an example will be described in which the inspection device 10 is a weight sorting device.

[0021] The inspection device 10 is installed at the rear end of the carry-in conveyor 4, which constitutes part of the production line 3, and is configured to weigh the items W sequentially carried in from the carry-in conveyor 4 to determine whether the items W are good or defective. The sorting machine 20 is configured to sort the items W based on the above-mentioned determination results by the inspection device 10. Items W selected as good are sent directly along the production line 3 and shipped out of the factory, while items W selected as defective are stored in a defective item storage box 30 and isolated for a certain period of time in a state where no foreign matter is mixed in. Note that the items W referred to here are packaged raw meat, fish, processed foods, medicines, etc., transported along the factory production line 3.

[0022] The inspection device 10 includes a control box 11, an approach conveyor 12, an inspection conveyor 13, an input sensor 15, and a load sensor 16. The approach conveyor 12 and the inspection conveyor 13 form part of the production line 3.

[0023] The control box 11 controls and manages the overall operation of the inspection device 10. The control box 11 also inspects a plurality of articles W or one article W0 (described later) transported on the inspection conveyor 13, and outputs inspection data including a determination result as to whether the articles W, W0 are good or bad and the time of the determination.

[0024] Fig. 3 is a diagram showing an example of inspection data output from the control box 11. The "Date and Time" and "Judgment Result" columns in Fig. 3 indicate the time and result of judgment as to whether the item W is good or bad. The judgment result "PASS" means a normal judgment indicating that the item W is a good product, and the judgment result "FAIL" means an abnormal judgment indicating that the item W is a defective product.

[0025] The approach conveyor 12 is a so-called belt conveyor, and is adjacent to the downstream side of the carry-in conveyor 4 so as to receive the articles W from the carry-in conveyor 4, and adjusts the intervals at which the articles W are sequentially delivered from the carry-in conveyor 4 and delivers them to the inspection conveyor 13.

[0026] The inspection conveyor 13 is disposed adjacent to the downstream side of the inlet conveyor 12 so as to receive the articles W from the inlet conveyor 12. The inspection conveyor 13 is placed on a load sensor 16, and the articles W sent from the inlet conveyor 12 are weighed by the load sensor 16 and sent to the sorting machine 20. The inspection conveyor 13 is also formed of a belt conveyor, just like the inlet conveyor 12.

[0027] The carry-in sensor 15 is composed of a light-emitting unit 25a and a light-receiving unit 25b, and is arranged between the entrance conveyor 12 and the inspection conveyor 13 so that the light-emitting unit 25a and the light-receiving unit 25b face each other. The light-receiving unit 25b detects light from the light-emitting unit 25a, and detects the presence of the item W when the item W blocks the light from the light-emitting unit 25a and the light-receiving unit 25b cannot detect the light. When the carry-in sensor 15 detects the presence of the item W, it outputs a carry-in detection signal to the load sensor 16. The carry-in detection signal is used to determine the timing for the load sensor 16 to start weighing.

[0028] The load sensor 16 weighs the item W on the inspection conveyor 13 and outputs a weighing signal to the control box 11. The load sensor 16 also operates and starts weighing the item W after a predetermined time T1 has elapsed since it received the carry-in detection signal from the carry-in sensor 15 (since the carry-in sensor 15 detected the presence of the item W). In the embodiment of the present invention, the "predetermined time T1" is the time required for the item W to completely transfer from the entrance conveyor 12 to the inspection conveyor 13 and for the weighing signal from the load sensor 16 to stabilize.

[0029] The sorting machine 20 includes a sorting conveyor 21, a sorting mechanism 22, and a rejection detection sensor 23. The sorting conveyor 21 constitutes a part of the production line 3.

[0030] The sorting conveyor 21 receives the articles W weighed by the inspection conveyor 13 and transfers them to an output tray (not shown) provided downstream, and is adjacent to the downstream side of the inspection conveyor 13. The sorting conveyor 21 also transfers the articles W received from the inspection conveyor 13 to a sorting position where they can be sorted by the sorting mechanism 22. The sorting conveyor 21 is composed of a belt conveyor, just like the inlet conveyor 12 and the inspection conveyor 13.

[0031] The sorting mechanism 22 is configured to perform a sorting operation on the items W, W0 on the sorting conveyor 21 in accordance with the judgment result output from the control box 11. For example, when the judgment result output from the control box 11 is an abnormal judgment indicating that the items W, W0 are defective, the sorting mechanism 22 performs a sorting operation to push the items W, W0 for which the judgment result is an abnormal judgment from the sorting conveyor 21 to the defective item storage box 30 and remove them from the sorting conveyor 21. On the other hand, when the judgment result output from the control box 11 is a normal judgment indicating that the item W is normal, the sorting mechanism 22 performs a sorting operation to continue transporting the items W (good items) for which the judgment result is a normal judgment on the sorting conveyor 21.

[0032] The sorting mechanism 22 is not limited to a push-out type that pushes the items W, W0 into the defective item storage box 30, but may also employ a flipper type that sorts the items W, W0 by changing the flow direction of the items W, W0 on the sorting conveyor 21 using an arm, or a drop-out type that sorts the items W, W0 by dropping them downward on the sorting conveyor 21. Furthermore, the sorting mechanism is not limited to a mechanical sorting method, and if the items W, W0 are light, an air jet type that blows the items W, W0 away with compressed air may be employed.

[0033] The removal detection sensor 23 is composed of a light-emitting unit 23a and a light-receiving unit 23b, and is arranged between the sorting conveyor 21 and the defective product storage box 30 so that the light-emitting unit 23a and the light-receiving unit 23b face each other. The light-receiving unit 23b detects light from the light-emitting unit 23a, and detects that the item W has been removed from the production line 3 when the item W blocks the light from the light-emitting unit 23a and the light-receiving unit 23b cannot detect the light. When the removal detection sensor 23 detects the removal of the item W, it outputs a removal detection signal to the control box 11.

[0034] The defective product storage box 30 is configured to store the defective products rejected by the sorting mechanism 22. The defective product storage box 30 is disposed opposite the sorting mechanism 22 across the sorting conveyor 21. The defective product storage box 30 is formed in a box shape with an opening 31 at the top, and the inspection objects W pushed out by the sorting mechanism 22 are stored inside through the opening 31.

[0035] If the control box 11 determines, based on the weighing signal output from the load sensor 16, that the weight of the item W is not within the allowable weight range, it determines that the item W is a defective item and outputs a drive signal to the sorting mechanism 22. Upon receiving the drive signal from the control box 11, the sorting mechanism 22 drives to push the item W on the sorting conveyor 21, causing the item W to drop into the defective item storage box 30. On the other hand, if the control box 11 determines, based on the weighing signal output from the load sensor 16, that the weight of the item W is within the allowable weight range, it determines that the item W is a non-defective item and does not output a drive signal to the sorting mechanism 22.

[0036] If the control box 11 receives a rejection detection signal from the rejection detection sensor 23 after counting a predetermined time T2 corresponding to the transport speeds of the inspection conveyor 13 and the sorting conveyor 21 from the time when the normal judgment is output, the control box 11 determines that there is an abnormality in the sorting operation of the sorter 20. On the other hand, if the control box 11 does not receive a rejection detection signal from the rejection detection sensor 23 after counting a predetermined time T2 corresponding to the transport speeds of the inspection conveyor 13 and the sorting conveyor 21 from the time when the abnormality judgment is output, the control box 11 also determines that there is an abnormality in the sorting operation of the sorter 20. The control box 11 is configured to record abnormalities in the sorting operation in association with the time of occurrence.

[0037] The following describes the configuration of the video recording system 2. The video recording system 2 captures and records the operation of the article inspection system 1, and includes an imaging device 40 and an image processing device 50 having a display unit 71 and an operation unit 72, as shown in FIGS.

[0038] The photographing device 40 photographs, at a predetermined frame rate, a video of a single item W0 or multiple items W conveyed on the inspection conveyor 13 and the sorting conveyor 21. As the photographing device 40, for example, a video camera or a network camera capable of photographing video can be used.

[0039] The photographing device 40 is configured to photograph at least the photographing range 5 shown in Fig. 2. The photographing range 5 may be any area through which each of the articles W, W0 is transported from the time the inspection device 10 judges whether the articles are good or bad until the sorting operation of the sorter 20 is completed, and may include, for example, the transport path 13a of the inspection conveyor 13, the transport path 21a of the sorting conveyor 21, at least a portion of the sorter 20, and at least a portion of the opening 31 of the defective item storage box 30. The present invention is characterized in that a single photographing device 40 photographs multiple articles W or one article W0 transported on the inspection conveyor 13 and the sorting conveyor 21.

[0040] Although Figure 1 shows an example in which the imaging device 40 is placed at the rear side of the inspection device 10, the imaging device 40 may be placed at any location, such as at the front side of the inspection device 10, as long as it can photograph the imaging range 5.

[0041] The inspection device 10, the photographing device 40, and the image processing device 50 are interconnected via a network 70 that allows data communication between them. As shown in Fig. 4, the image processing device 50 includes a mode switching unit 51, a video processing unit 52, an inspection data receiving unit 53, a video recording unit 54, a first item detection unit 55, a second item detection unit 56, a time difference acquisition unit 57, an average time difference calculation unit 58, a video editing unit 59, a cut-out video folder 60, and an edited video folder 61.

[0042] The mode switching unit 51 switches the operation mode of the image processing device 50 between an inspection mode in which multiple items W whose quality is unknown are transported on the inspection conveyor 13 and the sorting conveyor 21, and a preparation mode in which only one item W0 is transported on the inspection conveyor 13 and the sorting conveyor 21. The inspection mode is an operation mode when the inspection device 10 inspects multiple items W. The preparation mode is an operation mode for obtaining an average time difference, which will be described later.

[0043] The video processing unit 52 receives video of the items W and W0 captured by the camera device 40, records the time of capture by the camera device 40 for each frame image constituting the received video of the items W and W0, and stores each frame image from which the capture time can be identified. Furthermore, the video processing unit 52 generates a reconstructed video by reconstructing the stored frame images in chronological order. The video processing unit 52 outputs the reconstructed video to the video recording unit 54 in the inspection mode, and outputs the reconstructed video to the second item detection unit 56 in the preparation mode.

[0044] The test data receiving unit 53 is configured to receive test data output from the control box 11 of the test device 10. Furthermore, the test data receiving unit 53 is configured to output an instruction to the video recording unit 54 to cut out a video to be cut out, using a specific judgment result included in the test data output from the control box 11 as a trigger.

[0045] When an extraction instruction is output from the inspection data receiving unit 53 in the inspection mode, the video recording unit 54 extracts an extracted video of a predetermined time T0 from the reconstructed video output from the video processing unit 52 and saves it in the extracted video folder 60. Here, the predetermined time T0 is a time that includes the timing when a specific judgment result is output from the control box 11 and the timing when the sorting operation by the sorter 20 ends, and is, for example, a time of about several seconds before and after the timing when the specific judgment result is output from the control box 11.

[0046] The second item detection unit 56 detects one item W0 that passes through a specific detection area 13b (see FIG. 2) on the inspection conveyor 13 in the reconstructed video output from the image processing unit 52 in the preparation mode, and acquires the shooting time of the frame image in which the one item W0 was first detected. In the preparation mode, one item W0 placed on the carry-in conveyor 4 is transported sequentially through the carry-in conveyor 4, the run-up conveyor 12, the inspection conveyor 13, and the sorting conveyor 21. The transport of one item W0 in the preparation mode is repeated a predetermined number of times. Here, the predetermined number of repetitions is not particularly limited, but is, for example, 5 to 10 times.

[0047] The time difference acquisition unit 57 is configured to acquire the time difference between the determination time output from the control box 11 of the inspection device 10 for one item W0 and the photographing time acquired for one item W0 by the second item detection unit 56. Alternatively, the time difference acquisition unit 57 may acquire the time difference between the time when the carry-in detection signal is output from the carry-in sensor 15 for one item W0 and the photographing time acquired for one item W0 by the second item detection unit 56.

[0048] The average time difference calculation unit 58 is configured to calculate, as an average time difference, the average value of the time differences acquired multiple times by the time difference acquisition unit 57 when one item W0 is transported multiple times along the production line 3 in the preparation mode. The average time difference calculation unit 58 is also configured to record the calculated average time difference in the setting file 62.

[0049] When the judgment result output from the inspection device 10 for the extracted video saved in the extracted video folder 60 is a specific judgment result, the first item detection unit 55 calculates an addition time, which is the time obtained by adding the average time difference read from the setting file 62 to the judgment time of the specific judgment result, and detects the item W for which the judgment result by the inspection device 10 is a specific judgment result in multiple frame images of the extracted video from the shooting time closest to the addition time onwards. Here, the specific judgment result is either a normal judgment or an abnormal judgment, and is set in advance by, for example, a user operating the operation unit 72. In the frame image for the shooting time closest to the addition time, the item W for which the judgment result by the inspection device 10 is a specific judgment result is captured being transported on the inspection conveyor 13.

[0050] The detection of articles W and W0 by the first article detection unit 55 and the second article detection unit 56 may be performed using any known moving object detection technology. The first article detection unit 55, for example, creates a difference image between two temporally consecutive frame images in the extracted video, and detects the contour of article W transported on the inspection conveyor 13 and the sorting conveyor 21 from changes in the created difference image. Similarly, the second article detection unit 56 creates a difference image between two temporally consecutive frame images in the reconstructed video output from the video processing unit 52 in the preparation mode, and detects the contour of article W0 that has passed through a specific detection area 13b on the inspection conveyor 13 from changes in the created difference image.

[0051] The video editing unit 59 generates an edited video in which a mark identifying the item W detected by the first item detection unit 55 is superimposed on multiple frame images of the extracted video in which the item W is detected by the first item detection unit 55, and saves the edited video in the edited video folder 61.

[0052] FIG. 5 is a diagram illustrating, in an edited video generated by the video editing unit 59, a change in the position of an item W, for which the inspection device 10 has determined a specific result, over time as the item W is transported along the production line 3. For example, as shown in FIG. 5, the mark identifying the detected item W may be a frame-shaped figure surrounding the detected item W, or may be any figure such as a circle, ellipse, square, triangle, or star, or any text information including product information such as the weight of the item W. Furthermore, these marks may be appropriately colored. For example, if there are multiple items in the shooting range 5 for which the inspection device 10 has determined a specific result, it is desirable to color each of the items differently.

[0053] The display unit 71 is configured with a display device such as an LCD (Liquid Crystal Display) or a CRT (Cathode Ray Tube), and is configured to display edited moving images stored in the edited moving image folder 61. Furthermore, the display unit 71 is configured to display operation objects such as buttons, soft keys, pull-down menus, and text boxes for setting various conditions.

[0054] The operation unit 72 is for accepting operation inputs by the user, and is configured, for example, as a touch panel equipped with a touch sensor for detecting a contact position by a contact operation on an input surface corresponding to the display screen of the display unit 71. Alternatively, the operation unit 72 may be configured to include an input device such as a keyboard or a mouse.

[0055] For example, the user can select an edited video taken at a desired time or time period by operating the operation unit 72, and display the played edited video on the display unit 71. For example, if the user operates the operation unit 72 to select an edited video taken at the time when an abnormality in the sorting operation of the sorter 20, which is recorded in the control box 11 of the inspection device 10, occurs, the conveyance status of non-defective and defective products at the time of the abnormality in the sorting operation of the sorter 20, can be visually confirmed from the display unit 71. Furthermore, the user can operate the operation unit 72 to select the operation mode of the image processing device 50 in the mode switching unit 51.

[0056] The image processing device 50 is configured with a control device such as a computer including, for example, a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), a read-only memory (ROM), a random access memory (RAM), and a hard disk drive (HDD). For example, the image processing device 50 can be configured with software to include at least a portion of the mode switching unit 51, the video processing unit 52, the inspection data receiving unit 53, the video recording unit 54, the first item detection unit 55, the second item detection unit 56, the time difference acquisition unit 57, the average time difference calculation unit 58, and the video editing unit 59, by executing a predetermined program using the CPU or the GPU. The above programs are pre-stored in the ROM or the HDD. Alternatively, the above programs may be provided or distributed in an installable or executable format recorded on a computer-readable recording medium such as a compact disc or DVD. Alternatively, the above programs may be stored in a computer connected to a network such as the Internet and provided or distributed by downloading via the network.

[0057] An example of the processing of a video recording method using the video recording system 2 will be described below with reference to the flowcharts of Figures 6 and 7. The processing of each step in the flowcharts of Figures 6 and 7 is realized by the computer constituting the image processing device 50 executing the above program.

[0058] First, the mode switching unit 51 switches the operation mode of the image processing device 50 (step S1). The mode switching unit 51 switches the operation mode, for example, at the timing of a user's operation input to the operation unit 72, at a pre-specified time, or several seconds after the article inspection system 1 starts operating.

[0059] If the operation mode switched by the mode switching unit 51 in step S1 is the preparation mode (step S2: YES), the image processing device 50 executes the process of the preparation mode (step S3).

[0060] If the operation mode switched by the mode switching unit 51 in step S1 is the inspection mode (step S2: NO), the image processing device 50 executes the processing of the inspection mode from step S4 onwards.

[0061] In step S4, the production line 3 starts transporting a plurality of articles W (step S4).

[0062] Next, the test data receiving unit 53 starts receiving the test data output from the test device 10 (test data receiving step S5).

[0063] Next, the photographing device 40 starts photographing a moving image of the plurality of articles W being transported on the inspection conveyor 13 and the sorting conveyor 21 (photographing step S6).

[0064] Next, the video processing unit 52 records the shooting time of the frame images that make up the video captured by the camera device 40 in the shooting step S6 in each frame image, and begins generating a reconstructed video by reconstructing multiple frame images whose shooting times can be identified in chronological order (video processing step S7).

[0065] Next, when a cut-out instruction is output from the test data receiving unit 53 (step S8: YES), the video recording unit 54 cuts out a cut-out video of a predetermined time T0 from the reconstructed video generated in the video processing step S7 (video recording step S9).

[0066] Next, the first item detection unit 55 detects an item W whose judgment result is a specific judgment result from the extracted video based on the judgment time of the specific judgment result output from the inspection device 10 (first item detection step S10). For example, the first item detection unit 55 calculates an addition time, which is the time obtained by adding the average time difference obtained in step S40 described below to the judgment time of the specific judgment result output from the inspection device 10, and detects an item W whose judgment result by the inspection device 10 is a specific judgment result in multiple frame images of the extracted video after the shooting time closest to the addition time.

[0067] Next, the video editing unit 59 generates an edited video in which a mark identifying the item with a specific judgment result detected in the first item detection step S10 is superimposed on multiple frame images of the extracted video in which the item with a specific judgment result was detected in the first item detection step S10 (video editing step S11).

[0068] Next, the display unit 71 displays the edited moving image generated in the moving image editing step S11 (display step S12).

[0069] The process of step S3 will be described in detail below with reference to the flowchart of FIG.

[0070] The production line 3 starts transporting one item W0 (step S31).

[0071] Next, the test data receiving unit 53 starts receiving the test data output from the test device 10 (test data receiving step S32).

[0072] Next, the photographing device 40 starts photographing a moving image of one article W0 being transported on the inspection conveyor 13 and the sorting conveyor 21 (photographing step S33).

[0073] Next, the video processing unit 52 records the shooting time of the frame images that make up the video captured by the camera device 40 in the shooting step S33 in each frame image, and begins generating a reconstructed video by reconstructing multiple frame images whose shooting times can be identified in chronological order (video processing step S34).

[0074] Next, the time difference acquisition unit 57 acquires the determination time of the specific determination result output from the inspection device 10 (step S35).

[0075] Next, when one item W0 passes through a specific detection area 13b on the inspection conveyor 13 (step S36: YES), the second item detection unit 56 detects the one item W0 that has passed through the specific detection area 13b on the inspection conveyor 13 in the reconstructed video generated in the image processing step S34, and obtains the shooting time of the frame image in which the one item W0 was first detected (second item detection step S37).

[0076] Next, the time difference acquisition unit 57 acquires the time difference between the judgment time output from the inspection device 10 for one item W0 and the photographing time acquired for one item W0 by the second item detection step S37 (time difference acquisition step S38).

[0077] Next, when the number of times an item W0 is transported on the production line 3 reaches a predetermined number of repetitions (step S39: YES), the average time difference calculation unit 58 calculates the average value of the time differences acquired multiple times by the time difference acquisition step S38 as the average time difference (average time difference calculation step S40).

[0078] If the number of times an item W0 has been conveyed has not reached the predetermined number of repetitions (step S39: NO), the processes from step S31 onwards are executed again.

[0079] As described above, the video recording system 2 according to this embodiment is configured to use a single imaging device 40 to capture images of multiple items W being transported on the inspection conveyor 13 and sorting conveyor 21 from the time a specific judgment result is output by the inspection device 10 until the sorting operation of the sorter 20 is completed. Furthermore, the video recording system 2 according to this embodiment is configured to display on the display unit 71 a video in which a mark identifying an item with a specific judgment result is superimposed on multiple items W being transported on the inspection conveyor 13 and sorting conveyor 21.

[0080] As a result, the video recording system 2 according to the present embodiment visually identifies which of the multiple items W in the video has a specific judgment result, thereby streamlining the user's task of visually confirming that an abnormality has occurred in the sorting operation of the sorter 20. For example, even if the inspection device 10 fails to detect an abnormality in the sorting operation, the video recording system 2 according to the present embodiment allows the user to visually confirm whether there was actually an abnormality in the sorting operation of the sorter 20, i.e., whether the items to be sorted were properly sorted. Furthermore, the video recording system 2 according to the present embodiment marks items in the video with a specific judgment result, improving the traceability of items with a specific judgment result and contributing to reducing the risk of defective products being released onto the market. Furthermore, the video recording system 2 according to the present embodiment allows the user to accurately understand the sorting operation of the sorter 20, thereby contributing to improving the operation and handling of the production line 3. Furthermore, the video recording system 2 according to this embodiment can track articles with specific judgment results even on a high-speed production line 3 where, for example, 120 to 180 articles W are transported per minute.

[0081] Furthermore, before starting inspection of multiple items W whose pass / fail status is unknown, the video recording system 2 according to this embodiment calculates the average time difference between the judgment time of a specific judgment result by the item inspection system 1 and the photographing time at which the video recording system 2 detects the items with the specific judgment result on the inspection conveyor 13. This allows the video recording system 2 according to this embodiment to identify items with the specific judgment result from among multiple items W transported on the inspection conveyor 13 and sorting conveyor 21 in the video captured by the photographing device 40. Furthermore, the video recording system 2 according to this embodiment detects items with the specific judgment result on the inspection conveyor 13 before the sorting machine 20, thereby improving the traceability of items with the specific judgment result in the video from when the inspection device 10 outputs the specific judgment result to when the sorting operation of the sorting machine 20 is completed. [Explanation of symbols]

[0082] 1. Product Inspection System 2. Video Recording System 3 production lines 5. Shooting range 10 Inspection equipment 11 Control box 13 Inspection conveyor 13a Transport path 13b Detection area 15 Carry-in sensor 20 Sorting Machine 21 Sorting conveyor 21a Transport path 22 Sorting mechanism 23 Exclusion detection sensor 30 Defective Goods Storage Box 31 Aperture 40 Imaging equipment 50 Image processing device 51 Mode switching section 52 Video Processing Unit 53 Inspection data receiving unit 54 Video Recording Section 55 First object detection unit 56 Second object detection unit 57 Time difference acquisition unit 58 Average time difference calculation section 59 Video Editing Department 60 Extracted video folder 61 Edited Video Folder 62 Configuration Files 70 Network 71 Display section 72 Control section W,W0 Goods

Claims

1. an inspection device (10) that inspects items (W, W0) conveyed on an inspection conveyor (13) that constitutes a production line (3) and outputs inspection data including a judgment result indicating whether the item is a non-defective or defective item and the time of judgment; a sorting machine (20) that performs a sorting operation to remove, from a sorting conveyor (21) constituting the production line, the items for which the judgment result output from the inspection device is an abnormal judgment indicating that the items are defective, said video recording system (2) capturing and recording the operation of an item inspection system (1), said video recording system comprising: a photographing device (40) for photographing moving images of the articles conveyed on the inspection conveyor and the sorting conveyor; a video processing unit (52) that records the shooting time of each frame image that constitutes the video captured by the imaging device in the frame image, and generates a reconstructed video by reconstructing, in time series, the multiple frame images whose shooting times can be identified; an inspection data receiving unit (53) that receives the inspection data output from the inspection device; a video recording unit (54) that cuts out a predetermined length of video from the reconstructed video when a cut-out instruction is output from the inspection data receiving unit during an operation mode in which a plurality of the articles (W) are transported on the production line; a first item detection unit (55) that, when the judgment result output from the inspection device is an abnormal judgment indicating that the item is a defective product, detects the item whose judgment result is the abnormal judgment from the extracted video based on the judgment time of the abnormal judgment; a video editing unit (59) that generates an edited video in which a mark that identifies the item detected by the first item detection unit is superimposed on the extracted video; a display unit (71) that displays the edited video, the inspection data receiving unit outputs the cutout instruction to the video recording unit in response to the abnormality determination output from the inspection device; A video recording system characterized in that the specified time is a time including the timing when the abnormality determination is output from the inspection device and the timing when the sorting operation by the sorting machine is completed.

2. a second object detection unit (56) that, in an operation mode in which one object (W0) is transported on the production line, detects the contour of one object from a differential image of each of two time-series consecutive frame images in the reconstructed video, and acquires the photographing time of the frame image in which the contour is first detected in a specific detection area (13b) on the inspection conveyor; a time difference acquisition unit (57) that acquires a time difference between the determination time output from the inspection device for the one of the items and the photographing time acquired by the second item detection unit for the one of the items; an average time difference calculation unit (58) that calculates an average value of the time differences acquired multiple times by the time difference acquisition unit as an average time difference by conveying the one item multiple times on the production line, the first article detection unit calculates an added time, which is a time obtained by adding the average time difference to the judgment time of the abnormality judgment output from the inspection device, and detects the article for which the judgment result is the abnormality judgment in a plurality of frame images of the extracted video after the shooting time closest to the added time; The video recording system described in claim 1, characterized in that the video editing unit superimposes the mark identifying the item detected by the first item detection unit on multiple frame images of the extracted video in which the item is detected by the first item detection unit.

3. an inspection device (10) that inspects items (W, W0) conveyed on an inspection conveyor (13) that constitutes a production line (3) and outputs inspection data including a judgment result indicating whether the item is a non-defective or defective item and the time of judgment; a sorting machine (20) that performs a sorting operation to remove, from a sorting conveyor (21) constituting the production line, the items for which the judgment result output from the inspection device is an abnormal judgment indicating that the items are defective, the sorting machine (20) comprising: a photographing step (S6, S33) of photographing a video of the articles being transported on the inspection conveyor and the sorting conveyor; a video processing step (S7, S34) of recording the shooting times of frame images constituting the moving image shot in the shooting step in each of the frame images, and generating a reconstructed moving image by reconstructing the plurality of frame images whose shooting times can be identified in time series; an inspection data receiving step (S5, S32) in which the inspection data output from the inspection device is received by an inspection data receiving unit (53); a moving image recording step (S9) of extracting a predetermined length of extracted moving image from the reconstructed moving image on condition that an extraction instruction is output from the inspection data receiving unit during an operation mode in which a plurality of the articles (W) are transported on the production line; a first item detection step (S10) of detecting, when the judgment result output from the inspection device is an abnormal judgment indicating that the item is a defective product, the item whose judgment result is the abnormal judgment from the extracted video based on the judgment time of the abnormal judgment; a video editing step (S11) of generating an edited video in which a mark identifying the object detected in the first object detection step is superimposed on the extracted video; a display step (S12) of displaying the edited video, the inspection data receiving unit outputs the cut-out instruction to the video recording step in response to the abnormality determination output from the inspection device; A video recording method, characterized in that the specified time is a time including the timing when the abnormality determination is output from the inspection device and the timing when the sorting operation by the sorting machine is completed.

4. a second object detection step (S37) of detecting the contour of one of the objects (W0) from a differential image of each of two time-series consecutive frame images in the reconstructed video during an operation mode in which one of the objects (W0) is transported on the production line, and acquiring the photographing time of the frame image in which the contour is first detected in a specific detection area (13b) on the inspection conveyor; a time difference acquisition step (S38) of acquiring a time difference between the determination time output from the inspection device for the one of the articles and the photographing time acquired by the second article detection step for the one of the articles; an average time difference calculation step (S40) of calculating an average value of the time differences acquired multiple times by the time difference acquisition step as an average time difference by conveying the one article multiple times on the production line, The first article detection step calculates an added time, which is a time obtained by adding the average time difference to the judgment time of the abnormality judgment output from the inspection device, and detects the article whose judgment result is the abnormality judgment in a plurality of frame images of the extracted video that are taken after the shooting time closest to the added time, The video recording method described in claim 3, characterized in that the video editing step superimposes the mark identifying the item detected by the first item detection step on multiple frame images of the extracted video in which the item is detected by the first item detection step.

5. A program for causing a computer to execute the photographing step, the video processing step, the inspection data receiving step, the video recording step, the first item detection step, the video editing step, and the display step described in claim 3, and the second item detection step, the time difference acquisition step, and the average time difference calculation step described in claim 4.

Images