Image output device and image output system

Abstract

This image output device comprises: an acquisition unit that acquires a raw image and, as information associated with the raw image, associated information which includes the results of image processing performed on the raw image; a generation unit that generates, in advance, a thumbnail image of a raw image selected on the basis of the associated information and from among raw images acquired by the acquisition unit, and that generates, in response to a request, a thumbnail image of a raw image not generated in advance; and an output unit that, in response to a request, outputs a thumbnail image generated by the generation unit.

Description

Technical Field

[0001] This specification discloses an image output device and an image output system.

Background Art

[0002] Conventionally, in a recording and playback device that creates thumbnails of AV data recorded on HD or DVD and stores them in the HD cache area, a device that preferentially creates thumbnails with high displayability has been proposed (see, for example, Patent Document 1). In this recording and playback device, AV data is recorded on HD or DVD in units of titles or chapters, and when a title video selection screen is displayed, thumbnails that will be displayed during the screen transition to the chapter video selection screen are determined to have high displayability and are created in advance. Also, when a chapter video selection screen is displayed, thumbnails that will be displayed during the screen transition to the title video selection screen are created in advance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The technique described in Patent Document 1 can be applied to the case of pre-creating thumbnails of images for which screen transitions are made in a predetermined manner, such as AV data recorded in units of titles or chapters, but cannot be applied to the case of pre-creating thumbnails of images that are variously required from a plurality of images.

[0005] The main object of the present disclosure is to speed up the output of thumbnail images in response to requests by pre-generating thumbnail images and to further reduce the load when pre-creating thumbnail images. [Means for solving the problem]

[0006] This disclosure employs the following means to achieve the primary objectives described above.

[0007] The image output device disclosed herein is An acquisition unit that acquires the original image and associated information including the results of image processing performed on the original image as information associated with the original image, A generation unit pre-generates thumbnail images of selected original images from the original images acquired by the acquisition unit based on the accompanying information, and generates thumbnail images of original images that were not pre-generated upon request. An output unit that outputs a thumbnail image generated by the generation unit in response to a request, The gist of it is that it is equipped with the following features.

[0008] The image output device disclosed herein acquires original images and associated information including the results of image processing performed on those original images, and pre-creates thumbnail images of selected original images based on the associated information. When a user uses the results of image processing to select images to review later, they can pre-create thumbnail images of original images that are likely to be requested for output based on the associated information including the image processing results. As a result, the output of thumbnail images in response to requests can be accelerated by pre-generating thumbnail images. Furthermore, the load during pre-creation can be reduced compared to systems that generate thumbnail images for all acquired original images.

[0009] The image output system disclosed herein is The inspection device includes an imaging unit that captures the original image of the object to be inspected, An image processing unit performs image processing on the original image of the object to be inspected captured by the imaging unit to inspect the object, An acquisition unit that acquires the original image of the object to be inspected captured by the imaging unit and associated information including the inspection result of the object to be inspected by the image processing unit as information associated with the original image, A generation unit pre-generates thumbnail images of selected original images of the object to be inspected, based on the accompanying information, from among the original images of the object to be inspected acquired by the acquisition unit, and generates thumbnail images of original images that were not pre-generated upon request. An output unit that outputs a thumbnail image generated by the generation unit in response to a request, The gist of it is that it is equipped with the following features.

[0010] The image output system disclosed herein can achieve the same effects as the image output device disclosed herein. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic diagram of the production system including the image output system of this embodiment. [Figure 2] This is an explanatory diagram showing an example of a module that makes up a production system. [Figure 3] This flowchart shows an example of image reception processing performed by the management device. [Figure 4] This flowchart shows an example of image display processing performed by the management device. [Figure 5] This is an explanatory diagram showing an example of display information shown on a display device. [Modes for carrying out the invention]

[0012] Next, the forms for implementing this disclosure will be described with reference to the drawings.

[0013] Figure 1 is a schematic diagram of the production system 10 including the image output system of this embodiment. Figure 2 is an explanatory diagram showing an example of a module 20 that constitutes the production system. As shown in the figures, the production system 10 produces circuit boards with components mounted on them and comprises a plurality of modules 20 and a management device 30 that manages the plurality of modules 20.

[0014] The plurality of modules 20 include a printing machine, a printing inspection machine, a component mounting machine, a mounting inspection machine, and the like. Here, the printing machine forms a circuit pattern by printing solder on a substrate. The printing inspection machine inspects whether the solder is normally printed on the substrate by imaging the image of the substrate. The component mounting machine takes out components from the feeder 21 and mounts them on the substrate. The mounting inspection machine inspects whether the components are normally mounted on the substrate by imaging the image of the substrate. Each module 20 includes an imaging unit 20a that images an image of an inspection object (for example, a substrate, solder, components, etc.), and an image processing unit 20b that processes the image of the inspection object imaged by the imaging unit 20a to perform necessary inspections and confirmations.

[0015] As shown in FIG. 2, the component mounting machine as one of the modules 20 includes a feeder 21 as a component supply device that supplies components, a substrate transfer device 22 that transfers the substrate, a head 23 that picks up the components supplied from the feeder 21 to the component supply position with a vertically movable suction nozzle 24 (collecting member) and mounts them on the substrate, and a head moving device 25 that horizontally moves the head 23. In this embodiment, the head 23 holds a plurality of suction nozzles 24, and in one cycle from component collection to mounting, it is possible to pick up a plurality of components at once with the plurality of suction nozzles 24 and mount them on the substrate. Further, the component mounting machine includes a mark camera 26, a side camera 27, and a parts camera 28 as the imaging unit 20a.

[0016] The mark camera 26 is configured to be movable together with the head 23 by the head moving device 25, and images the upper surface of the component supplied by the feeder 21 from above, images the upper surface of the substrate transferred by the substrate transfer device 22 from above, or images the upper surface of the substrate on which the component is mounted by the head 23 from above. The image (original image) imaged by the mark camera 26 is output to the image processing unit 20b.

[0017] The side camera 27 is installed on the head 23 and captures an image of the vicinity of the tip of the suction nozzle 24 from the side when the suction nozzle 24 adsorbs a component. The image (original image) captured by the side camera 27 is output to the image processing unit 20b.

[0018] The parts camera 28 is installed between the feeder 21 and the substrate transfer device 22 and captures an image of the component from below when the suction nozzle 24 that has adsorbed the component passes above. The image (original image) captured by the parts camera 28 is output to the image processing unit 20b.

[0019] The image processing unit 20b acquires the images captured by the mark camera 26, the side camera 27, and the parts camera 28 respectively and performs image processing. Specifically, as the image processing for the image captured by the mark camera 26, when the substrate is carried in by the substrate transfer device 22, the image processing unit 20b acquires the image of the substrate captured by the mark camera 26, processes the acquired image, and confirms the position of the substrate. Also, when the suction nozzle 24 adsorbs the component supplied to the component supply position by the feeder 21, the image processing unit 20b processes the image of the component supply position captured by the mark camera 26 and confirms the presence or absence of the component and the position of the component at the component supply position. Further, after mounting the component on the substrate, the image processing unit 20b acquires the image of the substrate captured by the mark camera 26, processes the acquired image, and confirms the amount of misalignment (position misalignment amount and angle misalignment amount) of the component mounted on the substrate.

[0020] Also, as the image processing for the image captured by the side camera 27, when the suction nozzle 24 adsorbs a component, the image processing unit 20b acquires the image of the vicinity of the tip of the suction nozzle 24 captured by the side camera 27, processes the acquired image, and confirms the presence or absence of the adsorption of the component to the suction nozzle 24 and the adsorption posture.

[0021] Furthermore, as an image processing step for the image captured by the parts camera 28, the image processing unit 20b acquires the image captured by the parts camera 28 when the suction nozzle 24 that has picked up the part passes above the parts camera 28, and processes the acquired image to check the amount of suction displacement (positional displacement and angular displacement) of the part relative to the suction nozzle 24.

[0022] The image processing results from the image processing unit 20b are transmitted to the management device 30 along with the original image as information associated with the original image (status information). For example, when an image of a part (target part) picked up by the suction nozzle 24 is captured by the parts camera 28, the image processing results include the amount of positional displacement (X coordinate value, Y coordinate value) and angular displacement (θ angle value) of the target part relative to the suction nozzle 24 identified by the image processing, as well as information indicating whether the amount of positional displacement and angular displacement are within an acceptable range (presence or absence of errors and error codes). Similarly, when an image of a part (target part) mounted on a substrate is captured by the mark camera 26, the image processing results include the amount of positional displacement (X coordinate value, Y coordinate value) and angular displacement (θ angle value) of the target part relative to the substrate identified by the image processing, as well as information indicating whether the amount of positional displacement and angular displacement are within an acceptable range (presence or absence of errors and error codes). In addition to the image processing results described above, the status information includes parameters necessary for image processing (image processing conditions) and other information. Image processing conditions include, for example, shape data such as the outer shape of the part and tolerance data which is the allowable range of positional and angular displacement of the part; imaging conditions (such as the camera and light source used for imaging); and calibration data (such as the camera's distortion correction value). Other information includes, for example, the date and time the original image was captured; information identifying the type of target part (part type information); information identifying the type of substrate on which the target part is mounted (substrate type information); information identifying the feeder 21 used to supply the target part (feeder information); information identifying the head 23 used to collect and mount the target part (head information); and information identifying the suction nozzle 24 used to collect the target part (nozzle information).

[0023] The control device 30 manages the production line. The control device 30 also acquires and stores original images captured by the imaging unit 20a from each module 20 of the production line, and outputs (displays) them in response to user requests. The control device 30 comprises a processing unit 31, a communication unit 32, and a storage unit 33. Furthermore, the control device 30 is connected to an input device 41, such as a mouse or keyboard, and a display device 42 for displaying various information.

[0024] The processing unit 31 is configured as a microprocessor centered on a CPU, and in addition to the CPU, it includes ROM for storing processing programs and RAM for temporarily storing data. The communication unit 32 is connected to multiple modules 20 that make up the production line in a communicative manner and exchanges various information with these modules 20.

[0025] The storage unit 33 is configured as a storage device such as a hard disk drive or a solid-state drive. The storage unit 33 stores original images 34, status information 35, thumbnail images 36, etc. The original images 34 are captured by the imaging unit 20a (such as a mark camera 26, side camera 27, or part camera 28) and acquired by communication from the corresponding module 20. In this embodiment, the storage unit 33 stores all images captured during production on the production line. The status information 35 includes image processing results, image processing conditions, and other information, and is acquired by communication from the corresponding module 20 along with the original images 34. The thumbnail images 36 are those generated from the original images during the image reception processing and image display processing described later and are stored in the storage unit 33.

[0026] Next, the operation of the image output system and image output device included in the production line configured in this way will be described. Here, the image output system of this embodiment corresponds to the imaging unit 20a and image processing unit 20b provided in each module 20, and the management device 30, and the image output device of this embodiment corresponds to the management device 30.

[0027] Figure 3 is a flowchart showing an example of image reception processing performed by the processing unit 31 of the management device 30. This process is repeated at predetermined intervals. When image reception processing is performed, the processing unit 31 first determines whether or not it has received the original image from the module 20 (S100). If the processing unit 31 determines that it has not received the original image, it terminates the image reception processing.

[0028] When the processing unit 31 determines that it has received the original image, it stores the received original image in the storage unit 33 (S110). Next, the processing unit 31 acquires the image processing results associated with the original image (S120), and based on the information contained in the acquired image processing results, it determines whether the acquired image processing results are errors, that is, whether the acquired image processing results are abnormal results (S130). If the processing unit 31 determines that the image processing results are errors, it pre-generates a thumbnail image of the original image received in S100, that is, the abnormal image determined to be an error in the image processing results (S140), stores the pre-generated thumbnail image in the storage unit 33 in association with the received original image (S150), and terminates the image reception process.

[0029] On the other hand, if the processing unit 31 determines that the image processing result of the original image received in S100 is normal and not an error, it searches for other original images 34 stored in the storage unit 33 that have an error in their image processing result and that are highly related to the original image received in S100 (S160). In this embodiment, this process is performed by searching for original images 34 that have an error in their image processing result and that share at least one of the image processing conditions. For example, when searching for other original images that are highly related to the original image of the target part, examples include original images of other parts collected by the head 23 (multiple suction nozzles 24) in the same cycle as the target part, original images of other parts with the same shape as the target part, and original images of the target part taken with a different camera. If the processing unit 31 determines that there are other highly relevant original images 34 among the original images 34 whose image processing result is an error (YES in S170), it pre-generates a thumbnail image of the original image received in S100 (S140), stores the pre-generated thumbnail image in the storage unit 33 (S150), and terminates the image reception process. On the other hand, if the processing unit 31 determines that there are no other highly relevant original images 34 (NO in S170), it terminates the image reception process without pre-generating a thumbnail image.

[0030] Next, we will describe the image display process performed in response to a user request after a large number of original images 34 (including status information 35) have been stored in the storage unit 33. Figure 4 is a flowchart showing an example of the image display process performed by the processing unit 31 of the management device 30.

[0031] When the image display process is executed, the processing unit 31 of the management device 30 first determines whether or not a display request has been made by the user through the operation of the input device 41 (S200). If the processing unit 31 determines that no display request has been made, it terminates the image display process.

[0032] When the processing unit 31 determines that a display request has been made, it obtains the search conditions specified by the user (S210). It then searches the original images 34 stored in the storage unit 33 for original images 34 that match the specified search conditions (S220). Examples of search conditions include images taken at a specified date and time, images of a specified component type, images of a specified substrate type, images of a specified nozzle, images of a specified camera, and images in which an image processing error occurred with a specified error code.

[0033] Next, the processing unit 31 determines whether a thumbnail image 36 has already been generated for the original image 34 that matches the search criteria (S230). If the processing unit 31 determines that a thumbnail image 36 has already been generated, it retrieves the generated thumbnail image 36 from the storage unit 33 (S240). If it determines that a thumbnail image 36 has not yet been generated, it generates a thumbnail image 36 for the original image 34 that matches the search criteria (S250). When the processing unit 31 generates a thumbnail image 36, it stores the generated thumbnail image 36 in the storage unit 33 in association with the original image 34. The processing unit 31 then waits for the preparation of the thumbnail image 36 to be completed (S260). When the processing unit 31 determines that the preparation of the thumbnail image 36 is complete, it displays the thumbnail image 36 along with the corresponding status information 35 on the display device 42 (S270) and terminates the image display process. The list of thumbnail images 36 may be displayed all at once after all thumbnail images 36 to be displayed on the same screen have been acquired, or they may be displayed sequentially as they are acquired.

[0034] Figure 5 shows an example of the display information shown on the display device 42. As shown in the figure, the display information includes a thumbnail image of the original image, image processing results (for example, information such as positional displacement, angular displacement, and whether or not there are errors), and other information (for example, part type information, part shape, head information, nozzle information, camera information, etc.). This display information is displayed in a list up to a predetermined number (for example, 3) on one screen, and the user can move to the next screen by operating the next button and return to the previous screen by operating the previous button. The display information also includes filtering buttons for narrowing down the data to be displayed by specifying filtering conditions. Filtering buttons include specifying the date and time of imaging, specifying the error code of the image processing, specifying the part, specifying the part shape, specifying the head 23 used, specifying the suction nozzle 24 used, and specifying the camera used. When a user operates a filtering button, the processing unit 31 searches for data using the filtering conditions corresponding to the filtering button and displays the thumbnail image 36 and status information 35 of the data that match the filtering conditions on the display device 42.

[0035] In this case, if an error occurs on the production line during production, the user may want to examine images captured by the imaging unit 20a that resulted in an error in image processing, or images highly related to that error, in order to identify the cause of the error. In this case, the user can specify search conditions to the management device 30 and request a display, which will display a list of thumbnail images of the original images that resulted in an error in image processing, and highly related original images, along with information including the image processing results, on the display device 42. In this embodiment, the processing unit 31 of the management device 30 generates thumbnail images of received original images not only when the image processing result is an error among the original images received from each module 20, but also when the image processing result is normal, but at least one of the received original image and image processing conditions matches, and another original image 34 with an error in image processing is stored in the storage unit 33. Therefore, in response to a user request, thumbnail images of original images with an error in image processing and highly related original images can be displayed quickly, which is useful for identifying errors. Furthermore, compared to pre-generating thumbnail images of all original images received from each module 20, the load of pre-generating thumbnail images can be reduced.

[0036] Here, the correspondence between the main elements of the embodiment and the main elements of the present disclosure as described in the claims will be explained. Specifically, the processing unit 31 of this embodiment that executes S100 to S120 of the image reception processing corresponds to the acquisition unit of the present disclosure, the processing unit 31 that executes S140 of the image reception processing and S250 of the image display processing corresponds to the generation unit, and the processing unit 31 that executes S270 of the image display processing and the display device 42 correspond to the output unit. In addition, the imaging unit 20a (mark camera 26, side camera 27, part camera 28) corresponds to the imaging unit, and the image processing unit 20b corresponds to the image processing unit.

[0037] It goes without saying that this disclosure is not limited in any way to the embodiments described above, and can be implemented in various forms as long as they fall within the technical scope of this disclosure.

[0038] For example, in the embodiment described above, the processing unit 31 pre-generates thumbnail images of received original images 34 if, in addition to receiving original images 34 whose image processing result was an error, other original images 34 stored in the storage unit 33 have normal image processing results, but at least one of the received original images 34 and status information 35 (image processing conditions) matches and the image processing result is an error. However, the processing unit 31 may pre-generate thumbnail images only for the received original images 34 whose image processing result was an error, and generate thumbnail images for the other original images 34 in response to user requests.

[0039] Furthermore, in the embodiment described above, the processing unit 31 displays a thumbnail image of the original image 34 on the display device 42 upon request from the user. However, the processing unit 31 may also output or print information including the thumbnail image to another device upon request from the user.

[0040] Furthermore, in the above-described embodiment, the processing unit 31 acquires original images captured by the imaging unit 20a (mark camera 26, side camera 27, part camera 28) of the component mounting machine as module 20, along with status information, and pre-generates thumbnail images of the original images selected from the acquired original images based on the status information. However, this is not limited to a component mounting machine; in other modules 20 equipped with an imaging unit 20a, such as a printing press, printing inspection machine, or mounting inspection machine, thumbnail images may be pre-generated under similar conditions when acquiring original images captured by the imaging unit 20a.

[0041] As described above, the image output device of this disclosure acquires original images and associated information including the results of image processing performed on the original images, and pre-creates thumbnail images of selected original images based on the associated information. When a user uses the results of image processing to select images to be reviewed later, they can pre-create thumbnail images of original images that are likely to be requested for output based on the associated information including the results of image processing. As a result, the output of thumbnail images in response to requests can be accelerated by pre-generating thumbnail images. Furthermore, the load during pre-creation can be reduced compared to a system that generates thumbnail images for all acquired original images.

[0042] In the image output device of this disclosure, the generation unit may pre-generate thumbnail images for abnormal images, which are original images acquired by the acquisition unit that have yielded abnormal results as a result of the image processing. This allows the thumbnail image to be quickly output when a user wants to check an original image that has yielded an abnormal result as a result of the image processing. In this case, the accompanying information includes the image processing conditions used when performing the image processing, and the generation unit may pre-generate thumbnail images for original images acquired by the acquisition unit that have yielded normal results as a result of the image processing, and for which at least one of the image processing conditions is common to the abnormal image. This allows the thumbnail images to be quickly output when a user wants to check not only an original image that has yielded an abnormal result as a result of the image processing, but also related normal original images.

[0043] This disclosure is not limited to the form of an image output device, but may also be in the form of an image output system that includes an imaging unit and an image processing unit in addition to the image output device. [Industrial applicability]

[0044] This disclosure can be used in industries such as the manufacturing of image output devices and image output systems. [Explanation of symbols]

[0045] 10 Production system, 20 Module, 20a Imaging unit, 20b Image processing unit, 21 Feeder, 22 Substrate transport device, 23 Head, 24 Suction nozzle, 25 Head moving device, 26 Mark camera, 27 Side camera, 28 Parts camera, 30 Management device, 31 Processing unit, 32 Communication unit, 33 Storage unit, 34 Original image, 35 Status information, 36 Thumbnail image, 41 Input device, 42 Display device.

Claims

1. An acquisition unit that acquires the original image and associated information including the results of image processing performed on the original image as information associated with the original image, A generation unit pre-generates thumbnail images of selected original images from the original images acquired by the acquisition unit based on the accompanying information, and generates thumbnail images of original images that were not pre-generated upon request. An output unit that outputs a thumbnail image generated by the generation unit upon request, An image output device equipped with the following features.

2. An image output device according to claim 1, The generation unit pre-generates thumbnail images for abnormal images, which are original images from the acquisition unit that have yielded abnormal results as a result of the image processing. Image output device.

3. The image output device according to claim 2, The aforementioned supplementary information includes the image processing conditions used when performing the image processing. The generation unit pre-generates thumbnail images from the original images acquired by the acquisition unit, for which a normal result is obtained as an image processing result, and which share at least one of the image processing conditions with the abnormal image. Image output device.

4. The inspection device includes an imaging unit that captures the original image of the object to be inspected, An image processing unit performs image processing on the original image of the object to be inspected captured by the imaging unit to inspect the object, An acquisition unit that acquires the original image of the object to be inspected captured by the imaging unit and associated information including the inspection result of the object to be inspected by the image processing unit as information associated with the original image, A generation unit pre-generates thumbnail images of selected original images of the object to be inspected, based on the accompanying information, from among the original images of the object to be inspected acquired by the acquisition unit, and generates thumbnail images of original images that were not pre-generated upon request. An output unit that outputs a thumbnail image generated by the generation unit in response to a request, An image output system equipped with the following features.

Images