Image display system and image display method

Abstract

An image display system for displaying an image captured by an imaging device comprises: an imaging control section configured to specify one or more imaging conditions from a set of multiple imaging conditions and to cause the imaging device to capture an image of an object based on the imaging conditions; an image generation section configured to generate an image based on other imaging conditions from the multiple imaging conditions, using the image captured by the imaging device, wherein the other imaging conditions differ from the one or more imaging conditions specified by the imaging control section;and a display control section configured to cause a display device to show the image captured by the imaging device or produced by the image generation section and the imaging conditions of the image, so that a user is able to confirm the image and the imaging conditions.

Description

Technical field

[0001] The present description discloses an image display system and an image display method. State of the art

[0002] Previously, a system was proposed that captures and displays multiple images to determine suitable imaging conditions for capturing an object (see, for example, patent literature 1). In this system, images are captured and displayed by an imaging device based on all imaging conditions, and a user selects several images from the displayed images. Then, a composite image is generated by combining several images, and a suitable imaging condition is determined based on the imaging conditions for each selected image. Citation list for patent literature

[0003] Patent literature 1: WO 2020 / 255365 Summary of the invention; Technical task

[0004] Since the imaging device in the system described above is caused to take images based on all imaging conditions so that the user can select multiple images and imaging conditions, a lot of time is needed for image acquisition.

[0005] A main objective of the present disclosure is to appropriately confirm an image and its imaging conditions while simultaneously reducing the time required for image acquisition. Solution to the task

[0006] The present disclosure employs the following means to achieve the main objective described above.

[0007] The present disclosure provides an image display system for displaying an image captured by an imaging device, the image display system comprising: an imaging control section configured to set one or more imaging conditions from a range of imaging conditions and cause the imaging device to take an image of an object based on the set imaging conditions; an image generation section configured to generate an image based on other imaging conditions from among the multiple imaging conditions, using the image acquired by the imaging device, wherein the other imaging conditions differ from the one or more imaging conditions specified by the imaging control section; and a display control section configured to cause a display device to show the image captured by the imaging device or produced by the image generation section and the imaging conditions of the image, so that a user is able to confirm the image and the imaging conditions.

[0008] In the image display system of the present disclosure, one or more imaging conditions are specified among several imaging conditions, and the imaging device is caused to capture an image of an object. An image based on other imaging conditions, which differ from the specified imaging conditions, is then generated using the image captured by the imaging device. Furthermore, the display device is caused to show the image captured or generated by the imaging device and the imaging conditions of the image, so that a user is able to confirm the image and its imaging conditions. This makes it possible to display the image and its imaging conditions even if not all images were captured based on the multitude of imaging conditions.Therefore, it is possible to adequately confirm the image and its imaging conditions while simultaneously reducing the time required to capture an image. Brief description of the drawings Fig. Figure 1 is a schematic configuration diagram of the assembly device 10. Fig. Figure 2 is a schematic configuration diagram of the marker camera 20. Fig. Figure 3 is a block diagram illustrating an electrical connection relationship between the assembly device 10 and the management device 40. Fig. Figure 4 is a flowchart that illustrates an example of the imaging conditions registration processing. Fig. Figure 5 is a diagram that illustrates an example of imaging conditions. Fig. Figure 6 is a diagram that shows an example of the required images needed to generate a pseudo-image. Fig. Figure 7 is a diagram that shows an example of how a pseudo-image is generated. Fig. Figure 8 is a view showing an example of a component P that is a mapping target. Fig. Figure 9 is a diagram that illustrates an example of the relationship between shutter speed and luminance value. Fig. Figure 10 is a diagram that shows an example of a shutter speed coefficient. Fig. Figure 11 is a diagram that shows an example of a Confirmation Screen 50. Fig. Figure 12 is a diagram that shows an example of a confirmation screen 50. Description of embodiments

[0009] One embodiment of the present disclosure is described with reference to the drawings. Fig. Figure 1 is a schematic configuration diagram of the assembly device 10. Fig. Figure 2 is a schematic configuration diagram of the marker camera 20. Fig. Figure 3 is a block diagram showing an electrical connection relationship between the assembly device 10 and the management device 40. In the present embodiment, a left-right direction is shown. Fig. 1 an X-axis direction, a front-back direction a Y-axis direction and an up-down direction a Z-axis direction.

[0010] The assembly device 10 removes a component and attaches the component to the printed circuit board S and comprises a base 11, a printed circuit board transport device 12, an assembly head 14, a motion device 16, a feed device 18, a parts camera 19, a marking camera 20 and a control device 30 (see Fig. 3) Several assembly devices 10 are arranged side by side in a printed circuit board transport direction (X-axis direction) to form an assembly line. The assembly line, which includes assembly devices 10, is controlled by the management device 40 (see Fig. 3) managed.

[0011] The printed circuit board transport device 12 is installed on the base 11 and has a pair of conveyor belts that are provided at intervals in the front-back direction (Y-axis direction) and laid out in the left-right direction (X-axis direction). The printed circuit board S is transported from left to right in Fig. 1 transported by the drive of the conveyor belts.

[0012] The feeding device 18 is attached to a feeding table installed on the base 11, which is oriented in a left-right direction (X-axis direction). The feeding device 18 has a spool on which a tape is wound, which holds a component. The tape has several recesses for receiving components at equal intervals. The component in the recess is exposed by peeling back the film covering the surface of the tape immediately before the component feeding position and is collected (picked up) by the assembly head 14.

[0013] The assembly head 14 includes a holder that holds the nozzle 15, as well as a device for raising and lowering the holder. The nozzle 15 has a suction opening at its tip and picks up a component by means of a vacuum supplied to the suction opening by a vacuum source (not shown). The assembly head 14 can be a head having a single nozzle 15 or a rotating head having several nozzles 15 at equal intervals along a circumferential direction of a cylindrical head body. A mechanical or electromagnetic gripper can also be used instead of the nozzle 15 to pick up the component.

[0014] The motion device 16 moves the assembly head 14 along the XY plane (horizontal plane) in the front-back and left-right directions (XY axis directions). The part camera 19 is installed on the base 11, captures an image of the component picked up by the nozzle 15 from below, and outputs the captured image to the control device 30. The marking camera 20 is provided on a lower part of the assembly head 14 or the like and moves together with the assembly head 14 through the motion device 16 in the front-back and left-right directions (XY axis directions). The marking camera 20 captures an image of a target object from above and outputs the captured image to the control device 30.Examples of imaging objects include a marking attached to the circuit board S, a component in a recess of the tape fed by the feed device 18, a component after assembly on the circuit board S, and the like.

[0015] As in Fig. As shown in Figure 2, the marker camera 20 has an illumination section 21 and an imaging section 25. The illumination section 21 has a side illumination section 22 and an epi-illumination section (coaxial epi-illumination section) 23. The imaging section 25 has a lens 26, a shutter 27, and an imaging element 28. The side illumination section 22 illuminates the object with light at an oblique angle. The side illumination section 22 has multiple light sources that emit different colors of light, for example, a red LED 22r that emits red (R) light and a blue LED 22b that emits blue (B) light. The LEDs 22r and 22b are each arranged in a ring shape with multiple LEDs around the lens 26 when viewed from above. The epi-illumination section 23 illuminates the object with light from the same direction as the optical axis of the lens 26.The epi-illumination section 23 has a half-mirror 24 arranged at an angle of 45 degrees to the optical axis of the lens 26, and a light source that illuminates the half-mirror 24 with light in one direction (horizontal direction) orthogonal to the optical axis of the lens 26. The light source of the epi-illumination section 23 has several light sources with different light colors, for example, a red LED 23r that emits red (R) light, and a blue LED 23b that emits blue (B) light. The side-light section 22 and the epi-illumination section 23 can have a green LED that emits green (G) light.

[0016] Lighting section 21 has a side lighting pattern in which only side lighting section 22 is illuminated, an epi lighting pattern in which only epi lighting section 23 is illuminated, and an all-on pattern in which both side lighting section 22 and epi lighting section 23 are illuminated. Furthermore, in each lighting pattern, lighting section 21 activates either the R or the B lighting color.

[0017] The shutter 27 is positioned between the lens 26 and the imaging element 28. By either allowing or blocking the light passing through the lens 26, the shutter 27 adjusts the exposure time of the imaging element 28 during the exposure. The exposure time can be set by adjusting the shutter speed of the shutter 27. The imaging element 28 is configured as a monochrome imaging element (e.g., a monochrome CCD or similar) that produces a monochrome image based on the received light. The imaging element 28 receives the light emitted by the side illumination section 22 or epi-illumination section 23 and reflected by the object via the semi-mirror 24 and the lens 26, and produces an image.

[0018] The control device 30 has a CPU 32, a ROM 34, a RAM 36, a memory section 38, such as a hard disk or hard disk drive, as well as an input / output port and a communication port (not shown). The control device 30 outputs various control signals to the printed circuit board transport device 12, the assembly head 14, the motion device 16, the feeder 18, the parts camera 19, the marking camera 20, and the like. The control device 30 inputs detection signals from various sensors, including a position sensor that detects the position of the assembly head 14, and inputs images captured by the parts camera 19 and the marking camera 20. The control device 30 performs various processes by processing the images captured by the parts camera 19 and the marking camera 20.The control device 30 is connected to the management device 40 via a communication network to enable bidirectional communication and exchanges data and control signals.

[0019] The management device 40 is a general-purpose computer and comprises a management control section 42 with CPU, ROM, RAM, and the like; an input device 44, such as a keyboard or mouse, with which an operator can perform various input operations; a display 46, which shows various information; and a storage section 48, such as a hard disk or disk drive. The storage section 48 stores a production plan for printed circuit boards S. The production plan specifies which components are to be mounted on which printed circuit boards S in the assembly device 10 and how many such printed circuit boards S, on which the components are mounted, are to be produced. The management device 40 instructs the control device 30 of the assembly device 10 to produce the printed circuit boards S, on which the components are mounted, according to the production plan.

[0020] The assembly device 10, configured as described above, performs an assembly process in which a component is picked up and mounted onto a printed circuit board S, as follows. During assembly, the CPU 32 moves the assembly head 14 (nozzle 15) to a position above the feeder device 18, lowers the nozzle 15, and causes the nozzle 15 to pick up the component. Once the component has been picked up, the CPU 32 moves the assembly head 14 over the printed circuit board camera 19 to a position above the printed circuit board S and lowers the nozzle 15 to mount the component onto the printed circuit board S. The CPU 32 repeats this process for all components that are targets for assembly.

[0021] In the assembly fixture 10, the CPU 32 processes the images captured by the part camera 19 and the marking camera 20 to perform various recognition operations and executes further processing based on the recognition results. For example, the CPU 32 instructs the part camera 19 to capture an image of the component picked up by the nozzle 15, processes the captured image to detect the component's capture offset relative to the nozzle 15 (image center), and corrects the target assembly position to eliminate the capture offset. Furthermore, the CPU 32 instructs the marking camera 20 to capture an image of the marking applied to the circuit board S, processes the captured image to detect the marking, and sets the coordinates of the component's mounting position using the detected position as a reference.Furthermore, the CPU 32 instructs the marking camera 20 to capture an image of the component in the recess being fed to the feeder 18, processes the captured image to detect any positional deviation of the component relative to the center of the recess, and corrects the component's position during capture to eliminate this deviation. Additionally, the CPU 32 processes the image captured by the marking camera 20 to identify a component name, model number, and similar information, and determines whether the component is correct. Predefined imaging conditions are registered in memory section 38, ensuring that each image is captured under optimal conditions for the object being captured. The following describes, as an example, a procedure for registering imaging conditions for the marking camera 20 to capture an image of component P as the object.

[0022] Fig. Figure 4 is a flowchart illustrating an example of the imaging conditions registration processing, which is executed by the management control section 42 (CPU) of the management device 40. In the imaging conditions registration processing, the management control section 42 first specifies one or more imaging conditions from a set of multiple imaging conditions, which specify an illumination color, an illumination pattern, and a shutter speed (SS) (S100). For example, the marker camera 20 of the present embodiment can specify two illumination colors R and B, three illumination patterns (a side illumination pattern, an epi illumination pattern, or an all-on pattern), and twenty shutter speeds from 5 ms to 100 ms in 5 ms increments, thus providing a total of 120 imaging conditions.

[0023] For example, Administrative Control Section 42 specifies ten imaging conditions, conditions 1 to 10, in S100. As in Fig. As shown in Figure 5, conditions 1 through 8 comprise a total of eight conditions, each with two illumination colors (R and B), two illumination patterns (a side illumination pattern and an epi illumination pattern), and two shutter speeds (SS1 and SS2, where SS2 > SS1). Conditions 9 and 10 comprise a total of two conditions in which all of the multiple illuminations are off and the shutter speed is set to SS1 or SS2. Conditions 1 through 8 can also be described as imaging conditions of a first illumination pattern in which one of the multiple illuminations is on, and conditions 9 and 10 can also be described as imaging conditions of a second illumination pattern in which all of the multiple illuminations are off.Conditions 9 and 10 are not imaging conditions actually used in practice, but are defined for generating an image of the all-on pattern. Furthermore, the imaging conditions 1 through 10 are the minimum imaging conditions required to generate images for all other imaging conditions that differ from these 10 conditions, i.e., the remaining 110 imaging conditions.

[0024] Next, the management control section 42 instructs the marking camera 20 to take images of component P based on the imaging conditions specified in S100 (S110). In S110, the management control section 42 transmits the instruction to the control device 30 of the assembly device 10 to take images based on the imaging conditions specified in S100. The control device 30 instructs the marking camera 20 to take the images and transmits the taken images to the management control section 42. In S110, a total of ten images of component P are taken based on the specified ten imaging conditions. Compared to taking all 120 images based on the 120 imaging conditions, the number of taken images can therefore be significantly reduced, and the imaging time in S110 can be considerably shortened.

[0025] Subsequently, the administrative control section 42 performs a pseudo-image generation process to generate a pseudo-image using the captured images (S120). Fig. Figure 6 is a diagram showing an example of the images required to generate a pseudo-image. Fig. Figure 7 is a diagram showing an example of how a pseudo-image is generated. As in Fig. As shown in Figure 6, two images (Image 1 and Image 2) are selected that were taken under one of conditions 1 to 8, and a pseudo-image is created by adding the luminance values ​​of the respective pixels of the selected images and subtracting the luminance values ​​of the respective pixels of Image 3, which was taken under either condition 9 or 10. For example, by summing the luminance values ​​of the respective pixels of Image 1, taken under condition 1, and the luminance values ​​of the respective pixels of Image 2, taken under condition 3, and by subtracting the luminance values ​​of the respective pixels of Image 3, taken under condition 9, a pseudo-image No. 1 is created where the illumination color is R, the illumination pattern is the all-on pattern, and the shutter speed is SS1 (see Figure 6). Fig. 7).

[0026] As described above, in the present embodiment, an image of the all-on pattern of any illumination color is pseudo-generated by adding the luminance values ​​of an image of the side illumination pattern of the same illumination color and the luminance values ​​of an image of the epi illumination pattern of the same illumination color, and by subtracting the luminance values ​​of an image of an all-off pattern. The processing is performed by summing the luminance values ​​of two images (value "2") and then subtracting the luminance values ​​of one image (i.e., a quantity one less than the value "2") of the all-off pattern. The shutter speeds of the images whose luminance values ​​are to be summed and the images whose luminance values ​​are to be subtracted are identical.Since the images from conditions 9 and 10 of the all-off pattern for both illumination colors R and B can be used together, the number of required shots can also be reduced. This means that, compared to capturing each individual image of the side illumination pattern, the epi illumination pattern, and the all-on pattern for illumination color R, and each individual image of the side illumination pattern, the epi illumination pattern, and the all-on pattern for illumination color B, the number of images that need to be captured can be reduced, thus shortening the acquisition time of S110.

[0027] Next, the administrative control section 42 performs the closure time coefficient determination processing (S130). Fig. Figure 8 is a view showing an example of a component P that is a mapping target. As in Fig. As shown in Figure 8, component P, for example, has a plastic body B which, viewed from above, has a rectangular shape and several metal connections L protruding from the body B. Fig. Figure 9 is a diagram that shows an example of the relationship between shutter speed and luminance value. Fig. In diagram 9, the horizontal axis represents the shutter speed, the vertical axis the luminance value, and the actual change in luminance is represented by a solid line. As in Fig. As can be seen in Figure 9, the L terminals tend to have higher luminance values ​​than the B body, but a linear relationship is observed between the luminance value of each of them and the shutter speed. The management control section 42 receives an approximate linear comparison expression for each pixel (see the dashed line in Figure 9). Fig. 9) between the luminance value and the shutter speed.

[0028] Fig. Figure 10 is a diagram showing an example of a shutter speed coefficient. As in Fig. As shown in Figure 10, the management control section 42 determines the slope a and the intercept b of the linear comparison expression as the shutter speed coefficient for each pixel of the image in conjunction with the illumination color (R, B) and the illumination pattern (side illumination, epi-illumination, all-on). For example, for each pixel, the management control section 42 captures two points of the correspondence relationship between the luminance value and the shutter speed of two images taken under conditions 1 and 2, where the illumination color and illumination pattern are the same but the shutter speeds are different. Based on the correspondence relationship between the two points, it calculates and captures the slope a and the intercept b of each pixel for the case where the illumination color is R and the illumination pattern is the side illumination pattern.Management control section 42 also determines the shutter time coefficient from the pseudo-image generated during pseudo-image generation processing. For example, for each pixel, management control section 42 determines two points of the correspondence relationship between the luminance value and the shutter time from two pseudo-images No. 1 and No. 2, in which the illumination color and illumination pattern are the same, but the shutter times are different, and determines, based on the correspondence relationship between the two points, the slope a and the intercept b of each pixel for the case where the illumination color is R and the illumination pattern is the all-on pattern.

[0029] The administrative control section 42 then displays a confirmation screen so that the user (operator) can confirm the images and the imaging conditions (p. 140). For example, the administrative control section 42 displays confirmation screen 50, as shown in the Fig. 11 and Fig. The image is shown on a display 46, as shown in Figure 12. The confirmation screen 50 has an image display field 51, an illumination color setting field 52, an illumination pattern setting field 53, a slider 54, a shutter speed display field 55, a back button 56, and a setting button 57. A command pointer (mouse pointer) 50a is displayed on the confirmation screen 50 and is operated by the input device 44, for example, a mouse.

[0030] The image display field 51 shows an image based on the imaging conditions specified by the user. In the illumination color setting field 52, an illumination color can be set by clicking the command pointer 50a, and each time the command pointer 50a is clicked, the illumination color toggles between R and B. In the illumination pattern setting field 53, an illumination pattern can be set by clicking the command pointer 50a, and each time the command pointer 50a is clicked, the illumination pattern toggles between the side illumination pattern, the epi illumination pattern, and the all-on pattern. The slider 54 sets the shutter speed in 5 ms increments by moving it left and right while dragging the command pointer 50a, and increases the shutter speed when the slider 54 is moved from left to right.The shutter speed changed by the slider 54 is displayed in the shutter speed indicator 55. The Back button 56 is a button for returning to the imaging conditions and to an image before a setting is accepted. The Determine button 57 is a button for determining the imaging conditions of the currently displayed image. The Back button 56 and the Determine button 57 are clicked with the command pointer 50a.

[0031] When the confirmation screen 50 is displayed, the management control section 42 determines, based on the operation performed by the user on the confirmation screen 50 (S150), whether the imaging conditions determination operation (change operation) is performed. If it is determined that the determination operation is not performed, the process proceeds to S200. The imaging conditions determination operations correspond to setting an illumination color in the illumination color determination field 52, setting an illumination pattern in the illumination pattern determination field 53, and setting a shutter speed using the slider 54. If it is determined that the determination operation was performed, the management control section 42 determines whether the set imaging conditions (also referred to as set conditions) are different from the imaging conditions set in S100.The imaging conditions under which marker camera 20 captured the image (S160). If the specified conditions correspond to any of conditions 1 to 8, the management control section 42 in S160 determines that the specified conditions are not other imaging conditions, reads the image captured under the specified conditions (S170), displays the image in image display field 51 (S180), and proceeds to S200.

[0032] If, however, the specified conditions do not correspond to any of conditions 1 to 8, the management control section 42 in S160 determines that the specified conditions are other imaging conditions, generates an image based on the specified conditions (S190), displays the image in image display field 51 (S180), and proceeds to S200. In S190, the management control section 42 generates an image based on the specified illumination color, illumination pattern, and shutter speed, and displays the image in image display field 51. That is, the management control section 42 reads the shutter speed coefficient (slope a and intercept b) that corresponds to the specified illumination color and illumination pattern (see Fig. 10), calculates the luminance values ​​of the respective pixels based on the shutter speed coefficient and the set shutter speed, and displays the resulting image. In Fig. 12, for example, will be, since the slider is 54 compared to Fig. 11 was shifted to the right, thus setting a longer shutter speed, resulting in an image with higher luminance values, i.e., a brighter image than in Fig. 11, generated and displayed in image display field 51. If the calculated luminance values ​​exceed the upper limit (the maximum value of the luminance values), the upper limit is adopted. When the illumination pattern is changed to the all-on pattern, the management control section 42 generates an image by calculating the luminance values ​​of the respective pixels of the all-on pattern image generated in the pseudo-image generation process, based on the shutter speed coefficient and the set shutter speed, and displays the generated image in image display field 51.

[0033] Furthermore, the user can examine the image displayed in the image display field 51, select an image deemed suitable for image processing (i.e., an image in which component P can be easily identified by the image processor), and determine the imaging conditions by pressing the determination button 57 (performing a determination operation). The management control section 42 determines whether the determination of the imaging conditions has been performed (S200), and if it is determined that the determination has not been performed, the process returns to S150. A description of the case in which the return button 56 is pressed is omitted. If, on the other hand, it is determined that the determination operation has been performed, the management control section 42 registers the imaging conditions of the displayed image in conjunction with component P (S210) and terminates the imaging conditions registration process.The imaging conditions are transmitted from the management device 40 to the control device 30 of the assembly device 10 and registered in the memory section 38. Subsequently, when images of component P are being captured, the control device 30 instructs the marking camera 20 to take images based on the imaging conditions.

[0034] At this point, a correspondence relationship between the elements of the present embodiment and the elements of the present disclosure is clarified. The management control section 42, which performs S100 and S110 of the imaging conditions registration processing of the present embodiment, corresponds to an imaging control section of the present disclosure; the management control section 42, which performs S120, S130, and S190 of the processing, corresponds to an image generation section; and the management control section 42, which performs S140, S170, and S180 of the processing, corresponds to a display control section. The management control section 42, which performs S200 and S210 of the processing, corresponds to a determination section. In the present embodiment, an example of the image display method of the present disclosure is also illustrated by the description of the operation of the management device 40.

[0035] In the management device 40 of the embodiment described above, ten imaging conditions are selected from several (for example, 120) imaging conditions, and images based on the selected imaging conditions are captured by the marking camera 20 (imaging device). Furthermore, an image is generated based on other imaging conditions, wherein the other imaging conditions differ from the selected imaging conditions by using an image captured by the marking camera 20. The image captured or generated by the marking camera 20 and the imaging conditions of the image are displayed on the display 46 (display device) so that the user can confirm the image and the imaging conditions of the image.This makes it possible to display the image and its imaging conditions, even if not all images were captured under the various imaging conditions. Therefore, it is possible to adequately confirm the image and its imaging conditions while simultaneously reducing the time required to capture an image.

[0036] The ten imaging conditions define the minimum imaging conditions required to generate all images based on other imaging conditions, which differ from the defined conditions. Therefore, the number of images captured by the marker camera 20 can be minimized, and the time required for image acquisition can be reliably reduced.

[0037] Furthermore, when the marker camera 20 takes an image of the object, the imaging conditions are determined based on the user's determination process (confirmation result) in relation to the image displayed on the confirmation screen 50 of the display 46. By reducing the time required to take the image, the determination of the imaging conditions can also be carried out promptly.

[0038] Furthermore, the ten imaging conditions are defined as follows: the imaging conditions of the first illumination pattern for turning on any one of the multiple illuminations for each illumination color (R, B) and each illumination (side illumination, epi illumination), and the imaging conditions of the second illumination pattern for turning off all of the multiple illuminations. Then, from the images acquired under the first illumination pattern, two images are selected that are required to generate an image of the all-on pattern, in which all illuminations are turned on. An image of the all-on pattern is then generated by adding the luminance values ​​of the respective pixels of the selected images and subtracting the luminance values ​​of the respective pixels of an image acquired under the second illumination pattern.Therefore, it is possible to generate and display an image according to a lighting color and the presence or absence of lighting for each lighting, even without capturing images for all possible lighting patterns that can be determined by the multiple lightings provided for each of the multiple lighting colors.

[0039] Furthermore, for each pixel, a comparison expression between the shutter speed and the luminance value (shutter speed coefficient) is determined based on the luminance values ​​of the respective pixels from images taken under imaging conditions with two different shutter speeds (SS1 and SS2). An image corresponding to the shutter speed is then generated and displayed by calculating the luminance values ​​of the respective pixels corresponding to the specified shutter speed based on this comparison expression. Therefore, it is possible to generate and display an image corresponding to any shutter speed, even if images are not taken for all shutter speeds.

[0040] The present disclosure is in no way limited to the embodiment described above, and it goes without saying that the present disclosure can be embodied in various aspects, as long as the various aspects fall within the technical scope of the present disclosure.

[0041] In the embodiment described above, the images are captured with two different shutter speeds (SS1 and SS2), but the present invention is not limited thereto. That is to say, the images only need to be captured under two or more imaging conditions with different shutter speeds so that a comparison between the shutter speed and the luminance value can be determined for each pixel.

[0042] In this embodiment, eight imaging conditions of the first illumination pattern and two imaging conditions of the second illumination pattern are defined as the ten imaging conditions, and an image of the all-on pattern is generated using images of the first illumination pattern and one image of the second illumination pattern, but the present invention is not limited thereto. That is to say, one or more of the imaging conditions defined among the multiple imaging conditions may be other imaging conditions, wherein the other imaging conditions differ from the defined imaging conditions, i.e., imaging conditions required to generate an image for imaging conditions not captured by the marker camera 20.Elements such as the color of the illumination, the presence or absence of illumination, and the shutter speed are examples of imaging conditions, and not all of these elements need to be adjustable, or other elements besides these may be adjustable.

[0043] As an example, consider a case where both red (R) and blue (B) are switched on to obtain a violet light color (P). In this case, the images of condition 1 of the side illumination pattern with illumination color R and the image of condition 5 of the side illumination pattern with illumination color B are selected as the images of the first illumination pattern. Then, an image of the side illumination pattern with shutter speed SS1 and illumination color P can be generated by adding the luminance values ​​of the respective pixels of the two selected images and subtracting the luminance values ​​of the respective pixels of one image of condition 9, resulting in the image of the second illumination pattern.With an exposure time of SS2, it suffices to select two images from conditions 2 and 6 as images of the first illumination pattern, add their luminance values, select one image from condition 10 as the image of the second illumination pattern, and subtract the luminance values ​​of the first image. An image of the epi illumination pattern can be generated in the same way.

[0044] Furthermore, when generating an image of the all-on pattern for P, the images of the first illumination pattern are selected as follows: the image of condition 1 of the side illumination pattern with illumination color R, the image of condition 3 of the epi illumination pattern with illumination color R, the image of condition 5 of the side illumination pattern with illumination color B, and the image of condition 7 of the epi illumination pattern with illumination color B. Then, an image of the all-on pattern with an illumination color of P and a shutter speed of SS1 can be generated by adding the luminance values ​​of the respective pixels of the four selected images and subtracting three times the luminance values ​​of the respective pixels of the image of condition 9 as the image of the second illumination pattern.If the shutter speed is SS2, it suffices to select four images of condition 2, condition 4, condition 6, and condition 8 as images of the first illumination pattern, add their luminance values, select one image of condition 10 as the image of the second illumination pattern, and subtract three times its luminance value. As described above, only the luminance values ​​of the respective pixels need to be calculated to generate an image by selecting the required images from those taken under the first illumination pattern, summing the luminance values ​​of the respective pixels of the selected n images, and subtracting (n-1) times the luminance values ​​of the respective pixels of an image taken under the second illumination pattern.

[0045] In the present embodiment, ten imaging conditions are defined as the minimum required imaging conditions, but the present invention is not limited to this, and a number of imaging conditions exceeding the required minimum can also be defined. For example, fifteen imaging conditions can be defined, so that the number of shutter speed types is not two, but three. In this way, the setting accuracy of the shutter speed coefficient can be improved, and the pseudo-image can be generated more accurately.

[0046] The present invention includes an example in which the imaging conditions of the marking camera 20 are determined, but is not limited thereto, and the imaging conditions of the part camera 19 can also be determined. Furthermore, the present invention is not limited to determining the imaging conditions, and the image and the imaging conditions only need to be displayed on the display 46 so that the user can confirm the image and the imaging conditions. In addition, while the management device 40 (management control section 42) can have the functions of the imaging control section, the image generation section, and the display control section, the control device 30 or other devices can also have some of these functions.The imaging device is not limited to the device used in the assembly device 10, and the display device is not limited to the display 46 of the management device 40. The present invention is not limited to the technical field of component assembly and can be applied for image display in other technical fields.

[0047] The present description also discloses a technical idea wherein “the image display system according to claim 1 or 2” in claim 4 in the submitted version is amended to “the image display system according to one of claims 1 to 3”, and a technical idea wherein “the image display system according to claim 1 or 2” in claim 5 in the submitted version is amended to “the image display system according to one of claims 1 to 4”. Industrial applicability

[0048] The present disclosure may be used in a technical field or the like of image processing or a component assembly method. List of reference symbols

[0049] 10: Mounting device, 11: Base, 12: PCB transport device, 14: Mounting head, 15: Nozzle, 16: Motion device, 18: Feeding device, 19: Parts camera, 20: Marking camera, 21: Illumination section, 22: Side illumination section, 22b: Blue LED, 22r: Red LED, 23: Epi-illumination section, 23b: Blue LED, 23r: Red LED, 24: Semi-mirror, 25: Imaging section, 26: Lens, 27: Shutter, 28: Imaging element, 30: Control device, 32: CPU, 34: ROM, 36: RAM, 38: Memory section, 40: Management device, 42: Management control section, 44: Input device, 46: Display, 48: Memory section, 50: Confirmation screen, 50a: Command pointer, 51: Image display field, 52: Illumination color selection field, 53: Illumination pattern selection field, 54: Slider, 55: Shutter speed display field, 56: Back button, 57: Selection button, B: Body, L: Connector, P: Component, S: Circuit board. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] WO 2020 / 255365

[0003]

Claims

[1] Image display system for displaying an image captured by an imaging device, the image display system comprising: an imaging control section configured to specify one or more imaging conditions from a set of multiple imaging conditions and to cause the imaging device to acquire an image of an object based on the specified imaging conditions; an image generation section configured to generate an image based on other imaging conditions from among the multiple imaging conditions, using the image acquired by the imaging device, wherein the other imaging conditions differ from the one or more imaging conditions specified by the imaging control section; and a display control section configured to cause a display device to show the image captured by the imaging device or produced by the image generation section and the imaging conditions of the image, so that a user is able to confirm the image and the imaging conditions. [2] The image display system according to claim 1, wherein the imaging control section defines the one or more minimum imaging conditions required by the image generation section to generate all images based on the other imaging conditions. [3] The image display system according to claim 1 or 2, further comprising: a determination section that is configured to determine mapping conditions, when the imaging device captures the image of the object based on a user confirmation result for the image displayed on the display device. [4] The image display system according to claim 1 or 2, wherein the imaging conditions include a condition that specifies the presence or absence of illumination for each of several illuminations provided for each of several illumination colors, the imaging control section defines one or more imaging conditions, a first illumination pattern to turn on any one of the multiple illuminations for each illumination color and for each illumination, and a second illumination pattern to turn off all of the multiple illuminations, and The image generation section is configured to calculate luminance values ​​of respective pixels to generate an image by selecting images taken under the first illumination pattern that are required to generate an image based on an illumination color or the presence or absence of illumination that differs from the illumination color or the presence or absence of illumination specified by the imaging control section, summing the luminance values ​​of respective pixels of the selected images, and subtracting (the number of selected images - 1) times the luminance values ​​of respective pixels of an image taken under the second illumination pattern. [5] Image display system according to claim 1 or 2, wherein the imaging conditions include a condition for a shutter speed of the imaging device, the imaging control section defines one or more imaging conditions as two or more different shutter speeds, and The image generation section, for each pixel, establishes a comparison expression between the shutter speed and the luminance value based on luminance values ​​of respective pixels from images taken at two or more different shutter speeds, and produces an image by calculating luminance values ​​of respective pixels according to a shutter speed that differs from the shutter speed specified by the image generation control section, based on the comparison expression. [6] Image display method for displaying an image captured by an imaging device, the image display method comprising: (a) a step of specifying one or more imaging conditions from several imaging conditions and causing the imaging device to take an image of an object based on the specified imaging conditions; (b) a step of generating, using the image acquired by the imaging device, an image based on other imaging conditions from among the multiple imaging conditions, wherein the other imaging conditions differ from the one or more imaging conditions specified in step (a); and (c) a step in which a display device is caused to display the image captured by the imaging device or produced in step (b) and the imaging conditions of the image, so that a user is able to confirm the image and the imaging conditions.

Images