Image display system and image display method
The image display system addresses the inefficiency of capturing multiple images by generating pseudo images based on different conditions, allowing for quick confirmation and reducing acquisition time, thereby enhancing operational efficiency.
Patent Information
- Application Number
- PCT/JP2023/018992
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-22
- Publication Date
- 2024-11-28
AI Technical Summary
Conventional image display systems require extensive time to acquire images under multiple imaging conditions, making it inefficient for operators to select and confirm images and their corresponding conditions.
An image display system that sets one or more imaging conditions, captures images based on different conditions, and generates pseudo images to reduce the number of images needed, allowing for efficient display and confirmation of images and their conditions without capturing all possible combinations.
This approach significantly reduces the time required to acquire images while enabling accurate confirmation of image and imaging conditions, minimizing the number of images captured and improving operational efficiency.
Smart Images

Figure JP2023018992_28112024_PF_FP_ABST
Abstract
Description
Image display system and image display method
[0001] This specification discloses an image display system and an image display method.
[0002] A system has been proposed in the past that acquires and displays multiple images to set appropriate imaging conditions for capturing an image of an object (see, for example, Patent Document 1). In this system, an imaging device captures and displays images based on all imaging conditions, and an operator is prompted to select multiple images from the displayed images. The system then generates a composite image by combining the multiple images, and sets appropriate imaging conditions based on the imaging conditions of each selected image.
[0003] International Publication No. 2020 / 255365
[0004] In the above-described system, the worker is required to select from a plurality of images and imaging conditions, and the imaging device is caused to capture images based on all of the imaging conditions, which means that it takes a long time to acquire the images.
[0005] A primary object of the present disclosure is to enable appropriate confirmation of an image and its imaging conditions while reducing the time required to acquire an image.
[0006] The present disclosure has adopted the following means to achieve the above-mentioned main object.
[0007] The image display system of the present disclosure is an image display system that displays an image captured by an imaging device, and includes: an imaging control unit that sets one or more imaging conditions from among a plurality of imaging conditions and causes the imaging device to capture an image of an object based on the imaging conditions; an image generation unit that uses an image captured by the imaging device to generate an image based on an imaging condition other than the imaging condition set by the imaging control unit from among the plurality of imaging conditions; and a display control unit that displays on a display device the image captured by the imaging device or generated by the image generation unit, and the imaging conditions of the image, so that a user can check it.
[0008] In the image display system of the present disclosure, one or more of a plurality of imaging conditions are set and an imaging device is caused to capture an object, and an image based on imaging conditions other than the set imaging conditions is generated using the image captured by the imaging device. Furthermore, the image captured or generated by the imaging device and the imaging conditions of the image are displayed on a display device so that a user can check them. This allows the image and its imaging conditions to be displayed without capturing all of the images based on the plurality of imaging conditions. Therefore, the image and its imaging conditions can be properly checked while reducing the time required to acquire the image.
[0009] 1 is a schematic diagram of a mounting device 10. A schematic diagram of a mark camera 20. A block diagram showing the electrical connection relationship between the mounting device 10 and a management device 40. A flowchart showing an example of an imaging condition registration process. An explanatory diagram showing an example of imaging conditions. An explanatory diagram showing an example of a required image necessary for generating a pseudo image. An explanatory diagram showing an example of how a pseudo image is generated. An explanatory diagram showing an example of a component P to be imaged. An explanatory diagram showing an example of the relationship between shutter speed and brightness value. An explanatory diagram showing an example of a shutter speed coefficient. An explanatory diagram showing an example of a confirmation screen 50. An explanatory diagram showing an example of a confirmation screen 50.
[0010] An embodiment of the present disclosure will be described with reference to the drawings. Fig. 1 is a schematic configuration diagram of a mounting device 10. Fig. 2 is a schematic configuration diagram of a mark camera 20. Fig. 3 is a block diagram showing the electrical connection relationship between the mounting device 10 and a management device 40. In this embodiment, the left-right direction in Fig. 1 is the X-axis direction, the front-rear direction is the Y-axis direction, and the up-down direction is the Z-axis direction.
[0011] The mounting device 10 picks up components and mounts them on the board S, and includes a base 11, a board transport device 12, a mounting head 14, a moving device 16, a feeder 18, a parts camera 19, a mark camera 20, and a control device 30 (see FIG. 3). A plurality of mounting devices 10 are arranged side by side in the board transport direction (X-axis direction) to form a mounting line. The mounting line including the mounting devices 10 is managed by a management device 40 (see FIG. 3).
[0012] The substrate transport device 12 is installed on a base 11 and has a pair of conveyor belts that are spaced apart from each other in the front-to-back direction (Y-axis direction) and that span the left-to-right direction (X-axis direction). The substrate S is transported from left to right in FIG. 1 by driving the conveyor belts.
[0013] The feeders 18 are attached to a feeder table installed on the base 11 so as to be aligned in the left-right direction (X-axis direction). The feeders 18 are equipped with reels around which tape for accommodating components is wound. The tape has a plurality of cavities for accommodating components formed at equal intervals. The components in the cavities are exposed when the film covering the surface of the tape is peeled off just before the component supply position, and are then picked up (sucked) by the mounting head 14.
[0014] The mounting head 14 includes a holder that holds the nozzle 15 and an elevator that raises and lowers the holder. The nozzle 15 has a suction port at its tip, and sucks up a component by negative pressure supplied to the suction port from a negative pressure source (not shown). The mounting head 14 may be a head with a single nozzle 15, or a rotary head with multiple nozzles 15 evenly spaced around the circumference of a cylindrical head body. Instead of the nozzle 15, a mechanical chuck or an electromagnetic chuck may be used as a component picker.
[0015] The moving device 16 moves the mounting head 14 back and forth and left and right (in the XY axis directions) along the XY plane (horizontal plane). The part camera 19 is installed on the base 11, images the components picked up by the nozzle 15 from below, and outputs the captured image to the control device 30. The mark camera 20 is installed below the mounting head 14, and moves back and forth and left and right (in the XY axis directions) together with the mounting head 14 by the moving device 16. The mark camera 20 images the object to be imaged from above and outputs the captured image to the control device 30. Examples of the object to be imaged include marks affixed to the board S, components in cavities in the tape fed out by the feeder 18, and components after being mounted on the board S.
[0016] As shown in FIG. 2 , the mark camera 20 includes an illumination unit 21 and an imaging unit 25. The illumination unit 21 includes a lateral illumination unit 22 and an epi-illumination unit (coaxial epi-illumination unit) 23. The imaging unit 25 includes a lens 26, a shutter 27, and an image sensor 28. The lateral illumination unit 22 irradiates light obliquely onto an object. The lateral illumination unit 22 includes multiple light sources with different illumination colors, for example, a red LED 22r that emits R (red) light and a blue LED 22b that emits B (blue) light. In a top view, multiple LEDs 22r, 22b are arranged in a ring shape around the lens 26. The epi-illumination unit 23 irradiates light onto the object from the same direction as the optical axis of the lens 26. The incident illumination unit 23 includes a half mirror 24 disposed at an angle of 45 degrees with respect to the optical axis of the lens 26, and a light source that irradiates the half mirror 24 with light in a direction (horizontal direction) perpendicular to the optical axis of the lens 26. The light source of the incident illumination unit 23 includes a plurality of light sources with different lighting colors, for example, a red LED 23r that emits R (red) light and a blue LED 23b that emits B (blue) light. Note that the lateral illumination unit 22 and the incident illumination unit 23 may each include a green LED that emits G (green) light.
[0017] The illumination unit 21 has lighting patterns including a side-illumination pattern in which only the side-illumination unit 22 is lit, an incident-illumination pattern in which only the incident-illumination unit 23 is lit, and a full-illumination pattern in which both the side-illumination unit 22 and the incident-illumination unit 23 are lit. In each lighting pattern, the illumination unit 21 lights up in either R or B.
[0018] The shutter 27 is disposed between the lens 26 and the image sensor 28, and adjusts the exposure time of the image sensor 28 during image capture by switching between passing and blocking light that has passed through the lens 26. This exposure time can be adjusted by setting the shutter speed of the shutter 27. The image sensor 28 is configured as a monochrome image sensor (e.g., a monochrome CCD) that generates a monochromatic image based on received light. The image sensor 28 receives light emitted from the lateral illumination unit 22 or the epi-illumination unit 23 and reflected by the object via the half mirror 24 and the lens 26, and generates an image.
[0019] The control device 30 includes a CPU 32, ROM 34, RAM 36, a storage unit 38 such as a hard disk or solid state drive, and input / output ports and communication ports (not shown). The control device 30 outputs various control signals to the board transport device 12, the mounting head 14, the moving device 16, the feeder 18, the part camera 19, the mark camera 20, etc. The control device 30 also receives detection signals from various sensors, including a position sensor that detects the position of the mounting head 14, and receives images captured by the part camera 19 and the mark camera 20. The control device 30 processes the images captured by the part camera 19 and the mark camera 20 to perform various processes. The control device 30 is connected to the management device 40 via a communication network so as to be able to communicate bidirectionally, and exchanges data and control signals with each other.
[0020] The management device 40 is a general-purpose computer that includes a management control unit 42 composed of a CPU, ROM, RAM, etc., an input device 44 such as a keyboard or mouse that allows an operator to perform various input operations, a display 46 that displays various information, and a storage unit 48 such as a hard disk or solid state drive. A production schedule for boards S is stored in the storage unit 48. The production schedule determines which components are to be mounted on which boards S by the mounting device 10, and how many boards S with such components mounted are to be produced. The management device 40 instructs the control device 30 of the mounting device 10 to perform production so that boards S with components mounted thereon are produced in accordance with the production schedule.
[0021] The mounting device 10 configured in this manner performs a mounting process in which components are picked up and mounted on the board S as follows. In the mounting process, the CPU 32 moves the mounting head 14 (nozzle 15) to above the component supply position of the feeder 18, and then lowers the nozzle 15 to pick up the component. Once the component has been picked up, the CPU 32 moves the mounting head 14 above the board S via the part camera 19, and then lowers the nozzle 15 to mount the component on the board S. The CPU 32 repeats this process for all components to be mounted.
[0022] In addition, in the mounting device 10, the CPU 32 processes images captured by the part camera 19 and the mark camera 20 to perform various recognition processes and then performs processing according to the recognition results. For example, the CPU 32 causes the part camera 19 to capture an image of a component picked up by the nozzle 15, processes the captured image to recognize a component pickup deviation relative to the nozzle 15 (image center), and corrects the target mounting position to eliminate the pickup deviation. The CPU 32 also causes the mark camera 20 to capture an image of a mark affixed to the board S, processes the captured image to recognize the mark, and sets the component mounting position coordinates using the recognized position as a reference position. The CPU 32 also causes the mark camera 20 to capture an image of a component in a cavity delivered to the component supply position of the feeder 18, processes the captured image to recognize a component position deviation relative to the cavity center, and corrects the component pickup position to eliminate the deviation. The CPU 32 also processes the image captured by the mark camera 20 to recognize the component name, model, etc., and determine whether the component is correct. In order to capture each image under optimal imaging conditions suited to the object to be imaged, predetermined imaging conditions are registered in the storage unit 38. The following describes an example of a process for registering imaging conditions for the mark camera 20 to image the component P as the object.
[0023] 4 is a flowchart showing an example of an imaging condition registration process, which is executed by the management control unit 42 (CPU) of the management device 40. In the imaging condition registration process, the management control unit 42 first sets one or more imaging conditions from among a plurality of imaging conditions that define a lighting color, a lighting pattern, and a shutter speed (SS) (S100). The mark camera 20 of this embodiment can set, for example, two lighting colors (R or B), three lighting patterns (side illumination pattern, epi-illumination pattern, and full illumination pattern), and 20 shutter speeds in 5 ms increments between 5 ms and 100 ms, providing a total of 120 imaging conditions.
[0024] The management control unit 42 sets ten imaging conditions, Condition 1 to Condition 10, in S100, for example. As shown in FIG. 5 , Conditions 1 to 8 are eight conditions in total: two illumination colors (R or B), two illumination patterns (side illumination or incident illumination), and two shutter speeds (SS1 or SS2 (>SS1). Conditions 9 and 10 are two conditions in total: a shutter speed of SS1 or SS2 with all of the multiple illuminations turned off. Conditions 1 to 8 are also referred to as imaging conditions for a first illumination pattern in which one of the multiple illuminations is turned on, and Conditions 9 and 10 are also referred to as imaging conditions for a second illumination pattern in which all of the multiple illuminations are turned off. Conditions 9 and 10 are not actually used imaging conditions, but are set for generating images of all illumination patterns. Conditions 1 to 10 are the minimum imaging conditions required to generate images of all imaging conditions other than the ten imaging conditions, i.e., the remaining 110 imaging conditions.
[0025] Next, the management control unit 42 causes the mark camera 20 to capture images of the component P based on the imaging conditions set in S100 (S110). In S110, the management control unit 42 transmits an instruction to capture images based on the imaging conditions determined in S100 to the control device 30 of the mounting device 10. The control device 30 controls the mark camera 20 to capture images and transmits the captured images to the management control unit 42. Note that in S110, a total of 10 images of the component P are captured based on the 10 imaging conditions that have been set. Therefore, the number of captured images can be significantly reduced compared to capturing all 120 images based on 120 imaging conditions, and the imaging time in S110 can be significantly shortened.
[0026] Next, the management control unit 42 executes a pseudo image generation process to generate a pseudo image using the captured images (S120). FIG. 6 is an explanatory diagram showing an example of a required image for generating a pseudo image. FIG. 7 is an explanatory diagram showing an example of how a pseudo image is generated. As shown in FIG. 6, a pseudo image is generated by selecting two images (image 1 and image 2) captured under any of conditions 1 to 8, adding the luminance values of each pixel of the selected images, and subtracting the luminance value of each pixel of image 3 captured under any of conditions 9 and 10 by one image. For example, by adding the luminance value of each pixel of image 1 captured under condition 1 and the luminance value of each pixel of image 2 captured under condition 3, and subtracting the luminance value of each pixel of image 3 captured under condition 9, pseudo image No. 1 is generated, with the illumination color set to R, the illumination pattern set to all illumination patterns, and the shutter speed set to SS1 (see FIG. 7).
[0027] As described above, in this embodiment, an image of a full-on pattern of any lighting color is generated by adding the luminance values of an image of a side-illumination pattern of the same lighting color and an image of an incident-illumination pattern of the same lighting color and subtracting the luminance value of an image of an off-pattern. The luminance values of two images (value 2) are added together and then subtracted from the luminance value of one image of the off-pattern, which is one value less than value 2. The shutter speeds for the image whose luminance values are added and the image whose luminance values are subtracted are the same. Furthermore, since the images under conditions 9 and 10 of the off-pattern can be used for both R and B lighting colors, the number of images that need to be captured can be reduced. In other words, the number of images that need to be captured can be reduced, thereby shortening the capture time in S110, compared to capturing images of the side-illumination pattern, incident-illumination pattern, and full-on pattern of the R lighting color and images of the side-illumination pattern, incident-illumination pattern, and full-on pattern of the B lighting color.
[0028] Next, the management control unit 42 executes a shutter speed coefficient setting process (S130). FIG. 8 is an explanatory diagram illustrating an example of a component P to be imaged. As illustrated, the component P includes a resin body B that is rectangular in top view and multiple metal leads L protruding outward from the body B. FIG. 9 is an explanatory diagram illustrating an example of the relationship between shutter speed and brightness value. In FIG. 9, the horizontal axis represents shutter speed, the vertical axis represents brightness value, and the solid line represents actual brightness change. As illustrated, the leads L tend to have higher brightness values than the body B, but both brightness values exhibit a linear relationship with the shutter speed. The management control unit 42 calculates an approximate linear relationship between brightness value and shutter speed (see the dotted line in FIG. 9) for each pixel.
[0029] FIG. 10 is an explanatory diagram showing an example of a shutter speed coefficient. As shown in the figure, the management control unit 42 associates the lighting color (R, B) with the lighting pattern (side illumination, incident illumination, full illumination) and acquires the slope a and intercept b of a linear relationship for each pixel in the image as the shutter speed coefficient. For example, the management control unit 42 acquires two points of correspondence between brightness value and shutter speed for each pixel from two images captured under conditions 1 and 2, which have the same lighting color and lighting pattern but different shutter speeds. Based on the correspondence between the two points, the management control unit 42 calculates and acquires the slope a and intercept b for each pixel when the lighting color is R and the lighting pattern is a side illumination pattern. The management control unit 42 also acquires a shutter speed coefficient from the pseudo image generated in the pseudo image generation process. For example, the management control unit 42 obtains two correspondences between brightness values and shutter speeds for each pixel from two pseudo images No. 1 and No. 2 that have the same lighting color and lighting pattern but different shutter speeds, and based on the correspondences between the two points, obtains the slope a and intercept b of each pixel when the lighting color is R and the lighting pattern is the full lighting pattern.
[0030] Next, the management control unit 42 displays a confirmation screen for allowing the user (operator) to confirm the image and imaging conditions (S140). For example, the management control unit 42 displays a confirmation screen 50 on the display 46, as shown in Figures 11 and 12. The confirmation screen 50 is provided with an image display field 51, a lighting color specification field 52, a lighting pattern specification field 53, a slider 54, a shutter speed display field 55, a back button 56, and a confirm button 57. The confirmation screen 50 also displays an instruction pointer (mouse pointer) 50a that can be operated by an input device 44, such as a mouse.
[0031] The image display field 51 displays an image based on the imaging conditions specified by the user. In the illumination color specification field 52, the illumination color can be specified by clicking the instruction pointer 50a, and the color switches between R and B each time the instruction pointer 50a is clicked. In the illumination pattern specification field 53, the illumination pattern can be specified by clicking the instruction pointer 50a, and the color switches between a side illumination pattern, an incident illumination pattern, and a full illumination pattern each time the instruction pointer 50a is clicked. The slider 54 can be dragged left and right to specify the shutter speed in 5 ms increments, with the shutter speed increasing as the slider is slid from left to right. The shutter speed display field 55 displays the shutter speed changed by the slider 54. The back button 56 is a button for returning to the imaging conditions and image previous to the acceptance of the specification. The confirm button 57 is a button for confirming the imaging conditions of the currently displayed image. The back button 56 and confirm button 57 are clicked with the instruction pointer 50a.
[0032] When the management control unit 42 displays the confirmation screen 50, it determines whether a designation operation (change operation) of the image capture conditions has been performed based on the user's operation on the confirmation screen 50 (S150). If it determines that a designation operation has not been performed, the process proceeds to S200. Designation operations of the image capture conditions include designating a lighting color in the lighting color designation field 52, designating a lighting pattern in the lighting pattern designation field 53, and designating a shutter speed using the slider 54. If it determines that a designation operation has been performed, the management control unit 42 determines whether the designated image capture conditions (also referred to as designated conditions) are different from the image capture conditions set in S100, i.e., the image capture conditions used by the mark camera 20 (S160). If the designated conditions correspond to any of conditions 1 to 8, the management control unit 42 determines in S160 that the designated conditions are not different from the image capture conditions, reads out an image captured under the designated conditions (S170), displays it in the image display field 51 (S180), and proceeds to S200.
[0033] On the other hand, if the specified condition does not fall under any of Conditions 1 to 8, the management control unit 42 determines in S160 that the specified imaging condition is a different imaging condition, generates an image based on the specified condition (S190), displays it in the image display field 51 (S180), and proceeds to S200. In S190, the management control unit 42 generates an image based on the specified lighting color, lighting pattern, and shutter speed, and displays it in the image display field 51. That is, the management control unit 42 reads the shutter speed coefficient (slope a, intercept b) corresponding to the specified lighting color and lighting pattern (see FIG. 10), calculates the brightness value of each pixel based on the shutter speed coefficient and the specified shutter speed, and displays the generated image. For example, in FIG. 12, the slider 54 is slid to the right more than in FIG. 11, specifying a faster shutter speed, and therefore a brighter image with a higher brightness value than in FIG. 11 is generated and displayed in the image display field 51. Note that if the calculated brightness value exceeds the upper limit (maximum brightness value), the upper limit is used. In addition, when the image is changed to the full lighting pattern, the management control unit 42 calculates the brightness value of each pixel in the image of the full lighting pattern generated by the pseudo image generation process based on the shutter speed coefficient and the specified shutter speed to generate an image, and displays the generated image in the image display area 51.
[0034] The user can also confirm the images displayed in the image display field 51, select an image that is likely to be suitable for image processing, i.e., that will easily recognize the component P, and determine the imaging conditions by operating the decision button 57 (decision operation). The management control unit 42 determines whether the decision operation for the imaging conditions has been performed (S200). If it determines that the decision operation has not been performed, the process returns to S150. A description of the case where the back button 56 is operated will be omitted. On the other hand, if the management control unit 42 determines that the decision operation has been performed, the management control unit 42 registers the imaging conditions for the displayed image in association with the component P (S210) and terminates the imaging condition registration process. The imaging conditions are transmitted from the management device 40 to the control device 30 of the mounting device 10 and registered in the storage unit 38. Thereafter, when imaging the component P, the control device 30 controls the mark camera 20 based on the imaging conditions to perform imaging.
[0035] Here, the correspondence between the components of this embodiment and the components of the present disclosure will be clarified. The management control unit 42 that executes S100 and S110 of the imaging condition registration process of this embodiment corresponds to the imaging control unit of this disclosure, the management control unit 42 that executes S120, S130, and S190 of the same process corresponds to the image generation unit, and the management control unit 42 that executes S140, S170, and S180 of the same process corresponds to the display control unit. Furthermore, the management control unit 42 that executes S200 and S210 of the same process corresponds to the determination unit. In this embodiment, an example of an image display method of the present disclosure is also clarified by explaining the operation of the management device 40.
[0036] In the management device 40 of the embodiment described above, 10 imaging conditions are set from a plurality of imaging conditions (e.g., 120) and images based on the set imaging conditions are captured by the mark camera 20 (imaging device). Furthermore, images based on imaging conditions other than the set imaging conditions are generated using images captured by the mark camera 20, and the images captured or generated by the mark camera 20 and the imaging conditions of the images are displayed on the display 46 (display device) so that the user can check them. This makes it possible to display images and their imaging conditions without capturing all images based on multiple imaging conditions. Therefore, the time required to acquire images can be reduced while allowing the user to properly check the images and their imaging conditions.
[0037] Furthermore, the ten imaging conditions are set to the minimum imaging conditions necessary to generate all images based on imaging conditions other than the set imaging conditions, thereby minimizing the number of images captured by the mark camera 20 and reliably reducing the time required to acquire the images.
[0038] Furthermore, the imaging conditions for the mark camera 20 to capture an image of the target are determined based on the user's confirmation operation (confirmation result) for the image displayed on the confirmation screen 50 of the display 46. By reducing the time required to acquire an image, the imaging conditions can also be determined quickly.
[0039] Furthermore, as the ten imaging conditions, imaging conditions for a first lighting pattern in which one of the plurality of lights is turned on are set for each lighting color (R, B) and for each lighting type (side illumination, incident illumination), and imaging conditions for a second lighting pattern in which all of the plurality of lights are turned off are set. Then, two images necessary for generating an image of all lighting patterns in which all lights are turned on are selected from the images captured with the first lighting pattern, and the luminance values of each pixel are added together, and the luminance values of each pixel of the image captured with the second lighting pattern are subtracted by one image, thereby generating an image of all lighting patterns. Therefore, even if it is not necessary to capture images of all lighting patterns that can be set for the plurality of lights provided for each of the plurality of lighting colors, it is possible to appropriately generate and display images according to the lighting colors and whether each light is on or off.
[0040] Furthermore, a relational expression (shutter speed coefficient) between shutter speed and luminance value is set for each pixel based on the luminance value of each pixel in images captured under imaging conditions of two different shutter speeds (SS1, SS2). Then, the luminance value of each pixel corresponding to the specified shutter speed is calculated based on the relational expression, and an image corresponding to the shutter speed is generated and displayed. Therefore, an image corresponding to any shutter speed can be appropriately generated and displayed without capturing images at all shutter speeds.
[0041] It goes without saying that the present disclosure is not limited to the above-described embodiments, and can be embodied in various forms as long as they fall within the technical scope of the present disclosure.
[0042] In the above-described embodiment, images are captured at two different shutter speeds (SS1, SS2), but this is not limiting. That is, images may be captured under two or more imaging conditions with different shutter speeds so that the relationship between the shutter speed and the luminance value can be set for each pixel.
[0043] In the embodiment, eight imaging conditions for the first lighting pattern and two imaging conditions for the second lighting pattern are set as the ten imaging conditions, and images of all lighting patterns are generated using images of the first lighting pattern and images of the second lighting pattern, but this is not limited to this. That is, one or more imaging conditions set among the multiple imaging conditions may be imaging conditions necessary to generate images under imaging conditions other than the set imaging conditions, i.e., imaging conditions under which the mark camera 20 did not capture images. Note that items such as illumination color, whether or not lighting is on, and shutter speed are examples of imaging conditions, and it is not necessary to set any of these items, or items other than these may be set.
[0044] For example, consider a case where both R (red) and B (blue) are lit to produce a P (purple) illumination color. In this case, an image under condition 1 with an R illumination color and a side-illumination pattern and an image under condition 5 with a B illumination color and a side-illumination pattern are selected as the first illumination pattern image. Then, by adding the luminance values of each pixel of the two selected images and subtracting the luminance value of each pixel of the image under condition 9 as the second illumination pattern image by one image, an image with a shutter speed of SS1 and a side-illumination pattern with a P illumination color can be generated. If the shutter speed is SS2, two images under conditions 2 and 6 are selected as the first illumination pattern image, their luminance values are added together, and an image under condition 10 is selected as the second illumination pattern image and their luminance value is subtracted by one image. Images with an epi-illumination pattern can also be generated in a similar manner.
[0045] Furthermore, when generating an image of all lighting patterns of P, an image under condition 1 in which the lighting color is R and the side-illumination pattern, an image under condition 3 in which the lighting color is R and the side-illumination pattern, an image under condition 5 in which the lighting color is B and the side-illumination pattern, and an image under condition 7 in which the lighting color is B and the side-illumination pattern are selected as images of the first lighting pattern. Then, the luminance values of each pixel of the selected four images are added together, and the luminance value of each pixel of the image under condition 9 is subtracted by three images (multiplied by three) as the image of the second lighting pattern, thereby generating an image of all lighting patterns with a shutter speed of SS1 and a lighting color of P. If the shutter speed is SS2, four images under conditions 2, 4, 6, and 8 are selected as images of the first lighting pattern, and their luminance values are added together, and an image under condition 10 is selected as the image of the second lighting pattern, and its luminance value is subtracted by three images. In this way, the necessary images are selected from the images captured using the first lighting pattern, the brightness values of each pixel of the selected n images are added together, and the brightness values of each pixel of the images captured using the second lighting pattern are subtracted by a value (n-1) to calculate the brightness values of each pixel and generate an image.
[0046] In the embodiment, ten imaging conditions are set as the minimum necessary imaging conditions, but this is not limiting and a number of imaging conditions exceeding the minimum necessary may be set. For example, fifteen imaging conditions may be set so that there are three shutter speeds instead of two. This improves the accuracy of setting the shutter speed coefficient, allowing for more accurate generation of pseudo images.
[0047] In the embodiment, the imaging conditions of the mark camera 20 are determined as an example, but the invention is not limited to this and may be used to determine imaging conditions of the parts camera 19. Furthermore, the invention is not limited to determining imaging conditions, and may simply display an image and its imaging conditions on the display 46 so that the user can check them. Furthermore, the management device 40 (management control unit 42) has the functions of an imaging control unit, an image generation unit, and a display control unit, but the control device 30 or another device may have some of these functions. The imaging device is not limited to the one used in the mounting device 10, and the display device is not limited to the display 46 of the management device 40. Furthermore, the invention may be applied to image display in other technical fields, not limited to the technical field of component mounting processing.
[0048] This specification also discloses the technical idea of changing "the image display system according to claim 1 or 2" in claim 4 as originally filed to "the image display system according to any one of claims 1 to 3," and the technical idea of changing "the image display system according to claim 1 or 2" in claim 5 as originally filed to "the image display system according to any one of claims 1 to 4."
[0049] The present disclosure is applicable to technical fields such as image processing and component mounting processing.
[0050] 10 Mounting device, 11 Base, 12 Substrate transport device, 14 Mounting head, 15 Nozzle, 16 Moving device, 18 Feeder, 19 Parts camera, 20 Mark camera, 21 Illumination unit, 22 Side illumination unit, 22b Blue LED, 22r Red LED, 23 Incident illumination unit, 23b Blue LED, 23r Red LED, 24 Half mirror, 25 Imaging unit, 26 Lens, 27 Shutter, 28 Imaging element, 30 Control device, 32 CPU, 34 ROM, 36 RAM, 38 Storage unit, 40 Management device, 42 Management control unit, 44 Input device, 46 Display, 48 Storage unit, 50 Confirmation screen, 50a Instruction pointer, 51 Image display field, 52 Lighting color specification field, 53 Lighting pattern specification field, 54 Slider, 55 Shutter speed display field, 56 Back button, 57 decision button, B body, L lead, P component, S board.
Claims
1. An image display system that displays an image captured by an imaging device, comprising: an imaging control unit that sets one or more of a plurality of imaging conditions and causes the imaging device to capture an image of an object based on the imaging conditions; an image generation unit that uses an image captured by the imaging device to generate an image based on an imaging condition other than that set by the imaging control unit among the plurality of imaging conditions; and a display control unit that displays on a display device the image captured by the imaging device or generated by the image generation unit and the imaging conditions of the image so that a user can confirm the image.
2. The image display system according to claim 1, wherein the imaging control unit sets, as the one or more imaging conditions, a minimum imaging condition required for the image generation unit to generate all images based on the different imaging condition.
3. An image display system according to claim 1 or 2, further comprising a determination unit that determines imaging conditions for the imaging device to image an object based on a user's confirmation of the image displayed on the display device.
4. The image display system of claim 1 or 2, wherein the imaging conditions include conditions that define whether or not each of a plurality of lights provided for each of a plurality of lighting colors is turned on, and the imaging control unit sets, as the one or more imaging conditions, a first lighting pattern for each lighting color and for each lighting that turns on one of the plurality of lights, and sets a second lighting pattern for turning off all of the plurality of lights, and the image generation unit selects images necessary for generating an image based on a lighting color or on the presence or absence of lighting other than that set by the imaging control unit from the images captured with the first lighting pattern, adds up the luminance values of each pixel of the selected images, and subtracts the luminance value of each pixel of the images captured with the second lighting pattern by the number of images that is one less than the number of selected images, thereby calculating the luminance value of each pixel and generating an image.
5. The image display system of claim 1 or 2, wherein the imaging conditions include a shutter speed condition of the imaging device, the imaging control unit sets two or more different shutter speeds as the one or more imaging conditions, and the image generation unit sets a relational equation between the shutter speed and the luminance value for each pixel based on the luminance value of each pixel of the images captured at the two or more different shutter speeds, and generates an image by calculating, based on the relational equation, the luminance value of each pixel corresponding to a shutter speed other than that set by the imaging control unit.
6. An image display method for displaying an image captured by an imaging device, comprising: (a) a step of setting one or more of a plurality of imaging conditions and causing the imaging device to capture an image of an object based on the imaging conditions; (b) a step of generating an image based on an imaging condition other than the imaging condition set in step (a) from an image captured by the imaging device; and (c) a step of displaying on a display device the image captured by the imaging device or generated in step (b) and the imaging conditions of the image so that a user can confirm the image.
Citation Information
Patent Citations
Photographing apparatus
JP2006261912A
Imaging apparatus
JP2010171777A
Imaging device using compression sensing, imaging method, and imaging program
JP2017208641A
Image synthesis device, image acquisition device, image processing system, and image processing method
WO2020255365A1
