Clear image acquisition system, clear image acquisition method, and program

Abstract

To provide a clear image acquisition system that can acquire clear images of vibrating objects.SOLUTION: There is provided a clear image acquisition system according to an embodiment of the present invention. The clear image acquisition system includes: an imaging apparatus; and an information processing device having at least one processor. The imaging apparatus has: a camera; and a variable focus lens whose focus position can be adjusted. The processor is configured to execute a photographing step, an acquiring step, and a focus adjusting step. In the photographing step, multiple images of a vibrating object are acquired by the camera through the variable focus lens. In the acquiring step, Information on the vibrating object is acquired. In the focus adjusting step, the focus position of the variable focus lens is adjusted on the basis of the information.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a sharp image acquisition system, a sharp image acquisition method, and a program.

Background Art

[0002] In a manufacturing line, as disclosed in the following documents, a method of performing an appearance inspection of a product using an image obtained by photographing the product (workpiece) is known.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Vibration may exist in products on the manufacturing line. Therefore, in cases where a clear photographed image of the product cannot be obtained due to vibration, it is difficult to perform an appearance inspection by image processing.

[0005] In view of the above circumstances, the present invention aims to provide a sharp image acquisition system capable of acquiring a sharp image of a vibrating object.

Means for Solving the Problems

[0006] According to one aspect of the present invention, a sharp image acquisition system is provided. This sharp image acquisition system includes a photographing device and an information processing device having at least one processor. The photographing device has a camera and a variable focal length lens whose focal position is adjustable. The processor is configured to execute a photographing step, an acquisition step, and a focus adjustment step. In the photographing step, a plurality of images of a vibrating object are acquired by the camera through the variable focal length lens. In the acquisition step, information regarding the object during vibration is acquired. In the focus adjustment step, the focal position of the variable focal length lens is adjusted based on the information.

[0007] According to such an aspect, since the variable focal length lens can be automatically focused on the object, a sharp image can be efficiently acquired for a vibrating object.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Various characteristic matters shown in the embodiments described below can be combined with each other.

[0010] Incidentally, the program for realizing the software appearing in this embodiment may be provided as a non-transitory computer-readable medium that can be read by a computer, may be provided so as to be downloadable from an external server, or may be provided so that the program is started on an external computer and its function is realized on a client terminal (so-called cloud computing).

[0011] In addition, in this embodiment, the “unit” may include, for example, a hardware resource implemented by a circuit in a broad sense and an information process of software that can be specifically realized by these hardware resources. Also, in this embodiment, various information is handled, and these information are represented, for example, by physical values of signal values representing voltage and current, the high and low of signal values as a set of binary bits composed of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculation can be executed on a circuit in a broad sense.

[0012] In addition, a circuit in a broad sense is a circuit realized by appropriately combining at least a circuit, circuitry, a processor, a memory, etc. That is, it includes an application specific integrated circuit (ASIC), programmable logic devices (for example, a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0013] 1. Hardware Configuration In this section, the hardware configuration will be described.

[0014] <Clear Image Acquisition System 1> FIG. 1 is a configuration diagram showing the clear image acquisition system 1. The clear image acquisition system 1 includes a server device 2, a user terminal 3, and a photographing device 4. The server device 2 and the user terminal 3 are configured to be communicable through a telecommunication line. The server device 2 and the user terminal 3 are each an example of an information processing device. In one embodiment, the clear image acquisition system 1 is composed of one or more devices or components. For example, if the clear image acquisition system 1 consists only of the server device 2 or the user terminal 3, the clear image acquisition system 1 can be the server device 2 or the user terminal 3. Hereinafter, these components will be described.

[0015] <Server Device 2> FIG. 2 is a block diagram showing the hardware configuration of the server device 2. The server device 2 includes a communication bus 20, a communication unit 21, a storage unit 22, and a processor 23. The communication unit 21, the storage unit 22, and the processor 23 are electrically connected to each other inside the server device 2 via the communication bus 20.

[0016] <Communication unit 21> Although wired communication means such as USB, IEEE 1394, Thunderbolt (registered trademark), and wired LAN network communication are preferred for the communication unit 21, wireless LAN network communication, mobile communication such as 3G / LTE / 5G, and BLUETOOTH (registered trademark) communication may be included as necessary. That is, it is more preferable to implement as a set of these multiple communication means. That is, the server device 2 may communicate various information from the outside via the communication unit 21 and the network.

[0017] <Storage unit 22> The storage unit 22 stores various information defined as described above. This may be implemented as a storage device such as a solid state drive (SSD) that stores various programs related to the server device 2 executed by the processor 23, or as a memory such as a random access memory (RAM) that stores information (arguments, arrays, etc.) temporarily required for program operations. The storage unit 22 stores various programs, variables, etc. related to the server device 2 executed by the processor 23.

[0018] <Processor 23> The processor 23 performs processing and control of the overall operations related to the server device 2. The processor 23 is, for example, a central processing unit (CPU). The processor 23 realizes various functions related to the server device 2 by reading a predetermined program stored in the storage unit 22. That is, the information processing by software stored in the storage unit 22 is specifically realized by the processor 23, which is an example of hardware, and can be executed as each functional unit included in the processor 23. These will be described in more detail in the next section. Note that the processor 23 is not limited to being single, and may be implemented to have multiple processors 23 for each function. Or a combination thereof may also be used.

[0019] The server device 2 may be in an on-premises form or a cloud form. As the cloud-form server device 2, for example, in the form of SaaS (Software as a Service) or cloud computing, the above-described functions and processes may be provided.

[0020] <User terminal 3> FIG. 3 is a block diagram showing the hardware configuration of the user terminal 3. The user terminal 3 includes a communication bus 30, a communication unit 31, a storage unit 32, a processor 33, a display unit 34, and an input unit 35. The communication unit 31, the storage unit 32, the processor 33, the display unit 34, and the input unit 35 are electrically connected via the communication bus 30 inside the user terminal 3. The descriptions of the communication unit 31, the storage unit 32, and the processor 33 are omitted because they are the same as the descriptions of the respective units in the server device 2.

[0021] <Display unit 34> The display unit 34 displays a screen of a graphical user interface (GUI) operable by the user. The display unit 34 may be included in the housing of the user terminal 3 or may be externally attached. Specifically, the display unit 34 can be implemented as a display device such as a CRT display, a liquid crystal display, an organic EL display, or a plasma display. These display devices are preferably used and implemented according to the type of the user terminal 3.

[0022] <Input unit 35> The input unit 35 receives operation inputs made by the user. The operation inputs are transferred as command signals to the processor 33 via the communication bus 30. The processor 33 can execute predetermined controls and calculations based on the transferred command signals as necessary. The input unit 35 may be included in the housing of the user terminal 3 or may be externally attached. For example, the input unit 35 may be integrated with the display unit 34 and implemented as a touch panel. When the input unit 35 is implemented as a touch panel, the user can input tap operations, swipe operations, etc. to the input unit 35. As the input unit 35, instead of a touch panel, a switch button, a mouse, a QWERTY keyboard, etc. can be adopted.

[0023] FIG. 4 is a schematic diagram showing the configurations of the user terminal 3 and the imaging device 4. As shown in FIG. 4, the user terminal 3 may be composed of a personal computer 3A and a controller 3B. In this case, the processor of the personal computer 3A and the processor of the controller 3B may function as the processor 33 of the user terminal 3. Also, the user terminal 3 may be composed of only the personal computer 3A.

[0024] <Imaging device 4> As shown in FIG. 4, the imaging device 4 includes a camera 41, a plurality of lenses 42, a variable focal length lens 43, a light source 44, a mirror 45, and a displacement meter 46. Note that in the first and second embodiments of the functional configuration described later, the imaging device 4 does not necessarily have to have the displacement meter 46.

[0025] The camera 41 acquires a plurality of images of the object T that is the target placed on the vibration table VS (an example of the equipment that holds or transports the object T). The camera 41 is a high-speed camera (high-speed vision) incorporating an image sensor 41A.

[0026] A plurality of lenses 42 and a variable-focus lens 43 are arranged between the camera 41 and the object T (specifically, between the camera 41 and the mirror 45). The variable-focus lens 43 is a lens whose focal position is adjustable. Specifically, the variable-focus lens 43 is a liquid lens configured such that the focal position changes as the liquid interface changes according to the input voltage value. The input voltage value is adjusted by the user terminal 3 (specifically, the controller 3B).

[0027] The light source 44 irradiates the object T with light. The mirror 45 has a reflecting surface that reflects the reflected light from the object T toward the camera 41. As the mirror 45, for example, a two-axis rotating galvanometer mirror having two mirrors, an X-axis mirror and a Y-axis mirror, is used. However, the mirror 45 may be a fixed mirror.

[0028] The displacement meter 46 is configured to measure the position of the object T. Specifically, the displacement meter 46 measures the distance from a reference point to the vibrating object T. As the displacement meter 46, for example, a laser displacement meter is used.

[0029] 2. Functional Configuration In this section, the functional configuration of this embodiment will be described. Information processing by software stored in the storage unit 32 is specifically realized by a processor 33, which is an example of hardware, and can be executed as each functional unit included in the processor 33. This embodiment includes the first embodiment, the second embodiment, and the third embodiment. Hereinafter, the functions will be described for each of these embodiments. Note that in the first embodiment, the second embodiment, and the third embodiment, the hardware configuration is common.

[0030] <First Embodiment> FIG. 5 is a block diagram showing functions realized by the user terminal 3 (processor 33). In the first embodiment, the user terminal 3 (processor 33) includes a photographing unit 331, an acquisition unit 332, and a focus adjustment unit 333.

[0031] <Photographing Unit 331> The imaging unit 331 is configured to cause the camera 41 to acquire a plurality of images of the vibrating object T through the variable focal length lens 43. The vibration frequency of the object T is, for example, 10 Hz. The image acquisition speed (i.e., the shooting speed) of the camera 41 is on the order of several hundred Hz (for example, 800 - 900 Hz). Also, the resolution of the images handled by the acquisition unit 332 and the focus adjustment unit 333 is on the order of several hundred pixels × several hundred pixels (for example, 320 pixels × 240 pixels).

[0032] <Acquisition unit 332> The acquisition unit 332 is configured to acquire information regarding the object T during vibration (hereinafter, "object information"). Specifically, the acquisition unit 332 acquires, as the object information, the size of the object T in the latest first image among the plurality of images of the object T acquired by the camera 41. The size of the object T is, for example, derived from the contour shape of the object T in the image. Specifically, the size of the object T is defined by the radius of the minimum circumscribed circle of the contour of the object T, the circular conversion diameter of the contour shape of the object T (i.e., the diameter derived from the area when the area inside the contour is assumed to be the area of a perfect circle), the lengths in the vertical and horizontal directions of the contour shape, and the like.

[0033] More specifically, the acquisition unit 332 first stores the image of the object T acquired by the camera 41 in the memory as a monochrome image. Next, the acquisition unit 332 smooths the image by a known method to smooth the numerical change of adjacent pixels. After smoothing, the acquisition unit 332 binarizes each pixel to white (pixel value is 0) or black (pixel value is 255) using a threshold value. The threshold value here is, for example, any value between 30 and 60 (for example, 45), and pixels with a pixel value less than the threshold value are set to white, and pixels with a pixel value greater than or equal to the threshold value are set to black. Subsequently, the acquisition unit 332 extracts, from the binarized image, the contour that exists near the center of the angle of view and has the largest size as the contour of the object T. After extracting the contour, the acquisition unit 332 acquires the size (for example, the circumscribed circle diameter) of the figure defined by the extracted contour as the size of the object T.

[0034] When the vibration of the object T is three-dimensional (that is, when the object T also vibrates in a direction intersecting the shooting direction of the camera 41), the acquisition unit 332 further acquires, as object information, the movement amount of the object T in the first image and the second image acquired immediately before the first image (that is, one frame before the first image).

[0035] The movement amount of the object T is obtained, for example, as the distance between the center point of the object T in the first image and the center point of the object T in the second image. FIG. 6 is a conceptual diagram showing the movement of the center point of the object T between two images. In FIG. 6, the black circle represents the object T in the image (frame), and the dashed circle represents the object T in the image one frame before the current image. In FIG. 6A, the position (x k-1 , y k-1 ) of the center of the object T in the second image, which is the (k - 1)-th frame, and the movement amount l k-1 of the object T from the image before the second image are shown. In FIG. 6B, the position (x k , y k ) of the center of the object T in the first image, which is the k-th frame, and the movement amount l k of the object T from the second image are shown. The movement amount l k of the object T in the first image (k-th frame) is the distance between the center of the object T in the second image (k - 1)-th frame and the center of the object T in the first image.

[0036] The center of the object T can be, for example, the centroid of the contour of the object T extracted when calculating the size of the object T described above. The centroid of the contour can be obtained, for example, as the center point of the circumscribed circle of the contour. Further, the acquisition unit 332 may calculate the centroid of the contour using image moments.

[0037] <Focus adjustment unit 333> The focus adjustment unit 333 is configured to adjust the focus position of the variable focus lens 43 based on the object information acquired by the acquisition unit 332. The focus position adjustment process by the focus adjustment unit 333 is executed after one image is acquired by the imaging unit 331 and before the next image is acquired by the imaging unit 331. That is, the focus position adjustment is executed in real time in synchronization with the imaging of the object T.

[0038] Specifically, the focus adjustment unit 333 adjusts the focus position based on the size of the object T acquired by the acquisition unit 332 so that the focus of the variable focus lens 43 matches the object T. As a result, the in-focus position on the object T can be estimated with high accuracy and high speed, making it easier for the camera 41 to acquire a clear image of the vibrating object T.

[0039] Hereinafter, the focus position adjustment procedure by the focus adjustment unit 333 using the size of the object T will be described in detail. The focus adjustment unit 333 calculates the in-focus voltage value using the following formula (1), which is a relational expression between the size of the object T acquired by the acquisition unit 332, the reference voltage value which is the input voltage value immediately before in the variable focus lens 43, and the in-focus voltage value of the variable focus lens 43. The in-focus voltage value is the input voltage value when the focus of the variable focus lens 43 matches the object T. Also, the reference voltage value means the input voltage value to the variable focus lens 43 when the first image, which is the latest image of the object T, is acquired. Thereby, the focus of the variable focus lens 43 can be adjusted to match the object T with high accuracy and high speed.

Equation

[0040] In formula (1), V k+1 is the input voltage value (i.e., the in-focus voltage value) of the variable focus lens 43 when taking the next image (the (k + 1)-th frame). R k is the size of the object T (e.g., the radius of the circumscribed circle) in the first image (the k-th frame). In formulas (3) and (4), V kis the input voltage value of the variable focal length lens 43 at the time of shooting the first image. Further, K1, K2, K3, K4, and K5 in Expressions (2) to (4) are constants determined depending on the vibration condition of the object T, the definition of the size of the object T, parameters of the imaging device 4, and the like, respectively.

[0041] The above Expression (1) was derived by the following test using the sharp image acquisition system 1. First, the imaging device 4 was adjusted so that the distance of one pixel of the image was 0.02 mm. Further, as the object T, a black-painted disk with a diameter of 1 mm was used, and this object T was single-vibrated only in the Z direction (imaging direction). The center of vibration of the object T is the focal position of the variable focal length lens 43 in the initial state. Under this condition, while changing the input voltage value of the variable focal length lens 43 from 6V to 8V by 0.1V each time, the object T was imaged, and the circumscribed circle radius (unit: pixel) of the object T in the acquired image was measured every 9000 points. FIG. 7 is a graph showing the relationship between the input voltage value (horizontal axis) and the circumscribed circle radius (vertical axis) in the test. In FIG. 7, the minimum values of the circumscribed circle radius at the input voltage values of 6.0V, 7.0V, and 8.0V are plotted. As shown in FIG. 7, Expression (1) is derived by drawing an approximate curve based on the measurement results plotted for each input voltage value. The correlation coefficient of this approximate curve was 0.99. In the Expression (1) obtained by the test under the above conditions, K1, K2, K3, K4, and K5 in Expressions (2) to (4) are -4.836, 9.944, -1.077, -4.972, and 26.814, respectively.

[0042] The relational expression (Expression (1)) used by the focus adjustment unit 333 is a quadratic equation regarding the focusing voltage value (V k ) including a constant term (c - R k+1 ) calculated from the size of the object T and a coefficient (b) calculated from the reference voltage value. The solutions of the quadratic equation are two scalar values represented by the following Expressions (5) and (6).

Equation

[0043] The focus adjustment unit 333 selects, using the difference between each of the two solutions of the quadratic equation and the reference voltage value (the input voltage value immediately before in the variable focus lens 43), one of the two solutions as the focus voltage value to be used for photographing the next image. Thereby, the focusing accuracy of the variable focus lens 43 with respect to the object T can be improved.

[0044] Hereinafter, a detailed algorithm for selecting the focus voltage value from the two solutions of Equation (1) will be described. First, among the two solutions (voltage values), the one whose value falls within a certain range (for example, 6.0 to 8.0 V, which is the variation range of the input voltage value used when deriving Equation (1)) is selected. When both of the two solutions fall within this range, V of Equation (5), which is the larger of the two solutions, k+1H is tentatively selected as the focus voltage value.

[0045] Next, the following Equation (7) is used to determine whether the tentatively selected solution is appropriate. In Equation (7), ΔV kS is the difference between the focus voltage value V k+1S in the frame (image) to be acquired next and the input voltage value V k immediately before, and ΔV k-1S is the difference between the focus voltage value V kS in the previous frame and the input voltage value V k-1 . Also, S is the sign (H or L) selected in the previous frame, and S -1 is the sign not selected in the previous frame.

Equation

[0046] For example, when selecting V of Equation (5) as the focus voltage value of the frame to be acquired next, if the upper equation of Equation (7) is satisfied (that is, if the difference between the focus voltage value and the input voltage value is smaller than that in the previous frame), then V of Equation (5) k+1H is directly selected as the focus voltage value of the frame to be acquired next. On the other hand, when selecting V of Equation (5) as the focus voltage value of the frame to be acquired next, k+1H k+1H ​When [a certain option] is selected and the lower formula of formula (7) is satisfied (that is, when the difference between the focus voltage value and the input voltage value is greater than or equal to the difference in the previous frame), the V in formula (6) is used as the focus voltage value of the frame to be acquired next. k+1L Select [it].

[0047] Note that the procedure for selecting the focus voltage value from the two solutions of formula (1) is not limited to the above. For example, V in formula (5) and V in formula (6) can be alternately selected for each frame. k+1H and k+1L may be selected.

[0048] When the controller 3B is a DA converter, the command value of the input voltage value of the variable focus lens 43 from the personal computer 3A to the controller 3B needs to be an analog value. Therefore, for example, A represented by the following formula (8) is input to the controller 3B.

Equation

[0049] When the vibration of the object T is three-dimensional, the focus adjustment unit 333 adjusts the focus position based on the size and movement amount of the object T acquired by the acquisition unit 332 so that the focus of the variable focus lens 43 matches the object T. Thereby, for the object T vibrating three-dimensionally, the in-focus position to the object T can be estimated with high accuracy and high speed.

[0050] Hereinafter, the adjustment procedure of the focus position by the focus adjustment unit 333 using the size and movement amount of the object T will be described in detail. The focus adjustment unit 333 calculates the focus voltage value using the following formula (9), which is a relational expression between the size and movement amount of the object T acquired by the acquisition unit 332, the reference voltage value which is the previous input voltage value in the variable focus lens 43, and the focus voltage value of the variable focus lens 43.

Equation

[0051] In formula (9), Vk+1 is the input voltage value (i.e., the focusing voltage value) of the variable focal length lens 43 when taking the next acquired image (the (k + 1)-th frame). R k is the size (e.g., the circumradius) of the object T in the first image (the k-th frame). In equations (10)-(12), l k is the amount of movement of the center of the object T from the second image in the first image. In equations (11) and (12), V k is the input voltage value of the variable focal length lens 43 at the time of taking the first image. Also, K in equations (10)-(12) 11 -K 24 are constants determined depending on the vibration conditions of the object T, the definitions of the size and movement amount of the object T, the parameters of the imaging device 4, etc., respectively.

[0052] The above equation (9) was derived by the following test using the sharp image acquisition system 1. First, the imaging device 4 was adjusted under the same conditions as when deriving equation (1). Also, as the object T, a black-painted disk with a diameter of 1 mm was used, and this object T was vibrated three-dimensionally in the Z direction (imaging direction) and the XY direction. The center of vibration of the object T is the focal position of the variable focal length lens 43 in the initial state. Under this condition, while changing the input voltage value of the variable focal length lens 43 from 6V to 8V by 0.1V each time, the object T was imaged, and the circumradius (unit: pixel) and the center movement amount (unit: pixel) of the object T in the acquired images were measured 9000 points each.

[0053] FIG. 8 is a graph showing the relationship between the input voltage value (horizontal axis) and the circumradius (vertical axis) in the test. In FIG. 8, the minimum value of the circumradius and the center movement amounts of 0.0 pixel, 5.0 pixels, and 10.0 pixels at each input voltage value of 6.0V, 7.0V, and 8.0V are plotted. As shown in FIG. 8, for each input voltage value and movement amount, by drawing an approximate curve based on the plotted measurement results, equation (9) is derived. The correlation coefficient of this approximate curve was 0.99. In the equation (9) obtained by the test under the above conditions, K in equations (10)-(12) 11-K 24 are -0.0001, 0.214, -4.836, -0.332, 9.944, 0.011, -0.650, -1.077, 0.166, -4.972, -0.006, -0.034, 1.895, 26.814 respectively.

[0054] The relational expression (Equation (9)) used by the focus adjustment unit 333 is a quadratic equation regarding the focusing voltage value (V k ) that includes a constant term (c - R k+1 ) calculated from the size and movement amount of the object T and coefficients (a, b, c) calculated from the reference voltage value and the movement amount of the object T. The solutions of the quadratic equation are two scalar values represented by the following equations (13) and (14), similar to Equation (1).

Equation

[0055] The procedure for selecting the focusing voltage value from the two solutions of Equation (9) can be the same as in the case of Equation (1) described above.

[0056] <Processing Flow of the First Embodiment> Figure 9 is a flowchart showing the image acquisition process of the sharp image acquisition system 1 in the first embodiment. First, the user terminal 3 (processor 33) causes the camera 41 to acquire an initial image of the object T (step S110). The initial image is an image necessary for estimating the focusing voltage in the next image capture. Next, the user terminal 3 acquires the size of the object T (and further the movement amount of the object T if necessary) from the acquired latest image as object information (step S120).

[0057] After acquiring the object information, the user terminal 3 estimates the focusing voltage value using the object information (step S130). After estimating the focusing voltage value, the user terminal 3 sends the estimated focusing voltage value to the variable focus lens 43 as a command value of the input voltage value (step S140). As a result, the focal position of the variable focus lens 43 is changed to match the object T.

[0058] After transmitting the input voltage value, the user terminal 3 causes the camera 41 to capture an image of the object T (step S150). As a result, an image of the object T is acquired with the variable focal length lens 43 focused on the object T. After the image is captured, the user terminal 3 determines whether or not a shooting end condition is satisfied (step S160). If the shooting end condition is satisfied (step S160: YES), the user terminal 3 ends the image acquisition process. On the other hand, if the shooting end condition is not satisfied (step S160: NO), the user terminal 3 repeats each step from step S120 to step S150.

[0059] <Second Embodiment> In the second embodiment, the user terminal 3 (processor 33) includes a photographing unit 331, an acquisition unit 332, and a focus adjustment unit 333 shown in FIG. 5, similar to the first embodiment. The photographing unit 331 of the second embodiment has the same function as the photographing unit 331 of the first embodiment.

[0060] <Acquisition Unit 332> The acquisition unit 332 is configured to acquire object information. Specifically, the acquisition unit 332 acquires the amplitude and vibration frequency of the object T as object information. The amplitude and vibration frequency of the object T are acquired, for example, by actual measurement with a vibrometer for the vibrating object T, from the specifications of the vibration table VS, etc., and input to the user terminal 3. The acquisition unit 332 may receive the input of the amplitude and vibration frequency of the object T from the input unit 35, or may read the amplitude and vibration frequency of the object T stored in the storage unit 32 in advance.

[0061] <Focus Adjustment Unit 333> The focus adjustment unit 333 is configured to adjust the focus position of the variable focal length lens 43 based on the object information acquired by the acquisition unit 332. The focus position adjustment process by the focus adjustment unit 333 is continuously executed during the acquisition of a plurality of images by the photographing unit 331. That is, the focus position adjustment is executed in parallel with the shooting of the object T, asynchronously with the shooting timing of the object T.

[0062] Specifically, the focus adjustment unit 333 periodically changes the focus of the variable focus lens 43 based on the amplitude and vibration frequency of the object T acquired by the acquisition unit 332. Thereby, the variable focus lens 43 can be focused on the object T without performing a calculation process based on the acquired image of the object T.

[0063] Hereinafter, the procedure of the periodic change of the focus of the variable focus lens 43 will be described in detail. The focus adjustment unit 333 changes the input voltage value of the variable focus lens 43 with the voltage amplitude based on the amplitude of the object T and the voltage angular frequency based on the vibration frequency of the object T. The voltage amplitude is the amplitude of the input voltage value. The voltage angular frequency is the angular frequency of the input voltage value. Thereby, the timing at which the variable focus lens 43 focuses on the vibrating object T can be increased. As a result, the acquisition efficiency of a clear image of the object T is improved.

[0064] Time t k The change V of the input voltage value of the variable focus lens 43 at t is represented by, for example, the following formula (15). In formula (15), ΔV is the voltage amplitude. F0 is the vibration frequency of the object T, M is a constant multiplied by the frequency of the object T, and 2πMF0 / 1000 in formula (15) corresponds to the voltage angular frequency. V0 is the initial value (vibration center) of the input voltage value of the variable focus lens 43.

Equation

[0065] ΔV in formula (15) is obtained from the focus amplitude ΔL of the variable focus lens 43. The focus adjustment unit 333 sets a value larger than the amplitude of the object T (the distance from the vibration center to the outermost point of vibration) as the focus amplitude ΔL. For example, when the amplitude of the object T is 5 mm, the amplitude ΔL can be 7.2 mm. ΔV is obtained from this value of ΔL by the following formula (16). K in formula (16) 31is a constant determined depending on the characteristics of the variable focus lens 43, and takes a value, for example, of 0.01 or more and 0.1 or less.

Number

[0066] In Equation (16), for example, when K 31 is 0.0696 and ΔL is 7.2 mm, ΔV is approximately 0.50.

[0067] M in Equation (15) is, for example, a numerical value greater than 0 and less than or equal to 5. As the value of M, 0.20 or more and 0.30 or less is preferable. If M is less than 0.20, the time during which the focus of the variable focus lens 43 and the position of the object T deviate becomes long, so the acquisition frequency of a clear image tends to decrease. On the other hand, if M is greater than 0.30, the time during which the vibration direction (vector direction of vibration) of the focus of the variable focus lens 43 and the vibration direction of the object T do not match becomes long, so the acquisition frequency of a clear image tends to decrease. Note that the value of M does not affect the sharpness of the image.

[0068] When the controller 3B is a DA converter, the command value of the input voltage value of the variable focus lens 43 from the personal computer 3A to the controller 3B needs to be an analog value. Therefore, for example, A represented by the following Equation (17) is input to the controller 3B.

Number

[0069] The image of the object T is acquired at regular intervals regardless of the input voltage value of the variable focus lens 43 that changes periodically. Therefore, among the plurality of images of the object T acquired, there are images acquired at the timing when the variable focus lens 43 is in focus on the object T and images acquired at the timing when it is not in focus. Among these, the image acquired at the timing when the variable focus lens 43 is in focus on the object T becomes a candidate for a clear image.

[0070] <Processing Flow of the Second Embodiment> FIG. 10 is a flowchart showing the image acquisition process of the clear image acquisition system 1 in the second embodiment. First, the user terminal 3 (processor 33) acquires vibration information (amplitude and vibration frequency) of the object T (step S210). After acquiring the vibration information, the user terminal 3 starts the vibration of the focus of the variable focus lens 43 by periodically changing the input voltage value with the voltage amplitude and voltage angular frequency set based on the vibration information (step S220). While vibrating the focus of the variable focus lens 43, the user terminal 3 causes the camera 41 to acquire an image of the object T (step S230).

[0071] After acquiring the image, the user terminal 3 determines whether or not the shooting end condition is satisfied (step S240). If the shooting end condition is satisfied (step S240: YES), the user terminal 3 ends the vibration of the focus of the variable focus lens 43 (step S250) and ends the image acquisition process. On the other hand, if the shooting end condition is not satisfied (step S250: NO), the user terminal 3 repeats each of steps S220 and S230.

[0072] <Third Embodiment> In the third embodiment, the user terminal 3 (processor 33) includes a photographing unit 331, an acquisition unit 332, and a focus adjustment unit 333 shown in FIG. 5, similar to the first embodiment. The photographing unit 331 of the third embodiment has the same function as the photographing unit 331 of the first embodiment.

[0073] <Acquisition Unit 332> The acquisition unit 332 is configured to acquire object information. Specifically, the acquisition unit 332 acquires the position of the object T measured by the displacement meter 46 as object information. The acquisition unit 332 acquires the position of the object T output by the displacement meter 46 at a predetermined fixed interval.

[0074] <Focus Adjustment Unit 333> The focus adjustment unit 333 is configured to adjust the focal position of the variable focal lens 43 based on the object information acquired by the acquisition unit 332. The adjustment process of the focal position by the focus adjustment unit 333 is continuously executed during the acquisition of a plurality of images by the imaging unit 331. The adjustment of the focal position may be executed in synchronization with the imaging of the object T, or may be executed asynchronously with the imaging of the object T.

[0075] Specifically, the focus adjustment unit 333 adjusts the focal position based on the position of the object T acquired by the acquisition unit 332 so that the focus of the variable focal lens 43 matches the object T. Thereby, it is possible to obtain a clear image of the object T that vibrates in real time without performing a calculation process based on the acquired image of the object T.

[0076] Specifically, the focus adjustment unit 333 calculates a focusing voltage value based on the distance from the reference point to the object T. The reference point is, for example, the measurement reference of the displacement meter 46. Thereby, the focus of the variable focal lens 43 can be accurately and quickly adjusted to the object T.

[0077] The focusing voltage value V based on the distance to the object T kL is obtained by the following formula (18). In formula (18), L kL is the distance from the reference point to the object T. K 41 and K 42 are constants that are determined depending on the positions, postures, parameters, etc. of the variable focal lens 43 and the displacement meter 46, respectively. K 41 takes a value, for example, of 0.01 or more and 0.1 or less.

Equation

[0078] The measurement interval of the displacement meter 46 (that is, the acquisition interval of the position of the object T by the acquisition unit 332) is shorter than the acquisition interval of the images by the camera 41, for example, 100 μs. The focus adjustment unit 333 may adjust the focus of the variable focus lens 43 based on the measurement result of the displacement meter 46 at least immediately before the acquisition of the image of the object T by the camera 41. The adjustment of the focus of the variable focus lens 43 may be performed at regular intervals regardless of the acquisition timing of the image of the object T. That is, the focus adjustment unit 333 may adjust the focus of the variable focus lens 43 at an interval shorter than the shooting interval of the camera 41. For example, when the shooting speed of the camera 41 is 1 kHz, the adjustment speed (feedback speed) of the focus of the variable focus lens 43 may be 10 kHz.

[0079] <Processing Flow of the Third Embodiment> FIG. 11 is a flowchart showing the image acquisition process of the sharp image acquisition system 1 according to the third embodiment. First, the user terminal 3 (processor 33) acquires the position of the object T from the displacement meter 46 (step S310). After acquiring the position of the object T, the user terminal 3 calculates the focusing voltage value from the position of the object T (step S320).

[0080] After calculating the focusing voltage value, the user terminal 3 sends the calculated focusing voltage value as a command value of the input voltage value to the variable focus lens 43 (step S330). Thereby, the focal position of the variable focus lens 43 is changed to be adjusted to the object T.

[0081] After transmitting the input voltage value, the user terminal 3 causes the camera 41 to capture an image of the object T (step S340). Thereby, an image of the object T is acquired in a state where the variable focus lens 43 is focused on the object T. After the image is captured, the user terminal 3 determines whether or not the shooting end condition is satisfied (step S350). When the shooting end condition is satisfied (step S350: YES), the user terminal 3 ends the image acquisition process. On the other hand, when the shooting end condition is not satisfied (step S350: NO), the user terminal 3 repeats each step from step S310 to step S340.

[0082] 4. Function Summarizing the functions of this embodiment, they are as follows. That is, since the variable focus lens 43 can be automatically focused on the object T, a clear image can be efficiently obtained for the vibrating object T.

[0083] As described above, the embodiments of the present invention have been described. However, the present invention is not limited thereto and can be appropriately modified without departing from the technical idea of the invention.

[0084] 5. Others In the above embodiment, the user terminal 3 performs various storage and controls. However, instead of the user terminal 3, a plurality of external devices may be used. That is, various information and programs may be distributed and stored in a plurality of external devices using blockchain technology or the like.

[0085] The aspect of this embodiment is not limited to the clear image acquisition system 1, and may be an information processing method or a program. The clear image acquisition method includes each step of the clear image acquisition system 1. The program causes a computer to function as the information processing device of the clear image acquisition system 1.

[0086] In the clear image acquisition system 1, the processor 23 of the server device 2 may have the acquisition unit 332 and / or the focus adjustment unit 333. However, from the viewpoint of reducing the time lag between the image acquisition by the imaging device 4 and the focus adjustment of the variable focus lens 43 by the focus adjustment unit 333 (feedback to the imaging device 4), it is preferable that the acquisition unit 332 and the focus adjustment unit 333 are included in the processor 33 of the user terminal 3.

[0087] It may also be provided in each of the aspects described below.

[0088] (1) A sharp image acquisition system comprising a photographing device and an information processing device having at least one processor, wherein the photographing device has a camera and a variable focal length lens whose focal position is adjustable, and the processor is configured to execute a photographing step, an acquisition step, and a focus adjustment step. In the photographing step, a plurality of images of a vibrating object are acquired by the camera through the variable focal length lens. In the acquisition step, information about the object during vibration is acquired. In the focus adjustment step, the focal position of the variable focal length lens is adjusted based on the information.

[0089] (2) In the sharp image acquisition system according to (1) above, in the acquisition step, as the information, the size of the object in the latest first image among the plurality of images is acquired, and in the focus adjustment step, the focal position is adjusted based on the size so that the focus of the variable focal length lens matches the object.

[0090] (3) In the sharp image acquisition system according to (2) above, the variable focal length lens is configured such that the focal position changes according to an input voltage value. In the focus adjustment step, a relational expression among the size, a reference voltage value which is the input voltage value immediately before in the variable focal length lens, and a focusing voltage value of the variable focal length lens is used to calculate the focusing voltage value. Here, the focusing voltage value is the input voltage value when the focus of the variable focal length lens matches the object.

[0091] (4) In the sharp image acquisition system according to (3) above, the relational expression is a quadratic equation regarding the focusing voltage value including a constant term calculated from the size and a coefficient calculated from the reference voltage value. In the focus adjustment step, one of the two solutions is selected as the focusing voltage value using the difference between each of the two solutions of the quadratic equation and the reference voltage value.

[0092] (5) In the sharp image acquisition system according to any one of (2) to (4) above, in the acquisition step, as the information, the amount of movement of the object in the first image and the second image acquired immediately before the first image is further acquired, and in the focus adjustment step, based on the size and the amount of movement, the focus position is adjusted so that the focus of the variable focus lens matches the object. A sharp image acquisition system.

[0093] (6) In the sharp image acquisition system according to (1) above, in the acquisition step, as the information, the amplitude and the vibration frequency of the object are acquired, and in the focus adjustment step, based on the amplitude and the vibration frequency, the focus of the variable focus lens is periodically changed. A sharp image acquisition system.

[0094] (7) In the sharp image acquisition system according to (6) above, the variable focus lens is configured such that the focus position changes according to the input voltage value, and in the focus adjustment step, the input voltage value is changed by the voltage amplitude based on the amplitude and the voltage angular frequency based on the vibration frequency. A sharp image acquisition system.

[0095] (8) In the sharp image acquisition system according to (1) above, the imaging device further includes a displacement meter configured to measure the position of the object, and in the acquisition step, as the information, the position of the object measured by the displacement meter is acquired, and in the focus adjustment step, based on the position of the object, the focus position is adjusted so that the focus of the variable focus lens matches the object. A sharp image acquisition system.

[0096] (9) In the sharp image acquisition system according to (8) above, the variable focus lens is configured such that the focus position changes according to the input voltage value, and in the focus adjustment step, a focusing voltage value is calculated based on the distance from the reference point to the object, where the focusing voltage value is the input voltage value when the focus of the variable focus lens matches the object. A sharp image acquisition system.

[0097] (10) A sharp image acquisition method, comprising each step of the sharp image acquisition system according to any one of (1) to (9) above.

[0098] (11) A program for causing a computer to function as the information processing apparatus of the sharp image acquisition system according to any one of (1) to (9) above. Of course, this is not the limit.

[0099] Finally, although various embodiments according to the present disclosure have been described, these are presented as examples and are not intended to limit the scope of the invention. The novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. The embodiments and their modifications are included in the scope and gist of the invention, and are included in the invention described in the claims and its equivalent scope.

Explanation of Reference Numerals

[0100] 1: Sharp image acquisition system 2: Server device 3: User terminal 3A: Personal computer 3B: Controller 4: Imaging device 20: Communication bus 21: Communication unit 22: Storage unit 23: Processor 30: Communication bus 31: Communication unit 32: Storage unit 33: Processor 34: Display unit 35: Input unit 41: Camera 41A: Image sensor 42: Lens 43: Variable focus lens 44: Light source 45: Mirror 46: Displacement meter 331: Imaging unit 332: Acquisition unit 333: Focus adjustment unit T: Object VS: Shaking table

Claims

1. A sharp image acquisition system, including a photographing device, and an information processing device having at least one processor, wherein the photographing device includes a camera, and a variable focus lens whose focal position is adjustable, and the processor is configured to execute a photographing step, an acquisition step, and a focus adjustment step, in the photographing step, a plurality of images of a vibrating object are acquired by the camera through the variable focus lens, in the acquisition step, information about the object during vibration is acquired, and in the focus adjustment step, the focal position of the variable focus lens is adjusted based on the information. A sharp image acquisition system.

2. In the sharp image acquisition system according to Claim 1, in the acquisition step, as the information, the size of the object in the latest first image among the plurality of images is acquired, and in the focus adjustment step, the focal position is adjusted based on the size so that the focus of the variable focus lens matches the object. A sharp image acquisition system.

3. In the sharp image acquisition system according to Claim 2, the variable focus lens is configured such that the focal position changes according to an input voltage value, and in the focus adjustment step, a relational expression among the size, a reference voltage value which is the immediately preceding input voltage value in the variable focus lens, and a focusing voltage value of the variable focus lens is used to calculate the focusing voltage value, where the focusing voltage value is the input voltage value when the focus of the variable focus lens matches the object. A sharp image acquisition system.

4. In the sharp image acquisition system according to claim 3, The relational expression is a quadratic equation regarding the focusing voltage value including a constant term calculated from the size and a coefficient calculated from the reference voltage value, In the focus adjustment step, using the difference between each of the two solutions of the quadratic equation and the reference voltage value, one of the two solutions is selected as the focusing voltage value. A sharp image acquisition system.

5. In the sharp image acquisition system according to claim 2, In the acquisition step, as the information, the amount of movement of the object in the first image and the second image acquired immediately before the first image is further acquired, In the focus adjustment step, based on the size and the amount of movement, the focus position is adjusted so that the focus of the variable focus lens matches the object. A sharp image acquisition system.

6. In the sharp image acquisition system according to claim 1, In the acquisition step, as the information, the amplitude and the vibration frequency of the object are acquired, In the focus adjustment step, based on the amplitude and the vibration frequency, the focus of the variable focus lens is periodically changed. A sharp image acquisition system.

7. In the sharp image acquisition system according to claim 6, The variable focus lens is configured such that the focus position changes according to an input voltage value, In the focus adjustment step, the input voltage value is changed by a voltage amplitude based on the amplitude and a voltage angular frequency based on the vibration frequency. A sharp image acquisition system.

8. In the sharp image acquisition system according to claim 1, The imaging device further includes a displacement meter configured to measure the position of the object, In the acquisition step, as the information, the position of the object measured by the displacement meter is acquired, In the focus adjustment step, a sharp image acquisition system that adjusts the focal position so that the focus of the variable focal length lens matches the object based on the position of the object.

9. In the sharp image acquisition system according to claim 8, the variable focal length lens is configured such that the focal position changes according to an input voltage value, in the focus adjustment step, a focusing voltage value is calculated based on the distance from a reference point to the object, where the focusing voltage value is the input voltage value when the focus of the variable focal length lens matches the object, a sharp image acquisition system.

10. A sharp image acquisition method, comprising: each step of the sharp image acquisition system according to any one of claims 1 to 9, a sharp image acquisition method.

11. A program for causing a computer to function as the information processing device of the sharp image acquisition system according to any one of claims 1 to 9. ​

Images