Clear image acquisition system, clear image acquisition method, and program

Abstract

To provide a clear image acquisition system capable of acquiring a clear image of a vibrating object. According to one aspect of the present invention, a clear image capture system is provided. The clear image capture system includes a photographing device and an information processing device having at least one processor. The photographing device includes a camera and a variable-focus lens with an adjustable focus position. The processor is configured to execute a photographing step, an acquisition step, and a focus adjustment step. In the photographing step, the camera is caused to acquire multiple images of a vibrating object through the variable-focus lens. In the acquisition step, information about the vibrating object is acquired. In the focus adjustment step, the focus position of the variable-focus lens is adjusted based on the information.

Description

[Technical field]

[0001] The present invention relates to a clear image acquisition system, a clear image acquisition method, and a program. [Background technology]

[0002] 2. Description of the Related Art In a manufacturing line, a method is known in which an appearance inspection of a product (workpiece) is performed using an image of the product, as disclosed in the following document. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-47442 Summary of the Invention [Problem to be solved by the invention]

[0004] Products on a production line may be subject to vibrations, which can make it difficult to obtain clear images of the product, making it difficult to perform appearance inspections using image processing.

[0005] In view of the above circumstances, the present invention provides a clear image acquisition system capable of acquiring a clear image of a vibrating object. [Means for solving the problem]

[0006] According to one aspect of the present invention, a clear image acquisition system is provided. The clear image acquisition system includes a photographing device and an information processing device having at least one processor. The photographing device includes a camera and a variable-focus lens with an adjustable focus position. The processor is configured to execute a photographing step, an acquisition step, and a focus adjustment step. In the photographing step, the camera is caused to acquire a plurality of images of a vibrating object through the variable-focus lens. In the acquisition step, information about the vibrating object is acquired. In the focus adjustment step, the focus position of the variable-focus lens is adjusted based on the information.

[0007] According to this aspect, the variable-focus lens can be automatically focused on the object, so that a clear image of the vibrating object can be efficiently obtained. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a configuration diagram showing a clear image acquisition system 1. [Diagram 2] FIG. 2 is a block diagram showing a hardware configuration of the server device 2. [Diagram 3] FIG. 2 is a block diagram showing a hardware configuration of a user terminal 3. [Figure 4] 2 is a schematic diagram showing the configuration of a user terminal 3 and an imaging device 4. FIG. [Diagram 5] FIG. 2 is a block diagram showing functions realized by a user terminal 3 (processor 33). [Figure 6] FIG. 2 is a conceptual diagram showing the movement of the center point of an object T between two images. [Figure 7] 1 is a graph showing the relationship between the input voltage value (horizontal axis) and the circumscribed circle radius (vertical axis) in a test. [Figure 8] 1 is a graph showing the relationship between the input voltage value (horizontal axis) and the circumscribed circle radius (vertical axis) in a test. [Figure 9] 4 is a flowchart showing an image acquisition process of the clear image acquisition system 1 in the first embodiment. FIG. [Figure 10] FIG. 11 is a flowchart showing an image acquisition process of the clear image acquisition system 1 in the second embodiment. [Figure 11] FIG. 11 is a flowchart showing an image acquisition process of the clear image acquisition system 1 in the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the drawings. Various characteristic features shown in the following embodiments can be combined with each other.

[0010] Incidentally, the program for realizing the software appearing in this embodiment may be provided as a non-transitory computer-readable recording medium, or may be provided so as to be downloadable from an external server, or may be provided so that the program is started on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0011] In addition, in this embodiment, the term "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In addition, in this embodiment, various information is handled, and this information is represented, for example, by physical values ​​of signal values ​​representing voltage and current, high and low signal values ​​as a binary bit collection consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculation can be performed on the circuit in the broad sense.

[0012] In addition, a circuit in the broad sense is a circuit realized by at least appropriately combining a circuit, circuitry, a processor, a memory, etc. In other words, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0013] 1. Hardware Configuration This section describes the hardware configuration.

[0014] <Clear Image Acquisition System 1> FIG. 1 is a configuration diagram showing a clear image acquisition system 1. The clear image acquisition system 1 includes a server device 2, a user terminal 3, and a photographing device 4. The server device 2 and the user terminal 3 are configured to be able to communicate with each other through an electric communication line. The server device 2 and the user terminal 3 are each an example of an information processing device. In one embodiment, the clear image acquisition system 1 is composed of one or more devices or components. For example, if the clear image acquisition system 1 is composed of only the server device 2 or the user terminal 3, the clear image acquisition system 1 can be the server device 2 or the user terminal 3. These components will be described below.

[0015] <Server device 2> 2 is a block diagram showing a hardware configuration of the server device 2. The server device 2 includes a communication bus 20, a communication unit 21, a storage unit 22, and a processor 23. The communication unit 21, the storage unit 22, and the processor 23 are electrically connected via the communication bus 20 inside the server device 2.

[0016] <Communications Division 21> The communication unit 21 is preferably a wired communication means such as USB, IEEE1394, Thunderbolt (registered trademark), wired LAN network communication, etc., but may also include wireless LAN network communication, mobile communication such as 3G / LTE / 5G, BLUETOOTH (registered trademark) communication, etc. as necessary. In other words, it is more preferable to implement it as a collection of multiple communication means. In other words, the server device 2 may communicate various information from the outside via the communication unit 21 and the network.

[0017] <Storage section 22> The storage unit 22 stores various information defined by the above description. This can be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs and the like related to the server device 2 executed by the processor 23, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to the calculation of the program. The storage unit 22 stores various programs, variables, etc. related to the server device 2 executed by the processor 23.

[0018] <Processor 23> The processor 23 performs processing and control of the overall operations related to the server device 2. The processor 23 is, for example, a central processing unit (CPU). The processor 23 realizes various functions related to the server device 2 by reading out a predetermined program stored in the storage unit 22. That is, information processing by software stored in the storage unit 22 can be specifically realized by the processor 23, which is an example of hardware, and executed as each functional unit included in the processor 23. These will be described in more detail in the next section. The processor 23 is not limited to being single, and may be implemented by having multiple processors 23 for each function. Also, a combination of these may be used.

[0019] The server device 2 may be in an on-premise form or in a cloud form. As the server device 2 in a cloud form, the above-mentioned functions and processes may be provided in the form of, for example, SaaS (Software as a Service) or cloud computing.

[0020] <User terminal 3> 3 is a block diagram showing a hardware configuration of the user terminal 3. The user terminal 3 includes a communication bus 30, a communication unit 31, a storage unit 32, a processor 33, a display unit 34, and an input unit 35. The communication unit 31, the storage unit 32, the processor 33, the display unit 34, and the input unit 35 are electrically connected via the communication bus 30 inside the user terminal 3. The description of the communication unit 31, the storage unit 32, and the processor 33 will be omitted because they are similar to the description of each unit in the server device 2.

[0021] <Display section 34> The display unit 34 displays a screen of a graphical user interface (GUI) that can be operated by the user. The display unit 34 may be included in the housing of the user terminal 3, or may be attached externally. Specifically, the display unit 34 may be implemented as a display device such as a CRT display, a liquid crystal display, an organic EL display, or a plasma display. It is preferable that these display devices are implemented by selectively using them according to the type of the user terminal 3.

[0022] <Input section 35> The input unit 35 accepts an operation input made by a user. The operation input is transferred as a command signal to the processor 33 via the communication bus 30. The processor 33 may execute a predetermined control or calculation based on the transferred command signal as necessary. The input unit 35 may be included in the housing of the user terminal 3, or may be externally attached. For example, the input unit 35 may be implemented as a touch panel integrated with the display unit 34. When the input unit 35 is implemented as a touch panel, the user can input a tap operation, a swipe operation, or the like to the input unit 35. As the input unit 35, a switch button, a mouse, a QWERTY keyboard, or the like can be adopted instead of a touch panel.

[0023] Fig. 4 is a schematic diagram showing the configurations of the user terminal 3 and the photographing device 4. As shown in Fig. 4, the user terminal 3 may be composed of a personal computer 3A and a controller 3B. In this case, the processor of the personal computer 3A and the processor of the controller 3B may function as the processor 33 of the user terminal 3. The user terminal 3 may also be composed of only the personal computer 3A.

[0024] <Camera Unit 4> 4, the image capturing device 4 includes a camera 41, a plurality of lenses 42, a variable-focus lens 43, a light source 44, a mirror 45, and a displacement meter 46. In the first and second embodiments of the functional configurations described below, the image capturing device 4 does not necessarily have to include the displacement meter 46.

[0025] The camera 41 captures multiple images of a target object T placed on a vibration table VS (an example of equipment for holding or transporting the object T). The camera 41 is a high-speed camera (high-speed vision) with a built-in image sensor 41A.

[0026] The multiple lenses 42 and the variable-focus lens 43 are disposed between the camera 41 and the object T (specifically, between the camera 41 and the mirror 45). The variable-focus lens 43 is a lens whose focal position is adjustable. Specifically, the variable-focus lens 43 is a liquid lens configured such that the focal position changes as the liquid interface changes depending on an input voltage value. The input voltage value is adjusted by the user terminal 3 (specifically, the controller 3B).

[0027] The light source 44 irradiates light onto the object T. The mirror 45 has a reflective surface that reflects the light reflected from the object T toward the camera 41. As the mirror 45, for example, a two-axis rotating galvanometer mirror having two mirrors, an X-axis mirror and a Y-axis mirror, is used. However, the mirror 45 may be a fixed mirror.

[0028] The displacement meter 46 is configured to measure the position of the object T. Specifically, the displacement meter 46 measures the distance from a reference point to the vibrating object T. As the displacement meter 46, for example, a laser displacement meter is used.

[0029] 2. Functional configuration In this section, the functional configuration of this aspect will be described. Information processing by software stored in the storage unit 32 is specifically realized by the processor 33, which is an example of hardware, and can be executed as each functional unit included in the processor 33. This aspect includes a first embodiment, a second embodiment, and a third embodiment. Below, the functions of each of these embodiments will be described. Note that the hardware configuration is common to the first embodiment, the second embodiment, and the third embodiment.

[0030] First Embodiment 5 is a block diagram showing functions realized by the user terminal 3 (processor 33). In the first embodiment, the user terminal 3 (processor 33) includes an image capturing unit 331, an acquisition unit 332, and a focus adjustment unit 333.

[0031] <Camera Division 331> The photographing unit 331 is configured to cause the camera 41 to acquire multiple images of the vibrating object T through the variable-focus lens 43. The vibration frequency of the object T is, for example, 10 Hz. The image acquisition speed (i.e., the photographing speed) of the camera 41 is on the order of several hundred Hz (for example, 800-900 Hz). Furthermore, the resolution of the images handled by the acquisition unit 332 and the focus adjustment unit 333 is on the order of several hundred pixels x several hundred pixels (for example, 320 pixels x 240 pixels).

[0032] <Acquisition part 332> The acquisition unit 332 is configured to acquire information about the vibrating object T (hereinafter, "object information"). Specifically, the acquisition unit 332 acquires, as the object information, the size of the object T in the latest first image among a plurality of images of the object T acquired by the camera 41. The size of the object T is derived, for example, from the contour shape of the object T in the image. Specifically, the size of the object T is defined by the radius of the smallest circumscribing circle of the contour of the object T, the circle equivalent diameter of the contour shape of the object T (that is, the diameter derived from the area when the area inside the contour is assumed to be the area of ​​a perfect circle), the vertical and horizontal lengths of the contour shape, and the like.

[0033] More specifically, the acquisition unit 332 first stores the image of the object T acquired by the camera 41 in a memory as a monochrome image. Next, the acquisition unit 332 smoothes the image by a known method to smooth the numerical change of adjacent pixels. After the smoothing, the acquisition unit 332 binarizes each pixel to white (pixel value 0) or black (pixel value 255) using a threshold value. The threshold value here is, for example, an arbitrary value between 30 and 60 (for example, 45), and pixels whose pixel values ​​are less than the threshold value are white, and pixels whose pixel values ​​are equal to or greater than the threshold value are black. Next, the acquisition unit 332 extracts, from the binarized image, a contour that exists near the center of the angle of view and has the largest size as the contour of the object T. After extracting the contour, the acquisition unit 332 acquires the size of a figure defined by the extracted contour (for example, the circumscribed circle diameter) as the size of the object T.

[0034] When the vibration of object T is three-dimensional (i.e., when object T also vibrates in a direction intersecting the shooting direction of camera 41), acquisition unit 332 further acquires, as object information, the amount of movement of object T between the first image and the second image acquired immediately before the first image (i.e., one frame before the first image).

[0035] The amount of movement of object T is obtained, for example, as the distance between the center point of object T in the first image and the center point of object T in the second image. FIG. 6 is a conceptual diagram showing the movement of the center point of object T between two images. In FIG. 6, a black circle indicates object T in the image (frame), and a dashed circle indicates object T in the image immediately preceding the image. FIG. 6A shows the position (x k-1 ,y k-1 ) and the displacement of object T from the previous image in the second image, l k-1 FIG. 6B shows the position of the center of object T in the first image, which is the kth frame (x k ,y k ) and the displacement of object T from the second image l k The amount of movement l of object T in the first image (kth frame) is k is the distance between the center of object T in the second image (the k-1th frame) and the center of object T in the first image.

[0036] The center of object T may be, for example, the center of gravity of the contour of object T extracted when calculating the size of object T described above. The center of gravity of the contour is obtained, for example, as the center point of the circumscribing circle of the contour. The acquisition unit 332 may also calculate the center of gravity of the contour using an image moment.

[0037] <Focus adjustment section 333> The focus adjustment unit 333 is configured to adjust the focus position of the variable-focus lens 43 based on the object information acquired by the acquisition unit 332. The adjustment process of the focus position by the focus adjustment unit 333 is executed after the image capture unit 331 captures one image and before the image capture unit 331 captures the next image. In other words, the adjustment of the focus position is executed in real time in synchronization with the image capture of the object T.

[0038] Specifically, the focus adjustment unit 333 adjusts the focal position of the variable-focus lens 43 based on the size of the object T acquired by the acquisition unit 332 so that the focal position of the variable-focus lens 43 is aligned with the object T. This makes it possible to estimate the focusing position on the object T with high accuracy and high speed, making it easier for the camera 41 to acquire a clear image of the vibrating object T.

[0039] The procedure for adjusting the focal position by the focus adjustment unit 333 using the size of the object T will be described in detail below. The focus adjustment unit 333 calculates a focusing voltage value using the following formula (1) which is a relational expression between the size of the object T acquired by the acquisition unit 332, a reference voltage value which is the immediately preceding input voltage value in the varifocal lens 43, and a focusing voltage value of the varifocal lens 43. The focusing voltage value is the input voltage value when the varifocal lens 43 is focused on the object T. The reference voltage value means the input voltage value to the varifocal lens 43 when the first image which is the latest image of the object T is acquired. This allows the focus of the varifocal lens 43 to be adjusted to the object T with high accuracy and high speed.

number

[0040] In formula (1), V k+1 is the input voltage value (i.e., the focusing voltage value) of the variable-focus lens 43 when capturing the next image (the (k+1)th frame). R k is the size (e.g., the circumscribing circle radius) of the object T in the first image (the kth frame). In equations (3) and (4), V kis the input voltage value of the variable-focus lens 43 when the first image is captured. In addition, K1, K2, K3, K4, and K5 in equations (2)-(4) are constants that are determined depending on the vibration conditions of object T, the definition of the size of object T, the parameters of the image capture device 4, etc.

[0041] The above formula (1) was derived from the following test using the clear image acquisition system 1. First, the image capture device 4 was adjusted so that the distance of one pixel in the image was 0.02 mm. In addition, a black-painted disk with a diameter of 1 mm was used as the object T, and the object T was subjected to simple harmonic motion only in the Z direction (photographing direction). The center of vibration of the object T is the focal position of the varifocal lens 43 in the initial state. Under these conditions, the object T was photographed while changing the input voltage value of the varifocal lens 43 by 0.1 V between 6 V and 8 V, and the circumscribed circle radius (unit: pixel) of the object T in the acquired image was measured at 9000 points. FIG. 7 is a graph showing the relationship between the input voltage value (horizontal axis) and the circumscribed circle radius (vertical axis) in the test. In FIG. 7, the minimum value of the circumscribed circle radius at each input voltage value of 6.0 V, 7.0 V, and 8.0 V is plotted. As shown in FIG. 7, the formula (1) is derived by drawing an approximation curve based on the measurement results plotted for each input voltage value. The correlation coefficient of this approximation curve was 0.99. In addition, in the formula (1) obtained by the test under the above-mentioned conditions, K1, K2, K3, K4, and K5 in the formulas (2) to (4) are −4.836, 9.944, −1.077, −4.972, and 26.814, respectively.

[0042] The relational expression (expression (1)) used by the focus adjustment unit 333 includes a constant term (cR k ) and the focusing voltage value (V k+1 ) The solution of this quadratic equation is two scalar values ​​expressed by the following equations (5) and (6).

number

[0043] The focus adjustment unit 333 uses the difference between each of the two solutions of the quadratic equation and a reference voltage value (the immediately preceding input voltage value in the varifocal lens 43) to select one of the two solutions as a focus voltage value to be used for capturing the next image. This makes it possible to improve the focus accuracy of the varifocal lens 43 on the object T.

[0044] Below, we will explain the detailed algorithm for selecting the focusing voltage value from the two solutions of equation (1). First, of the two solutions (voltage values), the one whose value falls within a certain range (for example, 6.0 to 8.0 V, which is the variation range of the input voltage value used when deriving equation (1)) is selected. If both of the two solutions fall within this range, the larger of the two solutions, V in equation (5), is selected. k+1H is provisionally selected as the focusing voltage value.

[0045] Next, the following equation (7) is used to determine whether the provisionally selected solution is appropriate. In equation (7), ΔV kS is the focusing voltage value V in the frame (image) to be acquired. k+1S and the previous input voltage value V k Difference with ΔV k-1S is the focusing voltage value V in the previous frame kS and the input voltage value V k-1 S is the code (H or L) selected in the previous frame, and S -1 is a code that was not selected in the previous frame.

number

[0046] For example, the focus voltage value of the frame to be acquired is V in Equation (5). k+1H When the upper part of equation (7) is satisfied (i.e., the difference between the focus voltage value and the input voltage value is smaller than that of the previous frame), the focus voltage value of the frame to be acquired is set to V in equation (5). k+1H On the other hand, we will select V in equation (5) as the focusing voltage value for the frame to be acquired. k+1HWhen the lower part of equation (7) is satisfied (i.e., the difference between the focus voltage value and the input voltage value is equal to or greater than the difference of the previous frame), the focus voltage value of the frame to be acquired is set to V in equation (6). k+1L Select .

[0047] The procedure for selecting the focusing voltage value from the two solutions of equation (1) is not limited to the above. For example, V k+1H and V in Eq. (6) k+1L You may also select

[0048] When the controller 3B is a DA converter, the command value for the input voltage value of the variable-focal-length lens 43 from the personal computer 3A to the controller 3B needs to be an analog value. Therefore, for example, A expressed by the following formula (8) is input to the controller 3B.

number

[0049] When the vibration of the object T is three-dimensional, the focus adjustment unit 333 adjusts the focal position of the variable-focus lens 43 so that the focal point of the object T is aligned with the object T, based on the size and movement amount of the object T acquired by the acquisition unit 332. This makes it possible to estimate the focal position of the object T with high accuracy and high speed for the object T vibrating three-dimensionally.

[0050] The following describes in detail the procedure for adjusting the focal position by the focus adjustment unit 333 using the size and movement amount of the object T. The focus adjustment unit 333 calculates a focusing voltage value using the size and movement amount of the object T acquired by the acquisition unit 332, a reference voltage value which is the immediately preceding input voltage value in the varifocal lens 43, and the following equation (9) which is a relational equation between the focusing voltage value of the varifocal lens 43.

number

[0051] In formula (9), Vk+1 is the input voltage value (i.e., the focusing voltage value) of the variable-focus lens 43 when capturing the next image (the (k+1)th frame). R k is the size (e.g., circumscribing circle radius) of object T in the first image (kth frame). In equations (10)-(12), l k is the displacement of the center of object T in the first image from the second image. k is the input voltage value of the variable-focus lens 43 when the first image is captured. 11 -K 24 are constants that are determined depending on the vibration conditions of object T, the definition of the size and amount of movement of object T, the parameters of the image capture device 4, etc.

[0052] The above formula (9) was derived from the following test using the clear image acquisition system 1. First, the image capture device 4 was adjusted to the same conditions as when formula (1) was derived. In addition, a black-painted disk with a diameter of 1 mm was used as the object T, and this object T was three-dimensionally vibrated in the Z direction (image capture direction) and the XY directions. The center of vibration of the object T is the focal position of the varifocal lens 43 in the initial state. Under these conditions, the object T was photographed while changing the input voltage value of the varifocal lens 43 between 6V and 8V in increments of 0.1V, and the circumscribed circle radius (unit: pixel) and center movement amount (unit: pixel) of the object T in the acquired image were measured at 9000 points each.

[0053] FIG. 8 is a graph showing the relationship between the input voltage value (horizontal axis) and the circumscribed circle radius (vertical axis) in the test. In FIG. 8, the minimum circumscribed circle radius and the center shift amount of 0.0 pixel, 5.0 pixel, and 10.0 pixel are plotted for each input voltage value of 6.0 V, 7.0 V, and 8.0 V. As shown in FIG. 8, equation (9) is derived by drawing an approximation curve based on the plotted measurement results for each input voltage value and shift amount. The correlation coefficient of this approximation curve was 0.99. Note that in equation (9) obtained from the test under the above-mentioned conditions, K in equations (10)-(12) 11-K 24 are -0.0001, 0.214, -4.836, -0.332, 9.944, 0.011, -0.650, -1.077, 0.166, -4.972, -0.006, -0.034, 1.895, and 26.814, respectively.

[0054] The relational expression (equation (9)) used by the focus adjustment unit 333 includes a constant term (cR k ) and the focus voltage value (V k+1 ) The solution of this quadratic equation is two scalar values ​​expressed by the following equations (13) and (14), similar to equation (1).

number

[0055] The procedure for selecting the focus voltage value from the two solutions of equation (9) can be similar to that for equation (1) above.

[0056] <Processing flow of the first embodiment> 9 is a flow diagram showing an image acquisition process of the clear image acquisition system 1 in the first embodiment. First, the user terminal 3 (processor 33) causes the camera 41 to acquire an initial image of the object T (step S110). The initial image is an image required to estimate the focus voltage for capturing the next image. Next, the user terminal 3 acquires the size of the object T (and the amount of movement of the object T, if necessary) from the most recently acquired image as object information (step S120).

[0057] After acquiring the object information, the user terminal 3 estimates a focusing voltage value using the object information (step S130). After estimating the focusing voltage value, the user terminal 3 sends the estimated focusing voltage value as a command value for the input voltage value to the variable-focus lens 43 (step S140). As a result, the focal position of the variable-focus lens 43 is changed to match the object T.

[0058] After transmitting the input voltage value, the user terminal 3 causes the camera 41 to capture an image of the object T (step S150). As a result, an image of the object T is acquired with the variable-focus lens 43 focused on the object T. After capturing the image, the user terminal 3 determines whether or not a capture end condition is satisfied (step S160). If the capture end condition is satisfied (step S160: YES), the user terminal 3 terminates the image acquisition process. On the other hand, if the capture end condition is not satisfied (step S160: NO), the user terminal 3 repeats each step from step S120 to step S150.

[0059] <Second embodiment> In the second embodiment, the user terminal 3 (processor 33) includes an image capturing unit 331, an acquisition unit 332, and a focus adjustment unit 333, as in the first embodiment. The image capturing unit 331 in the second embodiment has the same functions as the image capturing unit 331 in the first embodiment.

[0060] <Acquisition part 332> The acquisition unit 332 is configured to acquire object information. Specifically, the acquisition unit 332 acquires the amplitude and vibration frequency of the object T as the object information. The amplitude and vibration frequency of the object T are acquired, for example, from actual measurement of the vibrating object T by a vibrometer, specifications of the vibration table VS, or the like, and input to the user terminal 3. The acquisition unit 332 may accept input of the amplitude and vibration frequency of the object T from the input unit 35, or may read out the amplitude and vibration frequency of the object T stored in advance in the storage unit 32.

[0061] <Focus adjustment section 333> The focus adjustment unit 333 is configured to adjust the focus position of the variable-focus lens 43 based on the object information acquired by the acquisition unit 332. The focus adjustment unit 333 performs the adjustment process of the focus position continuously while the imaging unit 331 acquires a plurality of images. In other words, the adjustment of the focus position is performed asynchronously with the imaging timing of the object T and in parallel with the imaging of the object T.

[0062] Specifically, the focus adjustment unit 333 periodically changes the focus of the variable-focus lens 43 based on the amplitude and vibration frequency of the object T acquired by the acquisition unit 332. This allows the variable-focus lens 43 to be focused on the object T without performing calculation processing based on the acquired image of the object T.

[0063] The procedure for periodically changing the focus of the varifocal lens 43 will be described in detail below. The focus adjustment unit 333 changes the input voltage value of the varifocal lens 43 with a voltage amplitude based on the amplitude of the object T and a voltage angular frequency based on the vibration frequency of the object T. The voltage amplitude is the amplitude of the input voltage value. The voltage angular frequency is the angular frequency of the input voltage value. This can increase the timing at which the varifocal lens 43 focuses on the vibrating object T. As a result, the efficiency of acquiring a clear image of the object T is improved.

[0064] time t k The change in the input voltage value of the variable-focus lens 43 in V t is expressed by, for example, the following formula (15). In formula (15), ΔV is the voltage amplitude. F0 is the vibration frequency of object T, M is a constant by which the frequency of object T is multiplied, and 2πMF0 / 1000 in formula (15) corresponds to the voltage angular frequency. V0 is the initial value (vibration center) of the input voltage value of the variable-focus lens 43.

number

[0065] ΔV in equation (15) is found from the amplitude ΔL of the focus of the variable-focus lens 43. The focus adjustment unit 333 sets a value for the amplitude ΔL of the focus that is greater than the amplitude of the object T (the distance from the center of vibration to the outermost point of vibration). For example, if the amplitude of the object T is 5 mm, the amplitude ΔL can be set to 7.2 mm. ΔV is found from the value of ΔL by the following equation (16). K in equation (16) 31is a constant determined depending on the characteristics of the variable-focus lens 43, and takes a value of, for example, 0.01 or more and 0.1 or less.

number

[0066] In formula (16), for example, K 31 If is 0.0696 and ΔL is 7.2 mm, ΔV is approximately 0.50.

[0067] In formula (15), M is, for example, a numerical value greater than 0 and equal to or less than 5. The value of M is preferably equal to or greater than 0.20 and equal to or less than 0.30. If M is less than 0.20, the time during which the focus of the variable-focus lens 43 and the position of the object T are misaligned increases, so that the frequency of acquiring clear images tends to decrease. On the other hand, if M is greater than 0.30, the time during which the vibration direction of the focus of the variable-focus lens 43 (the vector direction of vibration) and the vibration direction of the object T do not match increases, so that the frequency of acquiring clear images tends to decrease. Note that the value of M does not affect the clarity of the image.

[0068] When the controller 3B is a DA converter, the command value for the input voltage value of the variable-focal-length lens 43 from the personal computer 3A to the controller 3B needs to be an analog value. Therefore, for example, A expressed by the following formula (17) is input to the controller 3B.

number

[0069] Images of object T are acquired at regular intervals, regardless of the periodically changing input voltage value of the varifocal lens 43. Therefore, the multiple acquired images of object T include images acquired when the varifocal lens 43 focuses on object T and images acquired when the object T is not in focus. Of these, the image acquired when the varifocal lens 43 focuses on object T is a candidate for a clear image.

[0070] <Processing flow of the second embodiment> 10 is a flow diagram showing an image acquisition process of the clear image acquisition system 1 in the second embodiment. First, the user terminal 3 (processor 33) acquires vibration information (amplitude and vibration frequency) of the object T (step S210). After acquiring the vibration information, the user terminal 3 starts vibrating the focus of the variable-focus lens 43 by periodically changing the input voltage value with the voltage amplitude and voltage angular frequency set based on the vibration information (step S220). The user terminal 3 causes the camera 41 to acquire an image of the object T while vibrating the focus of the variable-focus lens 43 (step S230).

[0071] After acquiring the image, the user terminal 3 determines whether or not the shooting end condition is satisfied (step S240). If the shooting end condition is satisfied (step S240: YES), the user terminal 3 stops the vibration of the focus of the variable-focus lens 43 (step S250) and ends the image acquisition process. On the other hand, if the shooting end condition is not satisfied (step S250: NO), the user terminal 3 repeats steps S220 and S230.

[0072] <Third embodiment> In the third embodiment, the user terminal 3 (processor 33) includes an imaging unit 331, an acquisition unit 332, and a focus adjustment unit 333 shown in Fig. 5, similarly to the first embodiment. The imaging unit 331 of the third embodiment has the same functions as the imaging unit 331 of the first embodiment.

[0073] <Acquisition part 332> The acquisition unit 332 is configured to acquire object information. Specifically, the acquisition unit 332 acquires, as the object information, the position of the object T measured by the displacement meter 46. The acquisition unit 332 acquires the position of the object T output by the displacement meter 46 at a predetermined constant interval.

[0074] <Focus adjustment section 333> The focus adjustment unit 333 is configured to adjust the focus position of the variable-focus lens 43 based on the object information acquired by the acquisition unit 332. The focus adjustment unit 333 performs the adjustment process of the focus position continuously while the imaging unit 331 is acquiring a plurality of images. The adjustment of the focus position may be performed synchronously with the imaging of the object T, or may be performed asynchronously with the imaging of the object T.

[0075] Specifically, the focus adjustment unit 333 adjusts the focal position of the variable-focus lens 43 based on the position of the object T acquired by the acquisition unit 332 so that the focal point of the variable-focus lens 43 is aligned with the object T. This makes it possible to acquire a clear image of the vibrating object T in real time without performing calculation processing based on the acquired image of the object T.

[0076] Specifically, the focus adjustment unit 333 calculates a focusing voltage value based on the distance from a reference point to the object T. The reference point is, for example, a measurement reference of the displacement meter 46. This allows the focus of the variable-focus lens 43 to be adjusted to the object T with high accuracy and high speed.

[0077] Focus voltage value V based on distance to object T kL is calculated by the following formula (18). In formula (18), L kL is the distance from the reference point to object T. 41 and K 42 are constants that depend on the positions, orientations, parameters, etc. of the variable-focus lens 43 and the displacement meter 46, respectively, and K 41 For example, takes a value between 0.01 and 0.1.

number

[0078] The measurement interval of the displacement meter 46 (i.e., the acquisition interval of the position of object T by the acquisition unit 332) is shorter than the acquisition interval of the image by the camera 41, and is, for example, 100 μs. The focus adjustment unit 333 may adjust the focus of the variable-focus lens 43 based on the measurement result of the displacement meter 46 at least immediately before the image of object T is acquired by the camera 41. The adjustment of the focus of the variable-focus lens 43 may be performed at a constant interval regardless of the timing of acquisition of the image of object T. In other words, the focus adjustment unit 333 may adjust the focus of the variable-focus lens 43 at an interval shorter than the shooting interval of the camera 41. For example, when the shooting speed of the camera 41 is 1 kHz, the focus adjustment speed (feedback speed) of the variable-focus lens 43 may be 10 kHz.

[0079] <Processing flow of the third embodiment> 11 is a flow diagram showing an image acquisition process of the clear image acquisition system 1 in the third embodiment. First, the user terminal 3 (processor 33) acquires the position of the object T from the displacement meter 46 (step S310). After acquiring the position of the object T, the user terminal 3 calculates a focusing voltage value from the position of the object T (step S320).

[0080] After calculating the focusing voltage value, the user terminal 3 sends the calculated focusing voltage value as a command value for the input voltage value to the variable-focus lens 43 (step S330). As a result, the focal position of the variable-focus lens 43 is changed to match the object T.

[0081] After transmitting the input voltage value, the user terminal 3 causes the camera 41 to capture an image of the object T (step S340). As a result, an image of the object T is acquired with the variable-focus lens 43 focused on the object T. After capturing the image, the user terminal 3 determines whether or not a capture end condition is satisfied (step S350). If the capture end condition is satisfied (step S350: YES), the user terminal 3 terminates the image acquisition process. On the other hand, if the capture end condition is not satisfied (step S350: NO), the user terminal 3 repeats each of steps S310 to S340.

[0082] 4. Effect The operation of this embodiment can be summarized as follows: Since the variable-focus lens 43 can be automatically focused on the object T, a clear image of the vibrating object T can be efficiently obtained.

[0083] Although the embodiment of the present invention has been described above, the present invention is not limited to this, and can be modified as appropriate without departing from the technical concept of the invention.

[0084] 5.Other In the above embodiment, the user terminal 3 performs various storage and control, but multiple external devices may be used instead of the user terminal 3. That is, various information and programs may be distributed and stored in multiple external devices using block chain technology or the like.

[0085] The aspect of the present embodiment is not limited to the clear image acquisition system 1, and may be an information processing method or a program. The clear image acquisition method includes each step of the clear image acquisition system 1. The program causes a computer to function as an information processing device of the clear image acquisition system 1.

[0086] In the clear image acquisition system 1, the processor 23 of the server device 2 may have the acquisition unit 332 and / or the focus adjustment unit 333. However, from the viewpoint of reducing the time lag between image acquisition by the image capture device 4 and focus adjustment of the variable focus lens 43 by the focus adjustment unit 333 (feedback to the image capture device 4), it is preferable that the acquisition unit 332 and the focus adjustment unit 333 are included in the processor 33 of the user terminal 3.

[0087] It may be provided in any of the following ways:

[0088] (1) A clear image acquisition system comprising an imaging device and an information processing device having at least one processor, the imaging device having a camera and a variable focus lens with an adjustable focus position, the processor being configured to execute an imaging step, an acquisition step, and a focus adjustment step, the imaging step causing the camera to acquire multiple images of a vibrating object through the variable focus lens, the acquisition step acquiring information about the object while it is vibrating, and the focus adjustment step adjusting the focus position of the variable focus lens based on the information.

[0089] (2) In the clear image acquisition system described in (1) above, in the acquisition step, the size of the object in the latest first image among the multiple images is acquired as the information, and in the focus adjustment step, the focus position of the variable focus lens is adjusted based on the size so that the focus of the variable focus lens is aligned with the object.

[0090] (3) In the clear image acquisition system described in (2) above, the variable focus lens is configured so that the focus position changes depending on the input voltage value, and in the focus adjustment step, the focus voltage value is calculated using a relational equation between the size, a reference voltage value which is the immediately previous input voltage value in the variable focus lens, and a focus voltage value of the variable focus lens, wherein the focus voltage value is the input voltage value when the focus of the variable focus lens is on the object.

[0091] (4) In the clear image acquisition system described in (3) above, the relational equation is a quadratic equation related to the focusing voltage value including a constant term calculated from the size and a coefficient calculated from the reference voltage value, and in the focus adjustment step, one of the two solutions of the quadratic equation is selected as the focusing voltage value using the difference between each of the two solutions and the reference voltage value.

[0092] (5) A clear image acquisition system according to any one of (2) to (4) above, wherein in the acquisition step, the amount of movement of the object between the first image and a second image acquired immediately before the first image is further acquired as the information, and in the focus adjustment step, the focus position of the variable focus lens is adjusted so that the focus of the variable focus lens is aligned with the object based on the size and the amount of movement.

[0093] (6) A clear image acquisition system, as described in (1) above, wherein in the acquisition step, the amplitude and vibration frequency of the object are acquired as the information, and in the focus adjustment step, the focus of the variable focus lens is periodically changed based on the amplitude and the vibration frequency.

[0094] (7) In the clear image acquisition system described in (6) above, the variable focus lens is configured so that the focal position changes depending on an input voltage value, and in the focus adjustment step, the input voltage value is changed by a voltage amplitude based on the amplitude and a voltage angular frequency based on the vibration frequency.

[0095] (8) In the clear image acquisition system described in (1) above, the photographing device further has a displacement meter configured to measure the position of the object, and in the acquisition step, the position of the object measured by the displacement meter is acquired as the information, and in the focus adjustment step, the focus position of the variable focus lens is adjusted based on the position of the object so that the focus of the variable focus lens is aligned with the object.

[0096] (9) In the clear image acquisition system described in (8) above, the variable focus lens is configured so that the focus position changes depending on an input voltage value, and in the focus adjustment step, a focus voltage value is calculated based on a distance from a reference point to the object, where the focus voltage value is the input voltage value when the focus of the variable focus lens is on the object.

[0097] (10) A clear image acquisition method comprising each step of the clear image acquisition system described in any one of (1) to (9) above.

[0098] (11) A program for causing a computer to function as the information processing device of the clear image acquisition system described in any one of (1) to (9) above. Of course, this is not the case.

[0099] Finally, although various embodiments according to the present disclosure have been described, these are presented as examples and are not intended to limit the scope of the invention. The novel embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. The embodiments and their modifications are included within the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0100] 1: Clear image acquisition system 2: Server device 3: User terminal 3A: Personal computer 3B: Controller 4: Imaging device 20: Communication bus 21: Communications Department 22: Storage section 23: Processor 30: Communication bus 31: Communications Department 32: Storage section 33: Processor 34: Display section 35: Input section 41: Camera 41A: Image sensor 42: Lens 43: Variable focus lens 44:Light source 45: Mirror 46: Displacement meter 331: Photography Department 332: Acquisition Department 333: Focus adjustment section T : Object VS: Shaking table

Claims

1. A clear image acquisition system, comprising: A photographing device; An information processing device having at least one processor; Equipped with The imaging device is A camera and A variable focus lens whose focal position is adjustable; having the variable-focus lens is configured so that the focal position changes depending on an input voltage value; The processor is configured to perform the steps of photographing, acquiring, and focusing; In the photographing step, a plurality of images of a vibrating object are captured by the camera through the variable focus lens; In the acquiring step, a size of the object in a latest first image among the plurality of images is acquired as information about the object being vibrated; A clear image acquisition system, in which in the focus adjustment step, the focus position of the variable focus lens is adjusted by calculating the focus voltage value using a relational equation between the size, a reference voltage value which is the immediately previous input voltage value in the variable focus lens, and a focus voltage value of the variable focus lens, wherein the focus voltage value is the input voltage value when the focus of the variable focus lens is on the object.

2. 2. The clear image acquisition system according to claim 1, the relational expression is a quadratic equation related to the focusing voltage value, the quadratic equation including a constant term calculated from the size and a coefficient calculated from the reference voltage value, In the focus adjustment step, a difference between each of two solutions of the quadratic equation and the reference voltage value is used to select one of the two solutions as the focus voltage value.

3. A clear image acquisition system, comprising: A photographing device; An information processing device having at least one processor; Equipped with The imaging device is A camera and A variable focus lens whose focal position is adjustable; having The processor is configured to perform the steps of photographing, acquiring, and focusing; In the photographing step, a plurality of images of a vibrating object are captured by the camera through the variable focus lens; In the acquiring step, an amplitude and a vibration frequency of the object are acquired as information about the object during vibration; A clear image acquisition system, wherein the focus adjusting step periodically changes the focus of the variable focus lens based on the amplitude and the vibration frequency.

4. 4. The clear image acquisition system according to claim 3, the variable-focus lens is configured so that the focal position changes depending on an input voltage value; In the focus adjustment step, the input voltage value is changed with a voltage amplitude based on the amplitude and a voltage angular frequency based on the vibration frequency.

5. A method for acquiring a clear image, comprising: A method for obtaining a clear image, comprising the steps of the system for obtaining a clear image according to any one of claims 1 to 4.

6. A program, A program for causing a computer to function as the information processing device of the clear image acquisition system according to any one of claims 1 to 4.

Images