Electronic device

Abstract

To provide an electronic device that measures the distance.SOLUTION: An electronic device according to an embodiment includes a liquid crystal panel and a processor that controls an imaging element that images an object through the liquid crystal panel. The processor causes the liquid crystal panel to display a first particular image and a second particular image alternately. The processor causes the imaging element to image a first image of the object on which the first particular image is overlapped and a second image of the object on which the second particular image is overlapped alternately. The processor, after the first image is captured, measures the distance on the basis of the first image and the second image captured right before the first image and after the second image is captured, measures the distance on the basis of the second image and the first image captured right before the second image.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] FIELD OF THE INVENTION An embodiment of the present invention relates to an electronic device for measuring distance. [Background technology]

[0002] A technique for measuring the distance to each pixel by utilizing the blur of an image is known. A filter with a specific pattern formed on it is attached to a camera lens, and an image of an object with the specific pattern superimposed thereon is captured. When the object moves out of focus, the image becomes blurred. The degree of blur depends on the extent to which the object moves out of focus. When a specific pattern is superimposed, the shape of the blur differs depending on whether the object is closer to the in-focus position or farther away from the in-focus position. Therefore, the distance to the object for each pixel in the image is measured based on the degree and shape of the blur for each pixel. The specific pattern is called an encoding pattern.

[0003] Using only one image of an object superimposed with one coding pattern can result in multiple distance measurements. Using two images of an object superimposed with at least two different coding patterns can provide accurate distance measurements. However, with this method, the distance measurement rate for video is half the capture rate. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-165070 Summary of the Invention [Problem to be solved by the invention]

[0005] SUMMARY OF THE INVENTION An object of the present invention is to provide an electronic device for measuring distance. [Means for solving the problem]

[0006] An electronic device according to an embodiment includes a liquid crystal panel and a processor that controls an image sensor that captures an image of an object via the liquid crystal panel. The processor alternately displays a first specific image and a second specific image on the liquid crystal panel. The processor alternately captures a first image of the object on which the first specific image is superimposed and a second image of the object on which the second specific image is superimposed. After capturing the first image, the processor measures the distance based on the first image and a second image captured immediately before the first image, and after capturing the second image, measures the distance based on the second image and the first image captured immediately before the second image. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 2 is a diagram for explaining an example of a camera controlled by the electronic device according to the first embodiment. [Figure 2] FIG. 2 is an exploded perspective view illustrating an example of a camera controlled by the electronic device according to the first embodiment. [Figure 3] FIG. 1 is a block diagram illustrating an example of the electrical configuration of an electronic device according to a first embodiment. [Figure 4] FIG. 2 is a diagram for explaining an example of a coding pattern used in the electronic device according to the first embodiment. [Figure 5] 5 is a flowchart for explaining an example of camera control of the electronic device according to the first embodiment. [Figure 6] 5 is a flowchart for explaining an example of distance measurement in the electronic device according to the first embodiment. [Figure 7] 4 is a sequence chart illustrating an example of a processing flow of the electronic device according to the first embodiment. [Figure 8] 6 is a sequence chart for explaining another example of the processing flow of the electronic device according to the first embodiment. [Figure 9] FIG. 10 is a diagram for explaining an example cross-sectional structure of a camera according to a second embodiment. [Figure 10] FIG. 10 is an exploded perspective view showing an example of a camera according to a second embodiment. [Figure 11] FIG. 10 is a diagram for explaining an example cross-sectional structure of a camera according to a third embodiment. [Figure 12] FIG. 10 is a diagram for explaining an example cross-sectional structure of a camera according to a fourth embodiment. [Figure 13] FIG. 13 is a diagram for explaining an example cross-sectional structure of a camera according to a fifth embodiment. [Figure 14] FIG. 13 is a diagram for explaining an example cross-sectional structure of a camera according to a sixth embodiment. [Figure 15] FIG. 13 is a diagram for explaining an example cross-sectional structure of a camera according to a seventh embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments will be described with reference to the drawings. The following description exemplifies devices and methods for embodying the technical concepts of the embodiments. The technical concepts of the embodiments are not limited to the structures, shapes, arrangements, materials, etc. of the components described below. Modifications that can be easily conceived by those skilled in the art are naturally included within the scope of the disclosure. For clarity of explanation, the drawings may show schematic representations of the size, thickness, planar dimensions, or shape of each element, modified from the actual embodiment. Elements in multiple drawings may have different dimensional relationships or ratios. Corresponding elements in multiple drawings may be designated by the same reference numerals, and redundant description may be omitted. Some elements may be designated by multiple names, but these names are merely examples and do not necessarily mean that these elements may be designated by other names. Furthermore, elements that do not have multiple names may also be designated by other names. In the following description, "connected" means not only a direct connection but also a connection via another element.

[0009] First embodiment 1 and 2 are diagrams illustrating an example of a camera 60 controlled by an electronic device according to the first embodiment. Fig. 1(a) is a diagram illustrating an example of the structure of the camera 60 as viewed from the side. Fig. 1(b) is a diagram illustrating an example of the structure of the camera 60 as viewed from above.

[0010] The camera 60 includes a camera board 10. The position where the camera board 10 is arranged is referred to as the lower part of the camera 60. Multiple components of the camera 60 are arranged on the upper surface side of the camera board 10. The components are, for example, a camera module 12, an LCD driver 16, and a camera driver 18. Although not shown, other circuit components such as resistors and inductances are also arranged on the camera board 10. The camera board 10 includes wiring that electrically connects the multiple components and circuit components of the camera to each other.

[0011] The camera module 12 includes a camera housing 11 and an image sensor 14 disposed inside the camera housing 11. The image sensor 14 captures color images. Examples of the image sensor 14 include a CCD sensor and a CMOS sensor. In this specification, the side of the camera board 10 on which the camera module 12 is disposed is referred to as the upper side of the camera board 10.

[0012] The IR filter 28, liquid crystal panel 38, light shielding plate 42, cover board 44, and lens module 46 are arranged in this order above the camera module 12. The IR filter 28 is in close contact or nearly close contact with the camera module 12. The liquid crystal panel 38 is in close contact or nearly close contact with the IR filter 28. The light shielding plate 42 is in close contact or nearly close contact with the liquid crystal panel 38. The cover board 44 is in close contact or nearly close contact with the light shielding plate 42. The lens module 46 is in close contact or nearly close contact with the cover board 44.

[0013] One end of the flexible printed wiring films 20 and 22 is electrically connected to the camera board 10. The liquid crystal driver 16 is a drive circuit component. The liquid crystal driver 16 drives the liquid crystal panel 38 and causes the liquid crystal panel 38 to display a specific image. The liquid crystal driver 16 is an integrated circuit formed as an IC chip. The liquid crystal driver 16 supplies a drive signal to the liquid crystal panel 38 via the flexible printed wiring film 20. The drive signal includes a voltage signal and a control signal.

[0014] The LCD driver 16 supplies a drive signal to the LCD panel 38, causing the LCD panel 38 to display a coded pattern as a specific pattern. The image sensor 14 captures an image of the object on which the coded pattern is superimposed. The distance to the object for each pixel can be measured from this image.

[0015] The camera 60 may be used not only to capture images for distance measurement but also as a normal camera. In this case, the LCD driver 16 supplies a drive signal to the LCD panel 38, causing the LCD panel 38 to display an aperture pattern. The aperture pattern has a central portion that transmits light and a peripheral portion that blocks light. The diameter of the central portion is variable by a control signal. The diameter of the central portion is determined by the brightness of the surroundings of the camera measured by an optical sensor (not shown). The diameter of the central portion is smaller when the surroundings of the camera are bright and larger when the surroundings are dark. The diameter of the central portion (i.e., the aperture value) is also determined by the depth of field of the camera 60. The smaller the diameter of the aperture (the larger the aperture value), the deeper the depth of field.

[0016] The camera driver 18 is a drive circuit component. The camera driver 18 supplies a drive signal to the image sensor 14, causing the image sensor 14 to capture an image. The camera driver 18 is an integrated circuit formed as an IC chip. The camera driver 18 supplies a drive signal to the liquid crystal driver 16 via the camera board 10. The camera driver 18 drives the image sensor 14 in accordance with the drive of the liquid crystal panel, and captures an image. The camera driver 18 may also control the drive timing of the liquid crystal driver 16 in accordance with the drive timing of the image sensor 14, thereby synchronizing the capture of an image by the image sensor 14 with the display of an image (encoding pattern or aperture pattern) by the liquid crystal panel 38.

[0017] An image signal output from the image sensor 14 is supplied to a camera driver 18 via the camera board 10. The camera driver 18 supplies the image signal to an external electronic device via a flexible printed wiring film 22. The electronic device includes a processor that processes the image signal. The image signal processing includes distance measurement. The processor supplies a drive signal corresponding to the aperture pattern or the coding pattern to the liquid crystal driver 16 via the flexible printed wiring film 22. The processor also supplies a shooting drive signal for driving the camera to the camera driver 18 via the flexible printed wiring film 22. The liquid crystal driver 16 supplies a drive signal to the liquid crystal panel 38 via the flexible printed wiring film 20. The liquid crystal drive signal input to the camera board 10 from the flexible printed wiring film 20 may be transmitted to the liquid crystal driver 16 via the camera driver 18, or may be supplied to the liquid crystal driver 16 from wiring formed on the camera board. Details of the electronic device will be described later with reference to FIG. 3.

[0018] The IR filter 28 passes light other than infrared light and prevents infrared light from entering the image sensor 14.

[0019] The liquid crystal panel 38 includes an array substrate 32, a liquid crystal layer 34, and a counter substrate 36. The liquid crystal layer 34 is disposed between the counter substrate 36 and the array substrate 32. The counter substrate 36 is disposed below the liquid crystal layer 34. A black matrix, an overcoat layer, an alignment film, etc. are formed on the counter substrate 36. Because the liquid crystal panel 38 is disposed to control the transmission of visible light or the pattern of the coded apertures, the counter substrate 36 does not include a color filter. The counter substrate 36 may have a multi-layer substrate. The array substrate 32 is disposed above the liquid crystal layer 34. A common electrode, pixel electrodes, an alignment film, active elements, etc. are formed on the array substrate 32. An example of an active element is a thin film transistor (TFT). The array substrate 32 may have a multi-layer substrate. In this embodiment, an active matrix liquid crystal panel is used as the liquid crystal panel 38, but a passive matrix liquid crystal may also be used. As the passive matrix liquid crystal, for example, a TN type liquid crystal with a fast response speed may be used.

[0020] The flexible printed wiring film 20 is electrically connected to either the array substrate 32 or the counter substrate 36. In one example, the flexible printed wiring film 20 is connected to the surface of the array substrate 32 on the liquid crystal layer 34 side. With this configuration, it is possible to reduce the gap between the light-shielding plate 42 and the array substrate 32. Since the array substrate 32 requires a contact area with the flexible printed wiring film 20, the array substrate 32 is larger in size than the counter substrate 36.

[0021] The flexible printed wiring film 20 may be connected to the surface of the array substrate 32 on the light-shielding plate 42 side.

[0022] Furthermore, the flexible printed wiring film 20 may be connected to the surface of the counter substrate 36. In this case, the counter substrate 36 needs a contact area for the flexible printed wiring film 20, so the counter substrate 36 is larger in size than the array substrate 32.

[0023] The liquid crystal driver 16 generates a first voltage signal, a second voltage signal, and a control signal. The liquid crystal driver 16 supplies the first voltage signal to the pixel electrodes of the liquid crystal panel 38 via the active elements. The liquid crystal driver 16 supplies the second voltage signal to the common electrode of the liquid crystal panel 38. The liquid crystal driver 16 supplies the control signal to the control terminals of the active elements of the liquid crystal panel 38. The control signal corresponds to the image to be displayed (the coding pattern or the aperture pattern). The active elements are rendered conductive / non-conductive according to the pixels of the image to be displayed. The voltage applied between the pixel electrodes and the common electrode varies for each pixel. The transmittance of the liquid crystal layer 34 varies for each pixel. As a result, the liquid crystal panel 38 displays an image.

[0024] The light-shielding plate 42 is disposed on the array substrate 32. The light-shielding plate 42 has an opening in the center. Light incident on the light-shielding plate 42 via the lens 48 passes through the opening and enters the liquid crystal panel 38. The portion of the light-shielding plate 42 other than the opening is a light-shielding portion that does not transmit light. The light-shielding plate 42 controls the intrusion of external light that is unnecessary for imaging and travels from the lens 48 to the imaging element 14. Instead of providing the light-shielding plate 42, the same light-shielding pattern as the light-shielding plate 42 may be printed on the array substrate 32 or the counter substrate 36, so that the array substrate 32 or the counter substrate 36 also serves as the light-shielding plate 42.

[0025] The cover board 44 is placed on the light shielding plate 42. The cover board 44 is a plate-like member that protects the components of the camera 60 other than the lens module 46, namely the liquid crystal panel 38 and the camera module 12. Instead of providing the light shielding plate 42, the cover board 44 may be printed with the same light shielding pattern as the light shielding plate 42, so that the cover board 44 also serves as the light shielding plate 42.

[0026] The lens module 46 is disposed on the cover board 44. The lens module 46 includes a lens housing 47, a lens 48 disposed in the lens housing 47, and an actuator 50. The lens 48 may be composed of a single lens or multiple lenses. The actuator 50 is electrically connected to the camera driver 18 via wiring (not shown). The camera driver 18 determines the focus state from the output signal of the image sensor 14, and drives the actuator 50 to move the lens 48 in the optical axis direction so that the lens 48 is in focus. The camera driver 18 determines the focus state based on, for example, the phase difference between the output signals of two pixels.

[0027] 2 is an exploded perspective view illustrating an example of a camera 60 controlled by an electronic device according to an embodiment. While the flexible printed wiring films 20 and 22 are not shown in FIG. 2, contacts 20a and 22a of the camera substrate 10 to which the flexible printed wiring films 20 and 22 are connected are shown.

[0028] 3 is a block diagram illustrating an example of the electrical configuration of the electronic device 62 according to the first embodiment. The electronic device 62 includes a display unit 52, a processor 54, a coding pattern memory 56, a first memory 64a, a second memory 64b, and a correction kernel memory 66. The first memory 64a and the second memory 64b may be configured as separate independent memories, or may be configured as separate storage areas of a single memory. The camera 60 is connected to the processor 54.

[0029] The processor 54 is connected to the coding pattern memory 56, the first memory 64a, the second memory 64b, the correction kernel memory 66, and the display unit 52. The coding pattern memory 56 stores coding pattern data representing a coding pattern to be superimposed on an image of an object. The coding pattern is a pattern that becomes blurred when the object moves out of focus. The correction kernel memory 66 stores blur correction kernels for correcting image blur. The blur correction kernels vary depending on the distance to the object. The correction kernel memory 66 stores a number of blur correction kernels corresponding to a number of distances. The coding pattern memory 56 and the correction kernel memory 66 may be configured as read-only memories. In this case, the coding pattern memory 56 and the correction kernel memory 66 may be configured as independent, separate memories, or may be configured as separate storage areas of a single memory. Furthermore, if the coding pattern memory 56 and the correction kernel memory 66 are not configured as read-only memories, the coding pattern memory 56, the correction kernel memory 66, the first memory 64a, and the second memory 64b may be configured as separate storage areas of a single memory. Furthermore, the coding pattern memory 56, the correction kernel memory 66, the first memory 64a, and the second memory 64b may be configured as external memories to the processor 54 or as memories built into the processor 54.

[0030] The processor 54 supplies a drive signal corresponding to the coded pattern data to the camera driver 18. The camera driver 18 controls the drive timing of the liquid crystal driver 16 in synchronization with the drive of the image sensor 14.

[0031] The image signal output from the image sensor 14 is input to the processor 54 via the camera driver 18. The liquid crystal panel 38 displays the coded pattern, and the image sensor 14 captures an image of the object on which the coded pattern is superimposed. The image sensor 14 transmits the image signal to the processor 54. The processor 54 writes the two image signals into a first memory 64a and a second memory 64b.

[0032] The processor 54 performs a convolution operation on the image signal for each pixel of the image of the object by using a blur correction kernel. The processor 54 detects the blur correction kernel that produces a calculation result with the least blur, and determines the distance corresponding to that blur correction kernel as the measurement result of the distance to the object. Note that multiple measurement results may be obtained using only one image of the object on which one coding pattern is superimposed. When two images of the object on which at least two different coding patterns are superimposed are used, only one blur correction kernel produces a calculation result with the least blur. In this embodiment, the distance is measured based on two images of the object on which two different coding patterns are superimposed. The distance may also be measured using three or more coding patterns.

[0033] The type of coding pattern can be changed depending on the type of object and the measurement environment. The coding pattern memory 56 may store a plurality of coding patterns or a plurality of coding pattern groups, and a coding pattern or a coding pattern group may be selected depending on the type of object and the measurement environment.

[0034] The processor 54 displays a distance image representing the distance for each pixel on the display unit 52. An example of a distance image is an image in which each pixel is represented by a color according to the distance.

[0035] The processor 54 may not only display the distance information but also perform various controls using the distance information. For example, if the electronic device 62 is applied to a self-propelled robot, the processor 54 controls the movement of the robot so as to avoid the object according to the distance to the object.

[0036] FIG. 4 is a diagram illustrating an example of a coding pattern used in the electronic device according to the first embodiment. The coding pattern memory 56 stores a coding pattern group having two different first and second coding patterns. FIG. 4(a) shows an example of a first coding pattern 58a. FIG. 4(b) shows an example of a second coding pattern 58b. The coding patterns 58a and 58b are determined so that the degree of blur varies depending on the degree to which the object deviates from the focused state. Furthermore, the coding patterns 58a and 58b are determined so that the shape of the blur differs depending on whether the object is located in front of the focused position or behind the focused position. In other words, the coding patterns 58a and 58b must not have a point-symmetric shape.

[0037] 5 is a flowchart illustrating an example of camera control by the electronic device 62 according to the first embodiment. The processor 54 starts the processing of FIG. 5 in response to receiving an instruction signal for distance measurement. An example of the instruction signal is a signal generated while the shutter button of the camera 60 is being pressed.

[0038] Upon receiving the instruction signal, the processor 54 reads the first coded pattern signal from the coded pattern memory 56 and transmits the first coded pattern signal to the liquid crystal driver 16 via the camera driver 18 (S12).

[0039] The liquid crystal driver 16 drives the liquid crystal panel 38 in response to the first coded pattern signal, causing the liquid crystal panel 38 to display the first coded pattern 58a (S14).

[0040] The display image on the liquid crystal panel 38 does not change immediately after the start of driving, but changes after a display transition time has elapsed. The display transition time is determined depending on the display pattern. The longest display transition time is predictable. The liquid crystal panel 38 displays the first coding pattern 58a a predetermined time after the start of driving using the first coding pattern signal. The predetermined time is set to be equal to or longer than the longest display transition time. During the predetermined time, the display image on the liquid crystal panel 38 gradually changes from the previously displayed image to the first coding pattern 58a.

[0041] The processor 54 transmits a shooting start signal to the camera driver 18 after a predetermined time has elapsed since the liquid crystal panel 38 started to be driven by the first coded pattern signal (S16).

[0042] When the camera driver 18 receives the image capture start signal, it drives the image capture element 14 and causes the image capture element 14 to capture a first image of the object on which the first coded pattern 58a is superimposed (S18). The first image signal captured by the image capture element 14 is transmitted to the processor 54 via the camera driver 18.

[0043] The processor 54 writes the first image signal transmitted from the camera 60 into the first memory 64a (S20).

[0044] The processor 54 reads the second coded pattern signal from the coded pattern memory 56, and transmits the second coded pattern signal to the liquid crystal driver 16 via the camera driver 18 (S32).

[0045] The liquid crystal driver 16 drives the liquid crystal panel 38 in response to the second coded pattern signal, causing the liquid crystal panel 38 to display the second coded pattern 58b (S34).

[0046] The processor 54 transmits a shooting start signal to the camera driver 18 after a predetermined time has elapsed since the liquid crystal panel 38 started to be driven by the second coded pattern signal (S36).

[0047] When the camera driver 18 receives the image capture start signal, it drives the image capture element 14 and causes the image capture element 14 to capture a second image of the object on which the second coded pattern 58b is superimposed (S38). The second image signal captured by the image capture element 14 is transmitted to the processor 54 via the camera driver 18.

[0048] The processor 54 writes the second image signal transmitted from the camera 60 into the second memory 64b (S40).

[0049] Thereafter, the processor 54 reads out the first coded pattern signal from the coded pattern memory 56 and again executes the process of transmitting the first coded pattern signal to the liquid crystal driver 16 via the camera driver 18 (S12).

[0050] The processor 54 repeatedly executes the process of FIG. 5 until reception of the instruction signal is completed.

[0051] 6 is a flowchart for explaining an example of distance measurement by the electronic device 62 according to the first embodiment. The processor 54 executes the process of FIG. 6 while receiving an instruction signal for distance measurement.

[0052] The processor 54 determines whether or not writing of the first image signal onto which the first coding pattern 58a is superimposed into the first memory 64a is completed (S52). The processor 54 repeats this determination process until writing of the first image signal into the first memory 64a is completed.

[0053] When the processor 54 determines that writing of the first image signal into the first memory 64a has finished (YES in S52), it determines whether writing of the second image signal onto which the second encoding pattern 58b is superimposed into the second memory 64b has finished (S54). The processor 54 repeats this determination process until writing of the second image signal into the second memory 64b has finished.

[0054] When the processor 54 determines that writing of the second image signal to the second memory 64b has been completed (YES in S54), it reads the first image signal from the first memory 64a and reads the second image signal from the second memory 64b (S56).

[0055] The processor 54 sequentially reads out a large number of blur correction kernels from the correction kernel memory 66, sequentially performs convolution calculations with the large number of blur correction kernels for each pixel of the two image signals, detects the blur correction kernel that produces a calculation result that minimizes blur for each pixel, and determines the distance corresponding to that blur correction kernel as the measurement result of the distance to the pixel of the object (S58).

[0056] The processor 54 outputs the distance image (S60). One example of output is to display the distance image on the display unit 52 with each pixel expressed in a color according to the distance.

[0057] The processor 54 determines whether writing of the first image signal superimposed with the first coding pattern 58a to the first memory 64a or writing of the second image signal superimposed with the second coding pattern 58b to the second memory 64b has finished (S62). The processor 54 repeats this determination process until writing of the first image signal to the first memory 64a has finished.

[0058] When the processor 54 determines that writing of the first image signal to the first memory 64a has been completed, it again executes the process of S56 (reading the first image signal from the first memory 64a and reading the second image signal from the second memory 64b).

[0059] The processor 54 repeatedly executes the process of FIG. 6 until reception of the instruction signal is completed.

[0060] 7 is a sequence chart for explaining an example of the flow of processing in the electronic device 62 according to the first embodiment. The horizontal axis in FIG. 7 represents time.

[0061] The processor 54 periodically and alternately supplies the first coding pattern signal and the second coding pattern signal to the liquid crystal panel 38. In odd-numbered periods, the first coding pattern signal is supplied to the liquid crystal panel 38. In even-numbered periods, the second coding pattern signal is supplied to the liquid crystal panel 38. When the supplied coding pattern signal switches, the display on the liquid crystal panel 38 does not switch immediately. The display image on the liquid crystal panel 38 gradually changes from the start of input of the first coding pattern signal, and switches to the first coding pattern 58a after a predetermined time. Similarly, the display image on the liquid crystal panel 38 gradually changes from the start of input of the second coding pattern signal, and switches to the second coding pattern 58b after a predetermined time.

[0062] In odd-numbered periods, after the display image on the liquid crystal panel 38 is switched to the first coding pattern 58a, the image sensor 14 captures an image of the object on which the first coding pattern 58a is superimposed and outputs a first image signal.In even-numbered periods, after the display image on the liquid crystal panel 38 is switched to the second coding pattern 58b, the image sensor 14 captures an image of the object on which the second coding pattern 58b is superimposed and outputs a second image signal.

[0063] In odd-numbered periods, the first image signal I1a is written to the first memory 64a. The first memory 64a holds the first image signal I1a until the next first image signal I1b is written in the next odd-numbered period. In even-numbered periods, the second image signal I2a is written to the second memory 64b. The second memory 64b holds the second image signal I2a until the next second image signal I2b is written in the next even-numbered period.

[0064] The first image signal I1 or the second image signal I2 is updated in each cycle. Therefore, the pair of the first image signal I1 and the second image signal I2 is updated in each cycle. The processor 54 measures the distance using the latest pair of the first image signal I1 and the second image signal I2 in each cycle. In this way, since the distance is measured in each cycle, it is possible to measure the distance and output a distance image at a rate equal to the shooting rate of the video.

[0065] FIG. 8 is a sequence chart illustrating another example of the processing flow of the electronic device 62 according to the first embodiment. FIG. 8 shows an example of measuring distance using three different coding patterns A, B, and C. In this example, too, the first image signal of the object on which coding pattern A is superimposed, the second image signal of the object on which coding pattern B is superimposed, or the third image signal of the object on which coding pattern C is superimposed is updated in each cycle. Therefore, the set of the first image signal, the second image signal, and the third image signal is updated in each cycle. The processor 54 measures distance using the latest set of the first image signal, the second image signal, and the third image signal in each cycle.

[0066] According to the first embodiment, the distance is measured at each period, so that the distance can be measured at a rate equal to the shooting rate of the video.

[0067] Second embodiment Fig. 9 is a diagram illustrating an example of the structure of the camera 60a according to the second embodiment as viewed from the side, and Fig. 10 is an exploded perspective view illustrating an example of the camera 60a according to the second embodiment.

[0068] The camera 60a has a liquid crystal panel 38a instead of the liquid crystal panel 38 of the first embodiment. The liquid crystal panel 38a includes an array substrate 32, a liquid crystal layer 34, and a counter substrate 36. The liquid crystal panel 38a differs from the liquid crystal panel 38 in the arrangement order of the array substrate 32, the liquid crystal layer 34, and the counter substrate 36. In the liquid crystal panel 38a, the array substrate 32 is disposed below the liquid crystal layer 34. The counter substrate 36 is disposed above the liquid crystal layer 34.

[0069] The flexible printed wiring film 20 is connected to the surface of the array substrate 32 on the liquid crystal layer 34 side. The array substrate 32 needs a contact area with the flexible printed wiring film 20, so the array substrate 32 is larger in size than the counter substrate 36.

[0070] The flexible printed wiring film 20 may be connected to the surface of the array substrate 32 on the IR filter 28 side.

[0071] Furthermore, the flexible printed wiring film 20 may be connected to the surface of the counter substrate 36. In this case, the counter substrate 36 is larger in size than the array substrate 32 because a contact area with the flexible printed wiring film 20 is required.

[0072] Third embodiment FIG. 11 is a diagram illustrating an example of the structure of a camera 60b according to the third embodiment as viewed from the side.

[0073] The third embodiment relates to a modification of the first embodiment. The camera 60b differs from the camera 60 in that the liquid crystal driver 16 is arranged on the flexible printed wiring film 20, not on the camera board 10. Therefore, the third embodiment achieves the same effects as the first embodiment. Furthermore, according to the third embodiment, the camera board 10 does not require installation space for the liquid crystal driver 16, so the planar size of the camera board 10 can be reduced. Not only the liquid crystal driver 16, but also the camera driver 18 may be arranged on the flexible printed wiring film 22.

[0074] Fourth embodiment FIG. 12 is a diagram illustrating an example of the structure of a camera 60c according to the fourth embodiment as viewed from the side.

[0075] The fourth embodiment relates to a modification of the second embodiment. The camera 60c differs from the camera 60 in that the liquid crystal driver 16 is arranged on the flexible printed wiring film 20, not on the camera board 10. Therefore, the fourth embodiment achieves the same effects as the second embodiment. Furthermore, according to the fourth embodiment, the camera board 10 does not require installation space for the liquid crystal driver 16, so the planar size of the camera board 10 can be reduced. Not only the liquid crystal driver 16, but also the camera driver 18 may be arranged on the flexible printed wiring film 22.

[0076] Fifth embodiment FIG. 13 is a diagram illustrating an example of the structure of a camera 60d according to the fifth embodiment as viewed from the side.

[0077] The fifth embodiment relates to a modification of the second embodiment. The camera 60d differs from the camera 60b in that solder balls 74 are used instead of the flexible printed wiring film 20 as components connecting the liquid crystal driver 16 and the liquid crystal panel 38a. The solder balls 74 are arranged on support portions 72 arranged in a two-dimensional array on the camera substrate 10a around the camera module 12. The support portions 72 include wiring therein, and the solder balls 74 are electrically connected to the camera substrate 10a via the support portions 72. The camera substrate 10a is also called a BGA (Ball Grid Array) substrate.

[0078] In the first to fourth embodiments, the flexible printed wiring film 20 that electrically connects the camera substrate 10 and the array substrate 32 curves from the camera substrate 10 and the array substrate 32 toward the periphery. Therefore, the planar size of the cameras 60, 60a, 60b, and 60c is slightly larger than that of the camera substrate 10. In the fifth embodiment, the solder balls 74 that electrically connect the camera substrate 10a and the array substrate 32 are disposed directly below the array substrate 32. Therefore, the planar size of the camera 60d is not larger than that of the camera substrate 10a.

[0079] Sixth embodiment FIG. 14 is a diagram illustrating an example cross-sectional structure of a camera according to the sixth embodiment.

[0080] In the sixth embodiment, the array substrate 32 of the first embodiment is replaced with an array substrate 32a made of a flexible printed wiring film. One end of the array substrate 32a is electrically connected to the camera substrate .

[0081] According to the sixth embodiment, the array substrate and the flexible printed wiring film are integrally configured, eliminating the need for a contact area between the array substrate and the flexible printed wiring film. This allows the planar size of the liquid crystal panel 38 to be smaller than that of the first embodiment, and eliminates the need for a process for connecting the array substrate and the flexible printed wiring film. Furthermore, the connection between the array substrate and the flexible printed wiring film does not become unstable.

[0082] In the sixth embodiment, the liquid crystal driver 16 may be disposed on the array substrate 32a, as in the third embodiment.

[0083] In the sixth embodiment, as in the modified example of the first embodiment, the counter substrate 36 may be made of a flexible printed wiring film instead of the array substrate 32.

[0084] Seventh embodiment FIG. 15 is a diagram illustrating an example cross-sectional structure of a camera according to the seventh embodiment.

[0085] In the seventh embodiment, the array substrate 32 of the second embodiment is replaced with an array substrate 32a made of a flexible printed wiring film. One end of the array substrate 32a is electrically connected to the camera substrate .

[0086] According to the seventh embodiment, a contact area between the array substrate and the flexible printed wiring film is not required. Therefore, the planar size of the liquid crystal panel 38 can be made smaller than that of the first embodiment, and the process of connecting the array substrate and the flexible printed wiring film is not required. Furthermore, the connection between the array substrate and the flexible printed wiring film does not become unstable.

[0087] In the seventh embodiment, the liquid crystal driver 16 may be disposed on an array substrate 32a made of a flexible printed wiring film, as in the fourth embodiment.

[0088] In the seventh embodiment, as in the modified example of the second embodiment, the counter substrate 36, instead of the array substrate 32, may be made of a flexible printed wiring film.

[0089] In the above-described embodiment, the liquid crystal panel 38 is disposed in front of the camera module 12 (the imaging surface of the image sensor 14), the lens module 46 is disposed in front of the liquid crystal panel 38, and light passing through the lens module 46 is incident on the image sensor 14 via the liquid crystal panel 38. The order in which the lens module 46 and the liquid crystal panel 38 are disposed may be reversed. That is, the lens module 46 may be disposed in front of the camera module 12, the liquid crystal panel 38 may be disposed in front of the lens module 46, and light passing through the liquid crystal panel 38 may be incident on the image sensor 14 via the lens module 46.

[0090] The present invention is not limited to the above-described embodiments, and the components can be modified and embodied in practice without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined. [Explanation of symbols]

[0091] 10...camera board, 12...camera module, 14...imaging element, 16...liquid crystal driver, 18...camera driver, 20, 22...flexible printed wiring film, 32...array substrate, 34...liquid crystal layer, 36...opposite substrate, 38...liquid crystal panel, 40...IR filter, 42...light shielding plate, 44...cover board, 46...lens module, 48...lens, 50...actuator, 58a, 58b...encoding pattern, 60...camera, 62...electronic device

Claims

1. An electronic device comprising a liquid crystal panel and a processor that controls an image pickup element that captures an image of an object through the liquid crystal panel, The processor: alternately displaying a first specific image and a second specific image on the liquid crystal panel; causing the image sensor to alternately capture a first image of the object on which the first specific image is superimposed and a second image of the object on which the second specific image is superimposed; After capturing the second image, a distance is measured based on the second image and the first image captured before the second image; After capturing the first image, the electronic device measures distance based on the first image and the second image captured before the first image.

2. The electronic device of claim 1 , wherein the first specific image and the second specific image are not point-symmetric.

3. The electronic device according to claim 1 , wherein the first specific image and the second specific image have shapes that are blurred to different degrees depending on the degree to which the object is out of focus.

4. 2. The electronic device according to claim 1, wherein the shapes of the first specific image and the second specific image are different in shape from each other depending on whether the object is located in front of or behind the focus position.

5. An electronic device comprising a liquid crystal panel and a processor that controls an image pickup element that captures an image of an object through the liquid crystal panel, The processor: a first specific image, a second specific image, and a third specific image are sequentially displayed on the liquid crystal panel; causing the image sensor to sequentially capture a first image of the object on which the first specific image is superimposed, a second image of the object on which the second specific image is superimposed, and a third image of the object on which the third specific image is superimposed; After capturing the third image, a distance is measured based on the third image and the first and second images captured before the third image; After capturing the first image, a distance is measured based on the first image and the second and third images captured before the first image; After capturing the second image, the electronic device measures distance based on the second image, the third image captured before the second image, and the first image.

6. An electronic device comprising a liquid crystal panel and a processor that controls an image pickup element that captures an image of an object through the liquid crystal panel, The processor: Sequentially displaying a plurality of specific images on the liquid crystal panel; causing the imaging element to sequentially capture a plurality of images of the object on which the plurality of specific images are superimposed; An electronic device that measures distance based on the plurality of images of the object as the imaging device captures each of the plurality of images of the object.

Images