Distance measuring device and distance measuring control method

Abstract

Distances measured by a plurality of distance measuring means for the same subject are detected. [Solution] The system includes a first imaging means 202 that captures an image within a first angle of view, a first distance measuring means 203 that measures the distance to an object included within the first angle of view, and a control means 110 that controls the distance measuring. When measuring the distance to an object included within a second angle of view that is also imaged by a second imaging means 208 using a second distance measuring means 207 separate from the first distance measuring means, the control means detects the distances measured to the same object by the first distance measuring means and the second distance measuring means using first data generated using the first imaging means and second data generated using the second imaging means.

Description

[Technical Field]

[0001] The present invention relates to a distance measuring device for measuring the distance to an object. [Background technology]

[0002] Some electronic devices, such as smartphones, tablets, and digital cameras, are equipped with LiDAR (Light Detection and Ranging or Laser Imaging Detection and Ranging), which irradiates a target with laser light and measures the distance to the target using the reflected light. When such distance measurement is performed simultaneously on the same target by multiple electronic devices, there is a risk that errors will occur in the distance measurement results if one electronic device receives the reflected light of the laser light irradiated on the target by another electronic device. This is called mutual interference in LiDAR.

[0003] Patent Document 1 discloses a technique for preventing mutual interference by separating the timing of emitting laser light and receiving reflected light in one electronic device (camera) from the timing of those in another electronic device. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2021-535406 Summary of the Invention [Problem to be solved by the invention]

[0005] The technique disclosed in Patent Document 1 has the disadvantage that the time during which each of the plurality of electronic devices can emit laser light and the time during which reflected light can be received are limited. [Means for solving the problem]

[0006] A distance measuring device according to one aspect of the present invention includes a first imaging means for capturing an image within a first angle of view, a first distance measuring means for measuring the distance to an object included within the first angle of view, and a control means for controlling the distance measuring. When measuring the distance to an object included within a second angle of view that is also captured by a second imaging means separate from the first distance measuring means, the control means detects distances to the same object by the first distance measuring means and the second distance measuring means using first data generated using the first imaging means and second data generated using the second imaging means. Another aspect of the present invention includes a distance measuring device including a first imaging means for capturing an image within the first angle of view, a first distance measuring means for measuring the distance to an object included within the first angle of view, and a control means for controlling the distance measuring. The control means is characterized in that, when measuring the distance to an object included in a second angle of view captured by the second imaging means using a second distance measuring means separate from the first distance measuring means, the control means changes settings related to distance measurement of at least one of the first distance measuring means and the second distance measuring means based on first data generated using the first imaging means and second data generated using the second imaging means capturing an image within the second angle of view. Note that an electronic device having the above-mentioned distance measuring device also constitutes another aspect of the present invention.

[0007] Another aspect of the present invention is a distance measurement control method characterized by comprising the steps of: acquiring first data generated by a first distance measurement means capturing an image within a first angle of view that includes an object to be measured; and second data generated by a second distance measurement means capturing an image within a second angle of view that includes an object to be measured; and using the first data and the second data to detect distance measurements of the same object by the first distance measurement means and the second distance measurement means.

[0008] Furthermore, a distance measurement control method according to another aspect of the present invention includes the steps of acquiring first data generated by a first distance measurement means capturing an image within a first angle of view that includes an object to be measured, and second data generated by a second distance measurement means capturing an image within a second angle of view that includes the object to be measured, and changing distance measurement settings of at least one of the first distance measurement means and the second distance measurement means based on the first data and the second data. Note that a program for causing a computer to execute processing according to the distance measurement control method also constitutes another aspect of the present invention. [Effects of the Invention]

[0009] According to the present invention, it is possible to detect distances measured by a plurality of distance measuring means to the same object, and in such cases, it is possible to perform good distance measurement. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing the configuration of a distance measuring device (camera) according to a first embodiment. [Figure 2] 3A and 3B are diagrams illustrating simultaneous distance measurement of the same object by a plurality of image capturing devices in the first embodiment. [Figure 3] 3 is a flowchart showing a process executed in the first embodiment. [Figure 4] FIG. 2 is a diagram showing specifications of an infrared laser in the first embodiment. [Figure 5] FIG. 10 is a block diagram showing the configuration of a distance measuring device according to a second embodiment. [Figure 6] 10 is a flowchart showing a process executed in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. [Example]

[0012] 1 shows the configuration of a first distance measuring device (hereinafter referred to as a first camera) 100 according to the first embodiment. The first camera 100 is mounted on electronic devices such as smartphones, tablets, and digital cameras.

[0013] In Figure 1, reference numeral 101 denotes a CMOS sensor (first imaging means) as an imaging element, 102 denotes a LiDAR sensor (first ranging means) as a ranging sensor, and 103 denotes a visible light image processing circuit. 104 denotes a distance map generation circuit, 105 denotes an image recognition circuit, and 106 denotes a 3D model generation circuit. 107 denotes a laser controller, 108 denotes an infrared laser array, and 109 denotes a proximity sensor. 110 denotes a system controller (control means), 111 denotes a communication unit, 112 denotes an encryption circuit, 113 denotes a public key data memory, 114 denotes a private key data memory, and 115 denotes a decryption circuit. 116 denotes a comparison and judgment circuit, and 117 denotes a display unit.

[0014] 1 (hereinafter referred to as the second camera), has a CMOS sensor and a LiDAR sensor and has the same configuration as the first camera 100. The second camera is also mounted on electronic devices such as smartphones, tablets, and digital cameras.

[0015] 1, S101 indicates visible light RAW data, S102 indicates LiDAR ranging data, and S103 indicates visible light image data. S104 indicates distance map data, S105 indicates captured image recognition data from the first camera, S106 indicates 3D model data, and S107 indicates a proximity sensor signal. S108 indicates public key data from the second camera, S109 indicates encrypted captured image recognition data from the first camera, S110 indicates public key data from the first camera, and S111 indicates private key data from the first camera. S112 indicates encrypted captured image recognition data from the second camera, S113 indicates communication packet data, and S114 indicates captured image recognition data from the second camera. S115 indicates a comparison result, S116 indicates a laser controller control signal, and S117 indicates pulse period change information. S118 indicates a laser pulse control signal, and S119 indicates a display control signal for displaying the state of mutual interference.

[0016] FIG. 2 schematically illustrates simultaneous distance measurement of the same object by the LiDARs of the first camera 200 and the second camera 206. Reference numeral 202 denotes the CMOS sensor (101) of the first camera 201, and 203 denotes the LiDAR sensor (102) of the first camera 201. The CMOS sensor 202 captures an image within an imaging angle of view θ2, which is a first angle of view. The imaging angle of view θ2 of the CMOS sensor 202 and the ranging angle of view θ3 of the LiDAR sensor 203 at least partially overlap. That is, the LiDAR sensor 203 can measure (detect) the distance to a subject (object) included within the imaging angle of view θ2 of the CMOS sensor 202. Note that the imaging angle of view θ2 and the ranging angle of view θ3 may be different from each other or may be the same from each other. Reference numeral 204 denotes a wireless antenna of the first camera 201, and 205 denotes a proximity sensor (109) of the first camera 201.

[0017] Reference numeral 207 denotes a CMOS sensor (second imaging means) of the second camera 206, and 208 denotes a LiDAR sensor (second ranging means) of the second camera 206. The CMOS sensor 207 captures an image within an imaging angle of view θ7 as a second angle of view. The imaging angle of view θ7 of the CMOS sensor 207 and the ranging angle of view θ8 of the LiDAR sensor 208 at least partially overlap. That is, the LiDAR sensor 208 can measure the distance to a subject (object) included within the imaging angle of view θ7 of the CMOS sensor 207. Note that the imaging angle of view θ7 and the ranging angle of view θ8 may be different from each other or may be the same as each other. Reference numeral 209 denotes a wireless antenna of the second camera 206, and 210 denotes a proximity sensor of the second camera 206. Reference numeral 211 denotes a subject as an object.

[0018] 3 shows a process (distance measurement control method) executed according to a program by system controller 110, which is made up of a computer such as a CPU, in first camera (local camera) 201. STEP means a process (step).

[0019] The system controller 110 starts the process in STEP 300 and then causes the CMOS sensor 101 to capture visible light images in STEP 301 .

[0020] Next, in STEP 302, the system controller 110 causes the visible light image processing circuit 103 to perform development processing of the visible light image data.

[0021] Next, in STEP 303, the system controller 110 causes the image recognition circuit 105 to perform processing to recognize the subject included in the visible light image data, and generates captured image recognition data (first data) of the camera itself indicating the result of the subject recognition processing.

[0022] Next, in STEP 304, the system controller 110 determines whether or not a nearby object (specifically, whether the second camera is close to the first camera) has been detected by the proximity sensor 109. If a nearby object has been detected, the system controller 110 performs the process of STEP 305; if not, the system controller 110 performs the process of STEP 317.

[0023] In STEP 305, the system controller 110 determines whether image information from the second camera (other camera) can be acquired by communication. If image information can be acquired, the process of STEP 306 is performed; if image information cannot be acquired, the process of STEP 315 is performed.

[0024] Next, in STEP 306, the system controller 110 receives public key data from the other camera.

[0025] Next, in STEP 307, the system controller 110 causes the encryption circuit 112 to encrypt the image recognition data of the image captured by the camera itself obtained by the image recognition circuit 105.

[0026] Next, in STEP 308, the system controller 110 transmits the encrypted captured image recognition data of its own camera via the communication unit 111 to the other cameras.

[0027] Next, in STEP 309, the system controller 110 transmits the public key data of the camera itself to the other cameras.

[0028] Next, in STEP 310, the system controller 110 receives the encrypted captured image recognition data (second data) from the other camera.

[0029] In STEP 311, the system controller 110 causes the decoding circuit 115 to decode the received encrypted image recognition data from the other camera.

[0030] Next, in STEP 312, the system controller 110 causes the comparison and determination circuit 116 to compare the captured image recognition data of the camera itself with the captured image recognition data of the other cameras.

[0031] Next, in STEP 313, the system controller 110 causes the comparison / determination circuit 116 to determine whether the 3D model subject included in the captured image recognition data of the subject camera and the 3D model subject included in the captured image recognition data of the other camera are the same. That is, it determines whether distance measurement is performed on the same subject by the subject camera and the other camera. If the 3D model subjects are the same, the process proceeds to STEP 314; if they are not the same, the process proceeds to STEP 317.

[0032] In STEP 314, system controller 110 determines whether the subject camera and the other camera are cameras with the same specifications for distance measurement (hereinafter referred to as cameras of the same model). If the cameras are not the same model, first distance measurement adjustment is performed in STEP 315. If the cameras are the same model, second distance measurement adjustment is performed in STEP 316. In other words, different distance measurement adjustments (changes to distance measurement settings) are performed depending on whether the subject camera and the other camera are the same model.

[0033] In STEP 315, the system controller 110 causes the laser controller 107 to perform a first distance measurement adjustment, such as adjusting the emission period (pulse period) and intensity of the infrared laser light (pulsed light) from the infrared laser array 108. Then, the system controller 110 performs the process of STEP 317.

[0034] In STEP 316, the system controller 110 causes the laser controller 107 to adjust the distance measurement period, the distance measurement cycle, and the distance measurement start timing as the second distance measurement adjustment. Then, the system controller 110 performs the process of STEP 317.

[0035] In STEP 317, the system controller 110 causes the laser controller 107 to irradiate pulsed light from the infrared laser array 108 toward the 3D model subject.

[0036] Next, in STEP 318, the system controller 110 causes the LiDAR sensor 102 to receive the pulsed light as reflected light.

[0037] Next, in STEP 319, the system controller 110 causes the distance map generating circuit 104 to generate a distance map.

[0038] Next, in STEP 320, the system controller 110 causes the 3D model generation circuit 106 to generate a 3D model, and then in STEP 321, this process ends.

[0039] 4 shows pulsed light emitted from the infrared laser array 108 and received by the LiDAR sensor 102 (203). 401 indicates the pulse width of the pulsed light, 402 indicates the pulse period, and 403 indicates the intensity of the pulsed light (hereinafter referred to as laser intensity). 404 indicates the ranging period as the light reception period by the LiDAR sensor 102, 405 indicates the ranging period, and 406 indicates the timing at which ranging starts.

[0040] Below, we will explain the details of the processing of each step in Fig. 3 and each piece of data in Fig. 2. First, in STEP 300, the processing for generating a 3D model starts.

[0041] In STEP 301, the subject 211, which is the object for generating a 3D model, is captured by the CMOS sensor 202 (101) of the first camera 201. As a result, visible light RAW data S101 is output and input to the visible light image processing circuit 103.

[0042] In STEP 302, development processing is performed by the visible light image processing circuit 103, thereby generating visible light image data S103 as the first captured image. The visible light image data S103 is input to the image recognition circuit 105 and the 3D model generation circuit .

[0043] In STEP 303, the image recognition circuit 105 performs image recognition processing, and as a result, captured image recognition data S105 of the camera itself is generated. The captured image recognition data S105 is input to the encryption circuit 112 and the comparison and determination circuit 116. The captured image recognition data S105 is data that represents the unique features of the subject 211, and is generated by a known technique, such as processing using a deep-learned convolutional neural network (CNN).

[0044] In STEP 304, the proximity sensor 205 (109) generates a proximity sensor signal S107 indicating whether or not the other camera (LiDAR sensor 208) is present in proximity to the camera itself (LiDAR sensor 203). The proximity sensor signal S107 is input to the system controller 110.

[0045] The proximity sensor 109 detects the presence of another camera in the vicinity of the own camera by detecting, for example, electromagnetic waves in the 6 GHz frequency band used in 5G (fifth generation mobile communication system). When another camera is present in the vicinity of the own camera, that is, when an environment in which distance measurement is performed on the same subject 211 by multiple cameras is detected, the process of STEP 305 is executed. On the other hand, when another camera is not present in the vicinity of the own camera, that is, when an environment in which distance measurement is performed on the same subject 211 by multiple cameras is not detected, the process of STEP 317 is executed.

[0046] In STEP 305, the system controller 110 determines whether or not it is possible to acquire image information from another camera via the communication unit 111 (whether or not communication with another camera can be started). For example, the system controller 110 sends a command to the other camera requesting the start of communication. If a response is received from the other camera within a predetermined time, it is determined that communication can be started, and if no response is received, it is determined that communication cannot be started. If it is determined that image information can be acquired from the other camera, STEP 306 is executed. On the other hand, if it is determined that image information can be acquired from the other camera, STEP 315 is executed.

[0047] The processing from STEP 306 to STEP 311 is a general encryption processing. In STEP 306, the system controller 110 executes a process of receiving the public key data of the other camera by the communication packet data S113 via the communication unit 111. The public key data S108 of the other camera is input to the encryption circuit 112.

[0048] In STEP 307, the captured image recognition data S105 of the camera itself is encrypted using the public key data S108 of the other camera by the encryption circuit 112. The encrypted captured image recognition data S109 of the camera itself is input to the system controller 110.

[0049] In STEP 308, the captured image recognition data S109 of the camera itself, encrypted by the communication packet data S113, is transmitted to the other camera by the system controller 110. In this way, the captured image recognition data S105 of the camera itself is encrypted and transmitted to the other camera, where it is used for processing.

[0050] In STEP 309, the system controller 110 transmits the public key data S110 of the camera itself, which has been stored in advance in the public key data memory 113, to the other cameras in the form of communication packet data S113. The other cameras perform the same processes as in STEP 306 to STEP 308.

[0051] In STEP 310, the system controller 110 acquires the encrypted captured image recognition data S112 of the other camera via the communication packet data S113. The captured image recognition data S112 of the other camera indicates the result of image recognition processing on the visible light image data as the second captured image generated by imaging using the CMOS sensor 207 in the other camera. The image recognition data S112 is input to the decoding circuit 115.

[0052] In STEP 311, the decryption circuit 115 performs a decryption process on the encrypted captured image recognition data S112 of the other camera using the private key data S111 of the camera itself that is stored in advance in the private key data memory 114. This generates decrypted captured image recognition data S114 of the other camera. The encrypted captured image recognition data S112 of the other camera has been encrypted by the other camera using the public key data S110 of the camera itself, and therefore can only be decrypted using the private key data S111 of the camera itself.

[0053] In STEP 312, the comparison and determination circuit 116 compares the captured image recognition data S105 of the camera itself with the captured image recognition data S114 of the other camera. As a result, a comparison and determination result S115 is generated and input to the system controller 110. This comparison process is performed by a known technique, such as processing by a deep learning CNN.

[0054] In STEP 313, the system controller 110 executes the following branching process based on the comparison determination result S115. If the comparison determination result S115 indicates that the captured image recognition data S105 of the own camera and the captured image recognition data S114 of the other camera are equivalent, it is determined (detected) that the subjects for which 3D models are generated in the own camera and the other camera, i.e., the objects for which LiDAR ranging is performed, are the same. In this case, the process of STEP 314 is executed. On the other hand, if the comparison determination result S115 indicates that the captured image recognition data S105 of the own camera and the captured image recognition data S114 of the other camera are not equivalent, it is determined that the objects for which LiDAR ranging is performed in the own camera and the other camera are not the same. In this case, the process of STEP 317 is executed.

[0055] In STEP 314, the system controller 110 performs the following processing. The system controller 110 performs predetermined communication with the other camera via communication packet data S113, and determines whether the other camera is of the same model as the subject camera. Specifically, by receiving identification information (model name, serial number, etc.) from the other camera and comparing it with the identification information of the subject camera, it determines whether the specifications of the LiDAR sensor of the other camera are the same as the specifications of the LiDAR sensor of the subject camera. If the other camera is not of the same model as the subject camera, the processing of STEP 315 is performed, and if they are the same model, the processing of STEP 316 is performed.

[0056] In STEP 315, the system controller 110 performs the following first distance measurement adjustment. The system controller 110 changes the pulse period 402 from the default value to the laser controller 107 via a laser controller control signal S116. In this change, the period may be lengthened or shortened.

[0057] It is also preferable to change the setting of laser intensity 403 in conjunction with a change in pulse period 402. Specifically, when pulse period 402 is shortened, the number of times laser light is emitted per unit time increases, so laser intensity 403 is changed to an intensity that satisfies the Eye-Safe standard for lasers. When pulse period 402 is lengthened, the number of times laser light is emitted per unit time decreases, so it is possible to increase laser intensity 403 within a range that satisfies the Eye-Safe standard for lasers. Furthermore, when pulse period 402 is changed, distance measurement period 404 may be changed so that the number of laser light pulses used for one distance measurement does not change. Distance measurement period 405 may also be changed in accordance with the change in distance measurement period 404.

[0058] At this time, the system controller 110 outputs a display control signal S119 to the display unit 117 to display the fact that the LiDARs are in a state of mutual interference. This display notifies the user of the impact on the generation result of the 3D model due to the pulse period 402, ranging period 404, and ranging cycle 405 being changed from the default values.

[0059] In STEP 316, the system controller 110 performs the following second distance measurement adjustment. The system controller 110 uses a laser controller control signal S116 to cause the laser controller 107 to set the distance measurement cycle 405 to a time that is at least twice as long as the distance measurement period 404. The system controller 110 also performs predetermined communication with the other cameras via the communication unit 111, and sets the distance measurement start timing 406 of the camera itself so that it does not overlap with the distance measurement start timing of the other cameras. The system controller 110 also outputs a display control signal S119 to the display unit 117 to notify the user of the impact on the generation results of the 3D model caused by changing the distance measurement cycle 405 to a time that is at least twice as long as the distance measurement period 404.

[0060] In STEP 317, laser controller 107 generates a laser pulse control signal S118 indicating pulse width 401, pulse period 402, laser intensity 403, distance measurement period 404, distance measurement period 405, and distance measurement start timing 406, and outputs the signal to infrared laser array 108. Infrared laser array 108 has a predetermined number of infrared laser elements arranged two-dimensionally horizontally and vertically. These infrared laser elements are arranged in correspondence with the two-dimensional arrangement of distance map data S104, which will be described later. The infrared laser elements arranged two-dimensionally in this manner irradiate infrared laser light to the outside in accordance with laser pulse control signal S118.

[0061] In the first case where no other camera is present near the own camera in STEP 304, the pulse width 401, pulse period 402, laser intensity 403, ranging period 404, ranging period 405, and ranging start timing 406 remain at their default values. This is also the case in the second case where the own camera and the other camera measure the distances to different subjects in STEP 313. In both the first and second cases, there is no mutual interference between the LiDARs. Therefore, by leaving the pulse width 401, pulse period 402, laser intensity 403, ranging period 404, ranging period 405, and ranging start timing 406 at their default values, the own camera performs LiDAR ranging with the optimal ranging range, ranging resolution, ranging accuracy, and ranging frequency.

[0062] In the third case, where the camera cannot start communicating image information with the other camera in STEP 305, the pulse width 401, pulse period 402, laser intensity 403, ranging period 404, ranging period 405, and ranging start timing 406 are changed as described in STEP 315. This also applies to the fourth case, where the camera and the other camera are not the same model in STEP 314. Both the third and fourth cases are situations where mutual interference between LiDARs may occur. In the third case, it is not possible to confirm whether the other camera is simultaneously measuring the distance to the same subject 211 as the camera, which would result in mutual interference. However, in this embodiment, the above changes are made with a priority on reducing (and preferably avoiding) mutual interference. In the fourth case, the above changes are made to the other camera that is not the same model, thereby minimizing the effects of mutual interference as much as possible.

[0063] In the fifth case where the own camera and the other camera are the same model in STEP 314, the changes described in STEP 316 are made to the pulse width 401, pulse period 402, laser intensity 403, ranging period 404, ranging cycle 405, and ranging start timing 406. In the fifth case, there is a high possibility that mutual interference between LiDARs will occur, and the own camera and the other camera are the same model, so the changes described in STEP 315 are the same for both the own camera and the other camera. As a result, mutual interference cannot be reduced. Therefore, by making the changes described in STEP 316, the ranging cycle 405 becomes longer by more than twice, but mutual interference can be more reliably reduced.

[0064] In STEP 318, the LiDAR sensor 203 (102) receives the laser light reflected from the subject 211. As a result, LiDAR ranging data S102 is generated. The LiDAR ranging data S102 may be generated by a known technique using information on the time of flight (TOF) of the laser light emitted from the infrared laser array 108 and reflected by the subject 211.

[0065] In STEP 319, the distance map generation circuit 104 generates distance map data S104 based on the LiDAR ranging data S102 and pulse period change information S117 generated by the laser controller 107. The distance map data S104 is configured as a distance map with the same number of two-dimensional positions as the infrared laser array 108 by performing histogramming processing on the LiDAR ranging data S102 generated by a known technique to improve ranging accuracy. Note that the distance map data S104 including the histogramming processing may also be generated by a known technique. The pulse period change information S117 is used for the histogramming processing.

[0066] In STEP 320, 3D model data S106 is generated from the visible light image data S103 and the distance map data S104 and output by the 3D model generation circuit 106. The 3D model data S106 may also be generated using known techniques.

[0067] In STEP 321, the generation of a 3D model for the subject 211 in the first camera is completed.

[0068] According to this embodiment, a state in which the own camera and another camera are performing LiDAR ranging on the same object, i.e., a state in which mutual interference of the LiDARs may occur, is detected by comparing the captured image information of the own camera and another camera. This makes it possible to more reliably detect the occurrence of mutual interference and control the LiDAR ranging of the own camera (change the settings related to LiDAR ranging), thereby reducing mutual interference.

[0069] Furthermore, according to this embodiment, the settings for reducing mutual interference in the LiDAR are changed after determining whether the camera itself and the other camera are the same model, which makes it possible to further reliably reduce mutual interference.

[0070] Furthermore, according to this embodiment, the user is notified of the impact that the mutual interference reduction measures will have on the results of generating the 3D model, so that the user can decide at the time of image capture whether to accept the impact or to re-generate the 3D model.

[0071] In this embodiment, a case has been described in which control is performed to reduce mutual LiDAR interference when it is detected that the subject camera and another camera are performing LiDAR ranging on the same object. Alternatively, control to reduce mutual LiDAR interference may be performed simply when it is detected that at least a portion of the imaging angles of view of the subject camera and another camera overlap. Furthermore, in this embodiment, a case has been described in which LiDAR ranging of the subject camera is controlled when a state in which mutual LiDAR interference may occur is detected. However, communication may be performed from the subject camera to another camera to control the LiDAR ranging of the other camera. Furthermore, both the LiDAR ranging of the subject camera and the LiDAR ranging of the other camera may be controlled. In other words, it is sufficient to control at least one of the LiDAR ranging of the subject camera and the LiDAR ranging of the other camera to reduce mutual interference. These are the same as in Example 2, which will be described later. [Example]

[0072] Next, a description will be given of Example 2. Fig. 5 shows the configuration of a first camera 100A of Example 2. The first camera 100A of this example is provided with an image cropping circuit 501 and an FFT calculation circuit 502 instead of the image recognition circuit 105 shown in Example 1 (Fig. 1).

[0073] In this embodiment, visible light image data S103 output from visible light image processing circuit 103 is input to image cropping circuit 501, which generates cropped image data S501 in which a predetermined image range has been cropped. The cropped image data S501 is input to FFT calculation circuit 502. FFT calculation circuit 502 performs two-dimensional FFT (Fast Fourier Transform) processing on the cropped image data S501, and generates two-dimensional FFT data (first data) S502 that indicates the result. The two-dimensional FFT data is image data that indicates spatial frequency.

[0074] In this embodiment, the two-dimensional FFT data S502 of the first camera output from the FFT calculation circuit 502 is used instead of the captured image recognition data S105 of the first camera in the first embodiment. Furthermore, the encrypted two-dimensional FFT data S503 of the first camera is used instead of the encrypted captured image recognition data S109 of the first camera in the first embodiment. Furthermore, the encrypted two-dimensional FFT data S504 of the second camera is used instead of the encrypted captured image recognition data S112 of the second camera in the first embodiment, and the two-dimensional FFT data (second data) S505 of the second camera is used instead of the captured image recognition data S114 of the second camera in the first embodiment. Other components and data are the same as those in the first embodiment. Furthermore, the second camera in this embodiment has a configuration similar to that of the first camera in this embodiment.

[0075] The flowchart in Fig. 6 shows the processing executed by the system controller 110 of the first camera (own camera) according to a program. In this embodiment, STEP 303 of the first embodiment (Fig. 3) is eliminated, STEP 309 is provided after STEP 306, and STEP 603 is provided after STEP 309. Also, STEP 307 of the first embodiment is changed to STEP 607, and STEP 308 is changed to STEP 608. Also, STEP 310 of the first embodiment is changed to STEP 610, STEP 311 is changed to STEP 611, and STEP 312 is changed to STEP 612. The other steps are the same as those in the first embodiment.

[0076] After performing the processes in STEP 300 to STEP 306 and the next STEP 309, the system controller 110 causes the image cropping circuit 501 to crop an image from the visible light image data S103 in STEP 603 to generate cropped image data S501. The cropped image data S501 is input to the FFT calculation circuit 502. The system controller 110 causes the FFT calculation circuit 502 to perform two-dimensional FFT processing on the cropped image data S501. As a result, two-dimensional FFT data S502 of the camera itself is generated.

[0077] The cropped image data S501 is generated so that the image size is a power of 2 for two-dimensional FFT processing by the FFT calculation circuit 502. The camera's own two-dimensional FFT data S502 indicates information on the intensity of the spatial frequency components of the cropped image data S501, and although the amount of information is equivalent to that of the cropped image data S501, it has the robust feature of not changing with respect to shifts of the image within a two-dimensional plane.

[0078] In the first embodiment, the visible light image data S103 is directly used by the image recognition circuit 105 to generate the captured image recognition data S105 of the camera itself using deep learning CNN, etc. In contrast, in the present embodiment, the robust characteristics of the two-dimensional FFT data S502 of the camera itself are utilized.

[0079] Next, in STEP 607, the system controller 110 causes the encryption circuit 112 to encrypt the two-dimensional FFT data S503 of the camera itself.

[0080] Next, in STEP 608, the system controller 110 transmits the encrypted two-dimensional FFT data S503 of the camera itself via the communication unit 111 to the other cameras.

[0081] Next, in STEP 610, the system controller 110 receives the encrypted two-dimensional FFT data S504 of the other camera from the other camera.

[0082] Next, in STEP 611, the system controller 110 causes the decoding circuit 115 to decode the received encrypted two-dimensional FFT data S504 from the other camera.

[0083] Next, in STEP 612, the system controller 110 causes the comparison and determination circuit 116 to compare the two-dimensional FFT data S502 of the camera itself with the decoded two-dimensional FFT data S505 of the other camera. As a result, a comparison and determination result S115 is generated and input to the system controller 110.

[0084] Note that the processing from STEP 603 to STEP 612 may be performed multiple times in small block units, taking into consideration the image size of the visible light image data S103 and the image size in the 2D FFT processing. This is because the number of pixels on which the 2D FFT processing is performed is generally small compared to the number of pixels in the image sensor, in terms of the size of the circuitry and the computational load involved in the processing.

[0085] The processing from STEP 313 to STEP 320 and the end at STEP 321 are the same as in the first embodiment.

[0086] As described above, according to this embodiment, a state in which mutual interference between LiDARs may occur is detected by comparing the 2D FFT data of the camera itself with the 2D FFT data of the other camera. The 2D FFT data has a robust characteristic in that it does not change with respect to image shifts within a two-dimensional plane. Therefore, compared to the first embodiment, it is possible to more reliably detect the occurrence of mutual interference and reduce the mutual interference.

[0087] The above embodiment includes the following configurations.

[0088] (Configuration 1) a first imaging means for capturing an image within a first angle of view; a first distance measuring means for measuring a distance to an object included in the first angle of view; a control means for controlling distance measurement; A ranging device characterized in that, when performing ranging on an object that is included in a second angle of view imaged by a second imaging means and that is also imaged by a second imaging means and that is separate from the first ranging means, the control means detects ranging on the same object by the first ranging means and the second ranging means using first data generated using the first imaging means and second data generated using the second imaging means. (Configuration 2) 2. The distance measuring device according to claim 1, wherein the control means changes a setting relating to distance measurement of at least one of the first distance measuring means and the second distance measuring means when distance measurement of the same object is detected. (Configuration 3) 3. The distance measuring device according to configuration 2, wherein the control means changes the setting so as to reduce mutual interference between the first distance measuring means and the second distance measuring means. (Configuration 4) the first distance measuring means and the second distance measuring means each measure distance by emitting light toward the object and receiving light reflected from the object; 4. The distance measuring device according to configuration 2 or 3, wherein the control means changes a setting relating to light emission of at least one of the first distance measuring means and the second distance measuring means. (Configuration 5) 5. The distance measuring device according to configuration 4, wherein the control means changes at least one setting of the pulse width of the light, the light emission cycle of the light, and the intensity of the light. (Configuration 6) 4. The distance measuring device according to configuration 2 or 3, wherein the control means changes at least one setting among a period during which distance measurement is performed, a cycle of distance measurement, and a timing to start distance measurement. (Configuration 7) The distance measuring device according to any one of configurations 2 to 6, wherein the control means performs different changes to the settings when the specifications of the first distance measuring means and the second distance measuring means are the same and when the specifications are not the same. (Configuration 8) 8. The distance measuring device according to any one of configurations 2 to 7, wherein the control means changes the setting when it detects that the second distance measuring means is approaching the first distance measuring means. (Configuration 9) 9. The distance measuring device according to any one of configurations 2 to 8, wherein the control means notifies the user when the setting is changed. (Configuration 10) A ranging device according to any one of configurations 1 to 9, characterized in that the first data and the second data are data indicating the result of subject recognition processing on a captured image or data indicating the result of two-dimensional FFT processing on a captured image, respectively. (Configuration 11) a first imaging means for capturing an image within a first angle of view; a first distance measuring means for measuring a distance to an object included in the first angle of view; a control means for controlling distance measurement; A distance measuring device characterized in that, when measuring the distance to an object that is included in a second angle of view imaged by a second imaging means and that is also imaged by a second imaging means using a second distance measuring means separate from the first distance measuring means, the control means changes the distance measurement settings of at least one of the first distance measuring means and the second distance measuring means based on first data generated using the first imaging means and second data generated using a second imaging means that images within the second angle of view. (Configuration 12) 12. An electronic device comprising the distance measuring device according to any one of configurations 1 to 11. (Other Examples) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0089] The embodiments described above are merely representative examples, and various modifications and alterations are possible to each embodiment when implementing the present invention. [Explanation of symbols]

[0090] 100 Camera 1 101 CMOS sensor 102 LiDAR sensors 110 System Controller 200 Second Camera 207 CMOS sensor 208 LiDAR sensors

Claims

1. a first imaging means for capturing an image within a first angle of view; a first distance measuring means for measuring a distance to an object included in the first angle of view; a control means for controlling distance measurement; A ranging device characterized in that, when performing ranging on an object that is included in a second angle of view imaged by a second imaging means and that is also imaged by a second imaging means using a second ranging means separate from the first ranging means, the control means detects ranging on the same object by the first ranging means and the second ranging means using first data generated using the first imaging means and second data generated using the second imaging means.

2. 2. The distance measuring device according to claim 1, wherein the control means changes a setting relating to distance measurement of at least one of the first distance measuring means and the second distance measuring means when distance measurement for the same object is detected.

3. 3. The distance measuring device according to claim 2, wherein said control means changes said setting so as to reduce mutual interference between said first distance measuring means and said second distance measuring means.

4. the first distance measuring means and the second distance measuring means each measure distance by emitting light toward the object and receiving light reflected from the object; 3. The distance measuring device according to claim 2, wherein the control means changes a setting relating to light emission of at least one of the first distance measuring means and the second distance measuring means.

5. 5. The distance measuring device according to claim 4, wherein the control means changes at least one setting of the pulse width of the light, the light emission cycle of the light, and the intensity of the light.

6. 3. The distance measuring device according to claim 2, wherein the control means changes at least one setting of a period during which distance measurement is performed, a cycle of distance measurement, and a timing at which distance measurement is started.

7. The distance measuring device according to claim 2, characterized in that the control means makes different changes to the settings when the specifications of the first distance measuring means and the second distance measuring means are the same and when the specifications are not the same.

8. 3. The distance measuring device according to claim 2, wherein the control means changes the setting when it detects that the second distance measuring means is approaching the first distance measuring means.

9. 3. The distance measuring device according to claim 2, wherein the control means notifies the user when the setting is changed.

10. 2. The distance measuring device according to claim 1, wherein the first data and the second data are data indicating a result of subject recognition processing on a captured image or data indicating a result of two-dimensional FFT processing on a captured image, respectively.

11. a first imaging means for capturing an image within a first angle of view; a first distance measuring means for measuring a distance to an object included in the first angle of view; a control means for controlling distance measurement; A ranging device characterized in that, when performing ranging on an object that is included within a second angle of view imaged by a second imaging means and that is also imaged by a second imaging means using a second ranging means separate from the first ranging means, the control means changes the ranging settings of at least one of the first ranging means and the second ranging means based on first data generated using the first imaging means and second data generated using a second imaging means that images within the second angle of view.

12. An electronic device comprising the distance measuring device according to any one of claims 1 to 11.

13. A distance measurement control method in the case where there are a first distance measurement means and a second distance measurement means, each of which performs distance measurement, comprising: acquiring first data generated by the first distance measuring means capturing an image within a first angle of view that includes an object to be measured for distance, and second data generated by the second distance measuring means capturing an image within a second angle of view that includes an object to be measured for distance; a step of detecting distances measured by the first distance measuring means and the second distance measuring means to the same object using the first data and the second data.

14. A distance measurement control method in the case where there are a first distance measurement means and a second distance measurement means, each of which performs distance measurement, comprising: acquiring first data generated by the first distance measuring means capturing an image within a first angle of view that includes an object to be measured for distance, and second data generated by the second distance measuring means capturing an image within a second angle of view that includes an object to be measured for distance; A distance measurement control method, comprising: changing a setting relating to distance measurement of at least one of said first distance measurement means and said second distance measurement means based on said first data and said second data.

15. 15. A program for causing a computer to execute processing according to the distance measurement control method according to claim 13 or 14.

Images