Information processing apparatus, control apparatus, information processing method, and information processing program

Abstract

The present application provides an information processing device capable of work support for a subject, a control device, an information processing method, and an information processing program. The information processing device has a CPU (60A) capable of communicating with a camera device (10) and a rotating device (16) that rotates the camera device (10). The CPU (60A) performs the following processes: acquires first position information according to a rotation condition of the rotating device (16) corresponding to a failure site (92) of a power transmission line (91); acquires second position information according to a rotation condition of the rotating device (16) corresponding to an end portion (A) of the power transmission line (91) connected to a tower (81); acquires third position information according to a rotation condition of the rotating device (16) corresponding to an end portion (B) of the power transmission line (91) connected to a tower (82); and outputs distance information between any one of the end portion (A) of the power transmission line (91) and the end portion (B) of the power transmission line (91) and the failure site (92) according to the first position information, the second position information, and the third position information.

Description

Technical Field

[0001] This invention relates to an information processing device, a control device, an information processing method, and an information processing program. Background Technology

[0002] Patent document 1 describes an overhead line photography device and a video camera device, which includes: a telescope for observing the overhead line from a distance; a video camera device for capturing images of the overhead line; a gimbal mechanism that can rotate in the yaw and pitch directions; a yaw control unit; a pitch control unit; and a focusing sliding control unit, which, in order to understand the damage condition of the overhead line, takes images of the overhead line from the ground within a specified length range.

[0003] Patent document 2 describes a main camera that has a range-measuring function that utilizes autofocus, enabling it to measure the distance to an object and to measure the position of the object in a three-dimensional world coordinate system based on the line of sight (pan / pitch angle) and the measured distance.

[0004] Patent document 3 describes an overhead line sag monitoring system, which includes a camera, a known data acquisition mechanism, a coordinate transformation unit, an action control unit, a screen display unit, and a monitoring / computing unit. The system performs image correction processing on still images of the tower being measured, captured by the camera, to match the images with the structural data of the tower acquired by the known data acquisition mechanism. Figure 1 After that, the coordinates of the bottom of the overhead line's sag are calculated based on the corrected still image.

[0005] Previous technical documents Patent documents Patent Document 1: Japanese Utility Model Registration No. 3220181 Patent Document 2: Japanese Patent Application Publication No. 2000-083246 Patent Document 3: Japanese Patent Application Publication No. 2002-112421 Summary of the Invention

[0006] One embodiment of the present invention provides an information processing apparatus, control device, information processing method, and information processing program capable of providing operational support for a subject.

[0007] means for solving technical problems (1) An information processing apparatus includes a processor capable of communicating with a camera device and a rotating device that rotates the camera device. The processor described above performs the following processing: The first position information is obtained based on the rotation conditions of the aforementioned rotating device corresponding to the first subject; The second position information is obtained based on the rotation conditions of the aforementioned rotating device corresponding to the second subject; The third position information is obtained based on the rotation conditions of the aforementioned rotating device corresponding to the third subject; Based on the first position information, the second position information, and the third position information, output the distance information between the first subject and any one of the second or third subjects. (2) According to the information processing apparatus described in (1), wherein, The aforementioned distance information refers to the distance between the subject closer to the camera device among the second and third subjects and the first subject. (3) According to the information processing apparatus described in (1) or (2), wherein, The first, second, and third subjects mentioned above exist on the first surface. (4) The information processing apparatus according to any one of (1) to (3), wherein, The processor described above performs the following processing: The first position information is obtained based on the rotation conditions of the rotating device corresponding to the first subject and the imaging conditions of the imaging device. The second position information is obtained based on the rotation conditions of the rotating device corresponding to the second subject and the imaging conditions of the imaging device. The third position information is obtained based on the rotation conditions of the rotating device corresponding to the third subject and the imaging conditions of the imaging device. (5) The information processing apparatus according to any one of (1) to (3), wherein, The second subject mentioned above is the subject that is closer to the camera device than the third subject mentioned above. The processor described above performs the following processing: The second position information is obtained based on the rotation conditions of the rotating device corresponding to the second subject and the imaging conditions of the imaging device. The third position information is obtained based on the rotation conditions of the rotating device corresponding to the second subject, the imaging conditions of the imaging device, and the rotation conditions of the rotating device corresponding to the third subject. (6) The information processing apparatus according to any one of (1) to (3), wherein, The second subject mentioned above is the subject that is closer to the camera device than the third subject mentioned above. The processor described above performs the following processing: The second position information is obtained based on the rotation conditions of the rotating device corresponding to the second subject and the imaging conditions of the imaging device. The third position information is obtained based on the second position information, the distance between the second subject and the third subject, and the rotation conditions of the rotating device corresponding to the third subject. (7) According to the information processing apparatus described in (5) or (6), wherein, The aforementioned first subject, second subject, and third subject exist on the first surface. The processor described above performs the following processing: The first position information is obtained based on the second position information, the third position information, and the rotation conditions of the rotating device corresponding to the first subject. (8) The information processing apparatus according to any one of (5) to (7), wherein, The processor described above performs the following processing: Based on the imaging conditions of the imaging device corresponding to the second subject and the imaging conditions of the imaging device corresponding to the third subject, it is determined that the second subject is closer to the imaging device than the third subject. (9) The information processing apparatus according to any one of (5) to (8), wherein, The processor performs the following processing: it obtains the third position information based on information indicating the relative height relationship between the second and third subjects. (10) The information processing apparatus according to any one of (1) to (9), wherein, The rotation conditions of the aforementioned slewing device include at least one of the slewing device's panning and pitching states. (11) The information processing apparatus according to any one of (1) to (10), wherein, The recording conditions of the aforementioned camera device include the focusing information of the aforementioned camera device. (12) A control device that controls the camera device and / or the rotary device based on information generated by the information processing device according to any one of (1) to (11). (13) An information processing method is provided, which is executed by an information processing device equipped with a processor, the processor being capable of communicating with a camera device and a rotating device that rotates the camera device. In the aforementioned information processing method, The processor described above performs the following processing: The first position information is obtained based on the rotation conditions of the aforementioned rotating device corresponding to the first subject; The second position information is obtained based on the rotation conditions of the aforementioned rotating device corresponding to the second subject; The third position information is obtained based on the rotation conditions of the aforementioned rotating device corresponding to the third subject; Based on the first position information, the second position information, and the third position information, output the distance information between the first subject and any one of the second or third subjects. (14) An information processing program is executed by an information processing device equipped with a processor, the processor being capable of communicating with a camera device and a rotating device that rotates the camera device, the information processing program causing the processor to perform the following processing: The first position information is obtained based on the rotation conditions of the aforementioned rotating device corresponding to the first subject; The second position information is obtained based on the rotation conditions of the aforementioned rotating device corresponding to the second subject; The third position information is obtained based on the rotation conditions of the aforementioned rotating device corresponding to the third subject; Based on the first position information, the second position information, and the third position information, output the distance information between the first subject and any one of the second or third subjects.

[0021] Invention Effects According to the present invention, an information processing apparatus, a control apparatus, an information processing method, and an information processing program capable of providing operational support for a subject can be provided. Attached Figure Description

[0022] Figure 1 This diagram illustrates an example of a camera system 1 equipped with a control device according to this embodiment.

[0023] Figure 2 This is a block diagram illustrating an example of the structure of the electrical system of the rotary device 16 and the personal computer 11.

[0024] Figure 3 This is a block diagram illustrating an example of the structure of the optical and electrical systems of the camera device 10.

[0025] Figure 4This is a schematic diagram illustrating an example of maintenance work being carried out on a power transmission line.

[0026] Figure 5 This is a flowchart illustrating an example of processing performed by the CPU 60A of the information processing device 60.

[0027] Figure 6 This is an example diagram showing distance information displayed on the monitor 13a of the personal computer 11.

[0028] Figure 7 This is a diagram showing a modified example of distance information displayed on the monitor 13a of the personal computer 11.

[0029] Figure 8 This diagram illustrates the error that occurs when calculating the distance from the camera device to the tower based on focus information.

[0030] Figure 9 This is a flowchart illustrating a first modified example of the processing performed by the CPU 60A of the information processing device 60.

[0031] Figure 10 This is an explanation Figure 9 A diagram showing the calculation of the spatial coordinates of the distant reference position in the first variation.

[0032] Figure 11 This is a flowchart illustrating a second variation of the processing performed by the CPU 60A of the information processing device 60.

[0033] Figure 12 This is an explanation Figure 11 The diagram shows the calculation of the spatial coordinates of the distant reference position in the second variation.

[0034] Figure 13 This diagram illustrates an example of how an information processing device 60 for a personal computer 11 can install an information processing program from a storage medium containing an information processing program for camera control. Detailed Implementation

[0035] Hereinafter, an example of an embodiment of the present invention will be described with reference to the accompanying drawings.

[0036] <Camera System in Implementation> Figure 1 This diagram illustrates an example of the camera system 1 equipped with a control device according to this embodiment. Figure 1 As shown, the camera system 1 includes a camera device 10, a personal computer (PC) 11, and a rotating device 16. The personal computer (PC) 11 is an example of the "control device" in this invention.

[0037] The camera device 10 is mounted on a pillar, wall, or part of a building (e.g., roof) indoors or outdoors via a rotating device 16, and captures images of the subject. The camera device 10 is, for example, a camera capable of capturing distant views. The camera device 10 transmits the captured images and related information to a personal computer 11 via a communication line 12.

[0038] The personal computer 11 includes a display 13a, a keyboard 13b, a mouse 13c, and a secondary storage device 14. Examples of displays 13a include liquid crystal displays, plasma displays, and organic EL (electro-Luminescence) displays.

[0039] As an example of secondary storage device 14, an HDD (Hard Disk Drive) can be cited. Secondary storage device 14 is not limited to HDD, and can be any non-volatile memory such as flash memory, SSD (Solid State Drive) or EEPROM (Electrically Erasable and Programmable Read Only Memory).

[0040] The personal computer 11 receives video images or related information sent from the camera device 10, and displays the received video images or related information on the display 13a or stores them in the secondary storage device 14.

[0041] For example, the personal computer 11 communicates with the camera device 10 via communication line 12 to perform camera control. Camera control involves setting camera parameters for shooting on the camera device 10 and controlling the camera device 10 to perform shooting. Camera parameters include parameters related to exposure or parameters related to the width of the shooting range (focal length), etc.

[0042] Furthermore, the personal computer 11 communicates with the rotation device 16 via the communication line 12 to perform rotation control. Rotation control involves the rotation device 16 setting rotation parameters for rotating the camera device 10 and then controlling the rotation of the camera device 10. These rotation parameters include parameters related to the direction (pan / tilt) of the camera's field of view.

[0043] The personal computer 11 can set the rotation direction, rotation amount, rotation speed, etc. of the camera device 10 based on the operation of the keyboard 13b or mouse 13c or the touch operation of the screen of the monitor 13a.

[0044] <Structure of the electrical system of rotary device 16 and personal computer 11> Figure 2 This is a block diagram illustrating an example of the structure of the electrical system of the rotary device 16 and the personal computer 11. For example... Figure 2 As shown, the slewing device 16 includes a yaw axis slewing mechanism 71, a pitch axis slewing mechanism 72, a motor 73, 74, a driver 75, 76, and a communication I / F 78.

[0045] The yaw axis rotation mechanism 71 rotates the rotating device 16, on which the camera device 10 is mounted, in the yaw direction. The motor 73 generates power under the control of the driver 75. The yaw axis rotation mechanism 71, by receiving power generated by the motor 73, rotates the rotating device 16, on which the camera device 10 is mounted, in the yaw direction. The pitch axis rotation mechanism 72 rotates the rotating device 16, on which the camera device 10 is mounted, in the pitch direction. The motor 74 generates power under the control of the driver 76. The pitch axis rotation mechanism 72, by receiving power generated by the motor 74, rotates the rotating device 16, on which the camera device 10 is mounted, in the pitch direction.

[0046] The communication I / F78 is, for example, a network interface. The communication I / F78 controls the transmission of various information between itself and the personal computer 11. This network is, for example, a WAN (Wide Area Network) or LAN (Local Area Network). The communication I / F78 facilitates communication between the rotary device 16 and the personal computer 11.

[0047] The personal computer 11 includes a display 13a, a secondary storage device 14, an information processing device 60, a receiving device 62, and communication I / Fs 66 and 68. The information processing device 60 includes a CPU 60A, a storage device 60B, and a memory 60C.

[0048] The receiving device 62, display 13a, secondary storage device 14, CPU 60A, storage device 60B, memory 60C, and communication I / O devices 66 and 68 are each connected to bus 69. Additionally, Figure 2 In the example shown, for ease of illustration, bus 69 is represented as a single bus, but multiple buses can also be represented. Bus 69 can be a serial bus or a parallel bus that includes a data bus, an address bus, and a control bus.

[0049] The memory 60C temporarily stores various information and serves as working memory. RAM can be cited as an example of the memory 60C, but it is not limited to this; other types of storage devices may also be used. Various programs for the personal computer 11 (hereinafter referred to as "personal computer programs") are stored in the storage device 60B.

[0050] CPU 60A reads a personal computer program from storage device 60B and executes the read personal computer program on memory 60C, thereby controlling the personal computer 11 as a whole. The personal computer program includes the "information processing program" of this invention. CPU 60A is an example of the "processor" of this invention.

[0051] Communication I / F 66 is, for example, a network interface. Communication I / F 66 can be communicatively connected to the communication I / F (not shown) of the camera device 10, and performs various information transmission control between them. Communication I / F 68 is, for example, a network interface. Communication I / F 68 can be communicatively connected to the communication I / F 78 of the yaw axis slewing mechanism 16, and performs various information transmission control between the yaw axis slewing mechanism 71 and the pitch axis slewing mechanism 72.

[0052] CPU 60A receives camera images or camera-related information from camera device 10 via communication I / F 66. Furthermore, CPU 60A controls the driver 75 and motor 73 of rotary device 16 via communication I / F 68 and communication I / F 78, thereby controlling the rotation of yaw axis rotary mechanism 71, and controls the driver 76 and motor 74 of rotary device 16, thereby controlling the rotation of pitch axis rotary mechanism 72.

[0053] The receiving device 62 is, for example, a keyboard 13b, a mouse 13c, and a touch panel of a display 13a, which receives various instructions from the user. The CPU 60A acquires the various instructions received by the receiving device 62 and performs actions according to the acquired instructions. For example, when the receiving device 62 receives processing content for the camera device 10 or the rotating device 16, the CPU 60A causes the camera device 10 or the rotating device 16 to operate according to the instructions received by the receiving device 62.

[0054] The display 13a displays various information under the control of the CPU 60A. Examples of the various information displayed on the display 13a include various instructions received by the receiving device 62, and camera images of the subject received by the communication I / F 66, or information related to camera recording. The CPU 60A causes the various instructions received by the receiving device 62 and the camera images of the subject received by the communication I / F 66, or information related to camera recording, to be displayed on the display 13a.

[0055] Secondary storage device 14 is, for example, a non-volatile memory, which stores various information under the control of CPU 60A. Examples of the various information stored in secondary storage device 14 include, for instance, camera images of the subject received by communication I / F 66 or information related to camera recording. CPU 60A stores camera images of the subject received by communication I / F 66 or information related to camera recording in secondary storage device 14.

[0056] <Structure of the optical and electrical systems of the camera device 10> Figure 3 This is a block diagram illustrating an example of the structure of the optical and electrical systems of the camera device 10. For example... Figure 3 As shown, the imaging device 10 includes an optical system 15 and an imaging element 25. The imaging element 25 is located after the optical system 15. The optical system 15 includes an objective lens 15A and a lens group 15B. The objective lens 15A and the lens group 15B are arranged in the order of objective lens 15A and lens group 15B along the optical axis OA of the optical system 15 from the subject side (object side) to the light-receiving surface 25A side (image side) of the imaging element 25. The lens group 15B includes a vibration-damping lens 15B1, a focusing lens (not shown), and a zoom lens 15B2. The zoom lens 15B2 is supported so that it can be moved along the optical axis OA by a lens actuator 21 described later. The vibration-damping lens 15B1 is supported so that it can be moved in a direction orthogonal to the optical axis OA by a lens actuator 17 described later.

[0057] By using the zoom lens 15B2 to extend the focal length, the camera device 10 becomes the telephoto side, thus the angle of view becomes smaller (the imaging range becomes narrower). By using the zoom lens 15B2 to shorten the focal length, the camera device 10 becomes the wide-angle side, thus the angle of view becomes larger (the imaging range becomes wider).

[0058] In addition to the objective lens 15A and lens group 15B, the optical system 15 may also include various lenses (not shown). Furthermore, the optical system 15 may include an aperture. The positions of the lenses, lens groups, and aperture included in the optical system 15 are not limited; for example, even with... Figure 3 The technology of this invention is still applicable in different locations as shown.

[0059] The anti-vibration lens 15B1 can move in a direction perpendicular to the optical axis OA, and the zoom lens 15B2 can move along the optical axis OA.

[0060] The optical system 15 includes lens actuators 17 and 21. Lens actuator 17 applies a force to the anti-vibration lens 15B1 in a direction perpendicular to the optical axis of the anti-vibration lens 15B1. Lens actuator 17 is controlled by OIS (Optical Image Stabilizer) driver 23. By driving lens actuator 17 under the control of OIS driver 23, the position of anti-vibration lens 15B1 is changed in a direction perpendicular to the optical axis OA.

[0061] Lens actuator 21 causes a force for moving the zoom lens 15B2 along the optical axis OA of the optical system 15. Lens actuator 21 is controlled by lens driver 28. By driving lens actuator 21 under the control of lens driver 28, the position of zoom lens 15B2 moves along the optical axis OA. As the position of zoom lens 15B2 moves along the optical axis OA, the focal length of imaging device 10 changes.

[0062] Additionally, the outline of the camera image is, for example, in the pitch axis PA direction (reference). Figure 1 It has a short side and is in the yaw axis YA direction (refer to) Figure 1 In the case of a rectangle with a long side, the viewing angle along the pitch axis PA is narrower than that along the yaw axis YA, and also narrower than that along the diagonal.

[0063] Using the optical system 15 configured in this way, light representing the imaging area is imaged onto the light-receiving surface 25A of the imaging element 25, and the imaging element 25 captures the imaging area.

[0064] The vibrations applied to the camera device 10 can be caused by various factors, such as outdoor vibrations (e.g., vibrations caused by passing vehicles, wind, and road construction) or indoor vibrations (e.g., vibrations caused by the operation of air conditioners and people entering and exiting the device). Therefore, the camera device 10 experiences shaking due to the vibrations applied to it (hereinafter also referred to as "vibrations").

[0065] Furthermore, in this embodiment, "jitter" refers to the phenomenon in the imaging device 10 where the image of the subject on the light-receiving surface 25A of the imaging element 25 changes due to a change in the positional relationship between the optical axis OA and the light-receiving surface 25A. In other words, "jitter" can also be described as the phenomenon where the optical image obtained by the imaging device 10 is altered due to the tilting of the optical axis OA caused by vibration applied to the imaging device 10, resulting in a change in the image formed on the light-receiving surface 25A. The change in the optical axis OA refers to, for example, the tilting of the optical axis OA relative to a reference axis (e.g., the optical axis OA before the jitter occurs). Hereinafter, the jitter caused by vibration will also be simply referred to as "jitter".

[0066] Shaking is a noise component contained in the captured image and affects the image quality. Therefore, in order to remove the noise component caused by shaking and contained in the captured image, the camera device 10 is equipped with a lens-side shake correction mechanism 29, an imaging element-side shake correction mechanism 45 and an electronic shake correction unit 33 to correct shaking.

[0067] The lens-side shake correction mechanism 29 and the imaging element-side shake correction mechanism 45 are mechanical shake correction mechanisms. The mechanical shake correction mechanism is a mechanism that corrects shake by applying power generated by a drive source such as a motor (e.g., a voice coil motor) to the shake correction element (e.g., the anti-vibration lens 15B1 and / or the imaging element 25), thereby moving the shake correction element in a direction perpendicular to the optical axis of the camera optical system.

[0068] Specifically, the lens-side shake correction mechanism 29 corrects shake by applying power generated by a drive source such as a motor (e.g., a voice coil motor) to the anti-vibration lens 15B1, causing the anti-vibration lens 15B1 to move in a direction perpendicular to the optical axis of the imaging optical system. The imaging element-side shake correction mechanism 45 corrects shake by applying power generated by a drive source such as a motor (e.g., a voice coil motor) to the imaging element 25, causing the imaging element 25 to move in a direction perpendicular to the optical axis of the imaging optical system. The electronic shake correction unit 33 corrects shake by performing image processing on the captured image based on the amount of shake. That is, the shake correction unit (shake correction assembly) performs shake correction mechanically or electronically using hardware and / or software structures. Here, mechanical jitter correction refers to jitter correction achieved by mechanically moving jitter correction elements such as the anti-vibration lens 15B1 and / or the imaging element 25 using power generated by a drive source such as a motor (e.g., a voice coil motor), while electronic jitter correction refers to jitter correction achieved, for example, by performing image processing using a processor.

[0069] As an example, such as Figure 3 As shown, the lens-side jitter correction mechanism 29 includes an anti-vibration lens 15B1, a lens actuator 17, an OIS driver 23, and a position sensor 39.

[0070] As a method for jitter correction performed by the lens-side jitter correction mechanism 29, various known methods can be employed. In this embodiment, the jitter correction method employs a method that corrects jitter by moving the anti-vibration lens 15B1 according to the amount of jitter detected by the jitter amount detection sensor 40 (described later). Specifically, jitter correction is performed by moving the anti-vibration lens 15B1 in the jitter-eliminating direction only by an amount that eliminates jitter.

[0071] A lens actuator 17 is mounted on the anti-vibration lens 15B1. The lens actuator 17 is a displacement mechanism equipped with a voice coil motor, which drives the voice coil motor to move the anti-vibration lens 15B1 in a direction perpendicular to the optical axis of the anti-vibration lens 15B1. While a displacement mechanism equipped with a voice coil motor is used as the lens actuator 17 here, the technology of the present invention is not limited to this; other power sources such as stepper motors or piezoelectric elements can also be used instead of the voice coil motor.

[0072] The lens actuator 17 is controlled by the OIS driver 23. By driving the lens actuator 17 under the control of the OIS driver 23, the position of the anti-vibration lens 15B1 is mechanically varied in a two-dimensional plane perpendicular to the optical axis OA.

[0073] Position sensor 39 detects the current position of anti-vibration lens 15B1 and outputs a position signal indicating the detected current position. Here, as an example of position sensor 39, a device including a Hall element is used. Here, the current position of anti-vibration lens 15B1 refers to its current position within a two-dimensional plane of the anti-vibration lens. The two-dimensional plane of the anti-vibration lens refers to a two-dimensional plane perpendicular to the optical axis of the anti-vibration lens 15B1. Furthermore, in this embodiment, as an example of position sensor 39, a device including a Hall element is used; however, the technology of the present invention is not limited to this, and a magnetic sensor or optical sensor, etc., can be used instead of a Hall element.

[0074] The lens-side shake correction mechanism 29 corrects shake by moving the anti-shake lens 15B1 within the actual shooting range along at least one of the pitch axis PA direction and the yaw axis YA direction. That is, the lens-side shake correction mechanism 29 corrects shake by moving the anti-shake lens 15B1 within the two-dimensional plane of the anti-shake lens by an amount of movement corresponding to the amount of shake.

[0075] The imaging element side jitter correction mechanism 45 includes an imaging element 25, a BIS (Body Image Stabilizer) driver 22, an imaging element actuator 27, and a position sensor 47.

[0076] Similar to the shake correction method based on the lens-side shake correction mechanism 29, the shake correction method based on the imaging element-side shake correction mechanism 45 can also employ various well-known methods. In this embodiment, the shake correction method employs a method that corrects shake by moving the imaging element 25 according to the amount of shake detected by the shake amount detection sensor 40. Specifically, shake correction is performed by moving the imaging element 25 in the shake-eliminating direction only by an amount that eliminates shake.

[0077] An imaging element actuator 27 is mounted on the imaging element 25. The imaging element actuator 27 is a displacement mechanism equipped with a voice coil motor, which drives the voice coil motor to move the imaging element 25 in the vertical direction relative to the optical axis of the anti-vibration lens 15B1. Furthermore, while a displacement mechanism equipped with a voice coil motor is used as the imaging element actuator 27 here, the technology of the present invention is not limited to this; other power sources such as stepper motors or piezoelectric elements can also be used instead of the voice coil motor.

[0078] The imaging element actuator 27 is controlled by the BIS driver 22. By driving the imaging element actuator 27 under the control of the BIS driver 22, the position of the imaging element 25 is mechanically changed in a direction perpendicular to the optical axis OA.

[0079] The position sensor 47 detects the current position of the imaging element 25 and outputs a position signal indicating the detected current position. Here, as an example of the position sensor 47, a device including a Hall element is used. Here, the current position of the imaging element 25 refers to its current position within a two-dimensional plane of the imaging element. The two-dimensional plane of the imaging element refers to a two-dimensional plane perpendicular to the optical axis of the anti-vibration lens 15B1. Furthermore, in this embodiment, as an example of the position sensor 47, a device including a Hall element is used; however, the technology of the present invention is not limited to this, and a magnetic sensor or an optical sensor, etc., can be used instead of a Hall element.

[0080] The camera device 10 includes a computer 19, a DSP (Digital Signal Processor) 31, an image memory 32, an electronic image correction unit 33, a communication I / F 34, a image shake sensor 40, and a UI (User Interface) system device 43. The computer 19 includes a memory 35, a storage device 36, and a CPU (Central Processing Unit) 37. The camera device 10 uses a machine learning model within the processor of the computer 19 to detect a specific subject. The processor can be, for example, the CPU 37, or other processors.

[0081] Imaging element 25, DSP 31, image memory 32, electronic jitter correction unit 33, communication I / F 34, memory 35, storage device 36, CPU 37, jitter detection sensor 40, and UI system device 43 are connected to bus 38. Furthermore, OIS driver 23 is also connected to bus 38. Additionally, in Figure 3 In the example shown, for ease of illustration, bus 38 is represented as a single bus, but multiple buses can also be represented. Bus 38 can be a serial bus, or a parallel bus such as a data bus, address bus, and control bus.

[0082] The memory 35 temporarily stores various information and serves as working memory. An example of the memory 35 is RAM (Random Access Memory), but it is not limited to this; other types of storage devices may also be used. Various programs for the camera device 10 are stored in the storage device 36. The CPU 37 reads these programs from the storage device 36 and executes them on the memory 35, thereby controlling the camera device 10 as a whole. Examples of storage devices 36 include flash memory, SSD, EEPROM, or HDD. Furthermore, various non-volatile memories such as magnetoresistive memory or ferroelectric memory may be used instead of or in conjunction with flash memory.

[0083] Imaging element 25 is a CMOS (Complementary Metal Oxide Semiconductor) image sensor. Imaging element 25 captures images of the subject at a predetermined frame rate under the instruction of CPU 37. The "predetermined frame rate" here refers, for example, from tens of frames per second to hundreds of frames per second. Alternatively, imaging element 25 may have a built-in control device (imaging element control device), in which case the imaging element control device performs detailed internal control of imaging element 25 according to the imaging instructions output by CPU 37. Furthermore, imaging element 25 can also capture images of the subject at a predetermined frame rate under the instruction of DSP 31. In this case, the imaging element control device performs detailed internal control of imaging element 25 according to the imaging instructions output by DSP 31. DSP 31 is sometimes also referred to as an ISP (Image Signal Processor).

[0084] The light-receiving surface 25A of the imaging element 25 is formed by a plurality of photosensitive pixels (not shown) arranged in a matrix. In the imaging element 25, each photosensitive pixel is exposed and photoelectric conversion is performed on a per-pixel basis. The charge obtained by photoelectric conversion on a per-pixel basis is an analog imaging signal representing the subject. Here, as a plurality of photosensitive pixels, a plurality of photoelectric conversion elements sensitive to visible light are used (for example, photoelectric conversion elements equipped with color filters). In the imaging element 25, as a plurality of photoelectric conversion elements, photoelectric conversion elements sensitive to R (red) light (e.g., photoelectric conversion elements equipped with R color filters corresponding to R), photoelectric conversion elements sensitive to G (green) light (e.g., photoelectric conversion elements equipped with G color filters corresponding to G), and photoelectric conversion elements sensitive to B (blue) light (e.g., photoelectric conversion elements equipped with B color filters corresponding to B) are used. In the imaging device 10, imaging based on visible light (e.g., light on the short wavelength side of about 700 nanometers or less) is performed by using these photosensitive pixels. However, this embodiment is not limited to this, and imaging based on infrared light (e.g., light with a wavelength longer than approximately 700 nanometers) can also be performed. In this case, multiple photoelectric conversion elements sensitive to infrared light can be used as multiple photosensitive pixels. In particular, for SWIR (Short-wavelength infrared) imaging, for example, InGaAs sensors and / or Type II quantum well (T2SL) sensors can be used.

[0085] Imaging element 25 performs signal processing such as A / D (Analog / Digital) conversion on the analog camera signal to generate a digital camera signal, i.e., a digital image. Imaging element 25 is connected to DSP 31 via bus 38, and outputs the generated digital image to DSP 31 in frame units via bus 38.

[0086] Furthermore, while a CMOS image sensor has been described here as an example of imaging element 25, the technology of the present invention is not limited to this. A CCD (Charge Coupled Device) image sensor can also be used as imaging element 25. In this case, imaging element 25 is connected to bus 38 via an AFE (Analog Front End) without a built-in CCD driver. The AFE generates a digital image by performing signal processing such as A / D conversion on the analog imaging signal obtained from imaging element 25, and outputs the generated digital image to DSP 31. The CCD image sensor is driven by a CCD driver built into the AFE. Of course, a separate CCD driver can also be provided.

[0087] The DSP31 performs various digital signal processing operations on the digital image. These operations include, for example, desacrifice, interference removal, grayscale correction, and color correction. The DSP31 outputs the processed digital image to the image memory 32 frame by frame. The image memory 32 stores the digital images from the DSP31.

[0088] The jitter detection sensor 40, for example, is a device including a gyroscope sensor, that detects the amount of jitter in the camera device 10. In other words, the jitter detection sensor 40 detects jitter in each of a pair of axes. The gyroscope sensor detects jitter around the pitch axis PA, yaw axis YA, and roller axis RA (the axis parallel to the optical axis OA) (see reference). Figure 1 The jitter detection sensor 40 detects the jitter of the camera device 10 by converting the rotational jitter around the pitch axis PA and the rotational jitter around the yaw axis YA detected by the gyroscope sensor into jitter in a two-dimensional plane parallel to the pitch axis PA and the yaw axis YA.

[0089] Here, a gyroscope sensor is used as an example of the jitter detection sensor 40, but this is only one example; the jitter detection sensor 40 can also be an accelerometer. The accelerometer detects the amount of jitter in a two-dimensional plane parallel to the pitch axis PA and yaw axis YA. The jitter detection sensor 40 outputs the detected jitter to the CPU 37.

[0090] Furthermore, while an example of detecting jitter using a physical sensor, jitter detection sensor 40, has been given here, the technology of the present invention is not limited to this. For example, a motion vector obtained by comparing time-series images stored in image memory 32 can also be used as the jitter amount. Moreover, the final jitter amount can be derived from the jitter amount detected by the physical sensor and the motion vector obtained through image processing.

[0091] The CPU 37 acquires the amount of jitter detected by the jitter detection sensor 40, and controls the lens-side jitter correction mechanism 29, the imaging element-side jitter correction mechanism 45, and the electronic jitter correction unit 33 based on the acquired jitter amount. The jitter amount detected by the jitter detection sensor 40 is used for jitter correction based on the respective jitter of the lens-side jitter correction mechanism 29 and the electronic jitter correction unit 33.

[0092] The electronic shake correction unit 33 is a device that includes an ASIC (Application Specific Integrated Circuit). The electronic shake correction unit 33 corrects shake by performing image processing on the captured image in the image memory 32 based on the amount of shake detected by the shake amount detection sensor 40.

[0093] Furthermore, while an ASIC-based device is exemplified here as the electronic jitter correction unit 33, the technology of the present invention is not limited to this. For example, it can be a device including an FPGA (Field Programmable Gate Array) or a PLD (Programmable Logic Device). Moreover, for example, the electronic jitter correction unit 33 can be multiple devices including ASICs, FPGAs, and PLDs. Furthermore, the electronic jitter correction unit 33 can be a computer including a CPU, storage devices, and memory. The CPU can be a single unit or multiple units. Furthermore, the electronic jitter correction unit 33 can be implemented through a combination of hardware and software structures.

[0094] The communication I / F34 is, for example, a network interface, which enables the transmission and control of various information between the camera device 10 and the personal computer 11 via a network. This network is, for example, a WAN or LAN. The communication I / F34 facilitates communication between the camera device 10 and the personal computer 11.

[0095] UI system device 43 includes a receiver 43A and a display 43B. The receiver 43A, for example, is a hard key or a touch panel, which receives various instructions from the user. The CPU 37 acquires the various instructions received by the receiver 43A and performs actions according to the acquired instructions.

[0096] The display 43B displays various information under the control of the CPU 37. The various information displayed on the display 43B may include, for example, the content of various instructions received by the receiving device 43A and camera images.

[0097] <Example of maintenance work for faulty parts> Figure 4 This is a schematic diagram illustrating an example of maintenance work being performed on a power transmission line. For example... Figure 4As shown, for example, a camera (video recording device 10) is sometimes used to inspect the power transmission line 91 connecting tower 81 and tower 82. In this case, for example, a camera capable of telephoto shooting is used to photograph the power transmission line 91 and confirm whether there is a faulty part. Furthermore, when maintaining the faulty part confirmed by the inspection, for example, sometimes workers go to the site, use the camera to photograph the power transmission line 91 in the same way as during the inspection, and perform maintenance work on the faulty part 92. In this case, it is preferable to be able to quickly and accurately determine the location of the faulty part on the power transmission line 91. The present invention can appropriately provide the user with information related to the detection location of the faulty part detected in the photographed object such as the power transmission line 91.

[0098] Specifically, when inspecting transmission line 91, by setting up... Figure 4 The camera at position 10a of the imaging device captures an image of the power transmission line 91. The captured image of the power transmission line 91 is, for example, configured as an image formed by continuously connecting multiple segmented images captured sequentially from end A of the power transmission line 91 connected to tower 81 to end B of the power transmission line 91 connected to tower 82, according to the width (focal length) of the camera's imaging range.

[0099] While photographing the transmission line 91, an inspection of segmented camera images is performed. If a fault location 92 is detected in the transmission line 91, rotation information and camera information related to the fault location 92 are stored. The rotation information consists of the pan and pitch values ​​of the camera associated with the segmented camera image containing the detected fault location 92. The rotation information is the camera's rotation information at the moment the fault location 92 was photographed. The rotation information is an example of the "rotation conditions" in this invention. The camera's pan and pitch values ​​are the same as the pan and pitch values ​​of a rotation device 16, for example, on which the camera is mounted. The camera information consists of the camera's focus information associated with the segmented camera image containing the detected fault location 92. The focus information is information that allows calculation of the distance from the camera to the segmented camera image. The camera information is the camera information of the camera at the moment the fault location 92 was photographed. The camera information is an example of the "camera conditions" in this invention.

[0100] Similarly, the system stores rotation and camera information related to end A of the transmission line 91 connected to tower 81, and rotation and camera information related to end B of the transmission line 91 connected to tower 82. The rotation information for end A consists of the pan and pitch values ​​of a camera associated with a segmented camera image containing end A of the transmission line 91. The camera information for end A consists of the focus information of a camera associated with a segmented camera image containing end A. Similarly, the rotation information for end B consists of the pan and pitch values ​​of a camera associated with a segmented camera image containing end B of the transmission line 91. The camera information for end B consists of the focus information of a camera associated with a segmented camera image containing end B. The rotation information for the fault location 92 and the rotation information for ends A and B of the transmission line 91 can be measured based on the rotation information when the camera's rotation amount is 0 (zero).

[0101] In this invention, based on this rotation information and camera information, the location information of the fault location 92 can be used as the distance from the tower to the fault location 92, for example, the distance D from the end B of the transmission line 91 in the tower 82 to the fault location 92. Figure 4 (Provided to users.)

[0102] <Processing based on CPU 60A of information processing device 60> Figure 5 This is a flowchart illustrating an example of processing based on the CPU 60A of the information processing device 60. At the start of this processing, the user displays, for example, a camera image of a power line undergoing maintenance work on the monitor 13a of the personal computer 11. Figure 4As explained, the image of the transmission line is composed of multiple segmented images obtained by segmenting the transmission line 91 connecting tower 81 and tower 82 according to the width (focal length) of the camera's field of view. The user, in the image of the transmission line displayed on the display 13a, determines the fault location of the transmission line, a first candidate reference position, and a second candidate reference position, which become the reference point of the transmission line. The user can determine the fault location, the first candidate reference position, and the second candidate reference position by selecting a segmented image containing these from the transmission line's image. The selection of the segmented image can be done, for example, by the user touching the fault location, the first candidate reference position, and the second candidate reference position on the display 13a, or by recognizing the segmented image and using AI (Artificial Intelligence) detection. Furthermore, the first candidate reference position and the second candidate reference position can also be selected from the first and last segmented images among the multiple segmented image images. If the fault location, the first reference candidate location, and the second reference candidate location in the transmission line are selected, then the CPU 60A of the information processing device 60 begins... Figure 5 The processing is shown.

[0103] The faulty part is an example of the "first subject" in this invention, for example, it is Figure 4 The fault location 92 is shown. The first reference candidate location is an example of the "second subject" in this invention, for example, a device connected to... Figure 4 The transmission line holding part of the tower 82 at end B of the transmission line 91 shown. The second reference candidate position is an example of the "third subject" in this invention, for example, a position connected to... Figure 4 The transmission line holding part of the tower 81 at end A of the transmission line 91 shown.

[0104] First, the CPU 60A obtains the pan, pitch, and focus information of the camera when capturing the faulty area from the secondary storage device 14 (step S11). The pan, pitch, and focus information are the pan, pitch, and focus information of the camera that are associated with the segmented image containing the faulty area.

[0105] Next, the CPU60A calculates the spatial coordinates of the fault location based on the pan, pitch, and focus information obtained in step S11 (step S12). Spatial coordinates are an example of "location information" in this invention. Spatial coordinates are coordinates representing a location in actual space, such as geographic coordinates composed of latitude, longitude, and altitude. The spatial coordinates of the fault location can be calculated as follows: based on the camera's focus information, the coordinates of the location after moving from the spatial coordinates representing the camera's setting position to the camera direction determined by the pan and pitch values ​​of the camera when the camera's rotation is 0 (zero), and by moving an amount equivalent to the calculated focus distance.

[0106] Next, the CPU 60A retrieves the pan, pitch, and focus information of the camera when capturing the first reference candidate position from the secondary storage device 14 (step S13). The pan, pitch, and focus information are the pan, pitch, and focus information of the camera that is associated with the segmented image containing the first reference candidate position.

[0107] Next, the CPU60A calculates the spatial coordinates of the first reference candidate position based on the roll, pitch, and focus information obtained in step S13 (step S14). The spatial coordinates of the first reference candidate position can also be calculated in the same way as the spatial coordinates of the fault location mentioned above.

[0108] Next, the CPU 60A retrieves the pan, pitch, and focus information from the secondary storage device 14 when capturing the second reference candidate position (step S15). The pan, pitch, and focus information are the pan, pitch, and focus information of the camera that is associated with the segmented image containing the second reference candidate position.

[0109] Next, the CPU60A calculates the spatial coordinates of the second reference candidate position based on the roll, pitch, and focus information obtained in step S15 (step S16). The spatial coordinates of the second reference candidate position can also be calculated in the same way as the spatial coordinates of the fault location mentioned above.

[0110] Next, CPU60A compares the distance from the camera (image capture device) to the first reference candidate position and the distance from the camera to the second reference candidate position, and determines whether the first reference candidate position is closer to the camera than the second reference candidate position (step S17). The determination of the distance from the camera to the first reference candidate position and the distance to the second reference candidate position can be made based on the focus information of the first reference candidate position obtained in step S13 and the focus information of the second reference candidate position obtained in step S15, or based on the spatial coordinates of the first reference candidate position calculated in step S14 and the spatial coordinates of the second reference candidate position calculated in step S16.

[0111] If the first reference candidate position is closer to the camera than the second reference candidate position (step S17: Yes), the CPU 60A calculates the distance between the first reference candidate position and the fault location (step S18). The distance between the first reference candidate position and the fault location is calculated based on the spatial coordinates of the first reference candidate position calculated in step S14 and the spatial coordinates of the fault location calculated in step S12.

[0112] If the second reference candidate position is closer to the camera than the first reference candidate position (step S17: No), the CPU 60A calculates the distance between the second reference candidate position and the fault location (step S19). The distance between the second reference candidate position and the fault location is calculated based on the spatial coordinates of the second reference candidate position calculated in step S16 and the spatial coordinates of the fault location calculated in step S12.

[0113] Next, CPU 60A outputs the distance between the reference candidate location and the fault location calculated in step S18 or S19 to display 13a (step S20). Regarding the output on display 13a, [the following will be discussed / discussed]. Figure 6 and Figure 7 The following will be discussed in detail.

[0114] Furthermore, the above example describes a scenario where the transmission line is pre-segmented and pan, pitch, and focus information are stored in a corresponding relationship with each segmented camera image, but this is not a limitation. For example, a user can control the camera device 10 via a personal computer 11 at the maintenance site of the transmission line 91 to capture images of the fault location, the first reference candidate location, and the second reference candidate location, and store the pan, pitch, and focus information at that time.

[0115] <Image of PC 11 display> Figure 6This diagram illustrates an example of distance information displayed on the monitor 13a of the personal computer 11. The personal computer 11 displays distance information, calculated by the CPU 60A of the information processing device 60, such as distance between a reference candidate location near the fault location and the fault location, on the monitor 13a.

[0116] For example, such as Figure 6 As shown, the personal computer 11 displays distance information 100 on the display 13a, indicating the distance between the transmission line holding part 82a of the tower 82 connected to the end B of the transmission line 91 and the fault location 92 of the transmission line 91. In this example, the transmission line holding part 82a of the tower 82 connected to the end B of the transmission line 91 is a reference candidate location close to the fault location 92. The distance information can be the straight-line distance between the two, or it can be information obtained by decomposing the distance between the two into horizontal and vertical directions. Furthermore, the personal computer 11 displays a segmented camera image 101a including the fault location 92 of the transmission line 91 and a segmented camera image 101e including the transmission line holding part 82a of the tower 82 connected to the end B of the transmission line 91 on the display 13a. In this example, the distance information 100 indicates that the straight-line distance between the transmission line holding part 82a of the tower 82 connected to the end B of the transmission line 91 and the fault location 92 of the transmission line 91 is 15m.

[0117] Figure 7 This is a diagram illustrating a modified example of distance information displayed on the monitor 13a of the personal computer 11. Figure 6 In the example shown, only the segmented camera image 101a, including the faulty section 92 of the transmission line 91, and the segmented camera image 101e, including the transmission line holding section 82a of the tower 82 connected to the end B of the transmission line 91, are displayed on the display 13a. However, in this modified example, the image of the transmission line 91 existing between the transmission line 91 in segmented camera image 101a and the transmission line 91 in segmented camera image 101e is also displayed on the display 13a. Specifically, as... Figure 7 As shown, segmented camera images 101b, 101c, and 101d, obtained by capturing a segmented camera image 101a of the transmission line 91 existing between the transmission line 91, which includes a segmented camera image 101a of the transmission line 92 and a segmented camera image 101e of the transmission line holding part 82a of the tower 82 to which the transmission line 91 is connected, are displayed on the display 13a.

[0118] As described above, the information processing device 60 of this embodiment can output distance information between the reference candidate position and the faulty part 92 based on the position information obtained from the rotation conditions (tilt / slide values) of the rotation device 16 for the faulty part 92 and the imaging conditions (focus information) of the imaging device 10, and the position information obtained from the rotation conditions of the rotation device 16 for the reference candidate position (the transmission line holding part of the tower 81, 82) and the imaging conditions of the imaging device 10. Therefore, it is easy to determine the location of the faulty part 92 of the transmission line 91 in the actual space, and maintenance work on the faulty part 92 of the transmission line 91 can be carried out smoothly.

[0119] <Error when calculating distance based on focus information> Figure 8 This diagram illustrates the errors that occur when calculating the distance from the camera device to the tower based on focus information. Generally, the greater the distance to the subject, the lower the accuracy of the calculated distance.

[0120] For example, such as Figure 8 As shown, when the distance from the camera device 10 (camera) to the tower 81 is greater than the distance from the camera device 10 to the tower 82, the accuracy of calculating the distance between the camera device 10 and the tower up to the tower 81 (which is farther away) is lower than the accuracy of calculating the distance up to the tower 82 (which is closer away) based on the focus information. Specifically, the correct distance from the camera device 10 to end A of the power transmission line 91 connected to the tower 81 is distance D1. However, if the distance to end A of the power transmission line 91 is calculated based on the focus information obtained from the camera device 10, it may be calculated as a distance D2 that is longer than distance D1. In this case, the position of the end of the power transmission line 91 calculated based on distance D2 from the focus information is set to the position of end A2, which is farther than end A1.

[0121] Figure 9 This is a flowchart illustrating a first modified example of the processing based on the CPU 60A of the information processing device 60. The first modified example considers the case where the accuracy of the distance calculated based on the focus information of a subject far from the camera device 10 is low. Instead of using the focus information, it calculates the position information of the subject far from the camera device 10 and outputs the distance information from the fault location of the transmission line 91 to the reference position. Furthermore, the user's operations at the start of this modified processing are similar to... Figure 5 The operation content described in the text is the same.

[0122] First, the CPU 60A sets the reference candidate position closer to the camera (camera device 10) among the first and second reference candidate positions as the near-distance reference position, and sets the reference candidate position farther away from the camera as the far-distance reference position (step S31). The CPU 60A can determine which of the first and second reference candidate positions is closer to the camera based on the camera's focus information when shooting the first reference candidate position and the camera's focus information when shooting the second reference candidate position, or it can be specified by the user.

[0123] Next, CPU 60A retrieves the pan, pitch, and focus information from secondary storage device 14 when capturing the close-up reference position (step S32). The pan, pitch, and focus information are the pan, pitch, and focus information of a camera that is associated with the segmented image including the close-up reference position. For example, the close-up reference position is maintained... Figure 8 The position of the transmission line holding part at end B of the transmission line 91 in the tower 82 shown.

[0124] Next, CPU60A calculates the spatial coordinates of the near-field reference position based on the pan, pitch, and focus information obtained in step S32 (step S33). The spatial coordinates of the near-field reference position can be compared with the above... Figure 5 The spatial coordinates of the fault location described in step S12 are calculated in the same way.

[0125] Next, the CPU 60A retrieves the pan and pitch values ​​from the secondary storage device 14 when capturing the distant reference position (step S34). The pan and pitch values ​​are the pan and pitch values ​​of the camera that are associated with the segmented image including the distant reference position. For example, the distant reference position is maintained... Figure 8 The position of the transmission line holding part at end A of the transmission line 91 in the tower 81 shown.

[0126] Next, CPU60A calculates the spatial coordinates of the distant reference position based on the roll and pitch values ​​related to the distant reference position obtained in step S34 and the roll, pitch, and focus information related to the near reference position obtained in step S32 (step S35). The calculation of the spatial coordinates of the distant reference position uses... Figure 10 This will be explained later.

[0127] Next, CPU60A calculates the vertical plane including the near-distance reference position calculated in step S33 and the far-distance reference position calculated in step S35 (step S36). A fault location exists on the vertical plane including the near-distance and far-distance reference positions, which is a location on the transmission line 91. The vertical plane is a common plane containing the near-distance reference position, the far-distance reference position, and the fault location. The vertical plane is a common plane with permissible error.

[0128] Next, CPU 60A retrieves the pan and pitch values ​​from secondary storage device 14 when the faulty area was captured (step S37). The pan and pitch values ​​are the pan and pitch values ​​of the camera that are associated with the segmented image including the faulty area. For example, the faulty area is... Figure 8 The fault location shown is 92.

[0129] Next, the CPU60A calculates the spatial coordinates of the fault location based on the roll and pitch values ​​obtained in step S37 and the vertical plane calculated in step S36 (step S38). The spatial coordinates of the fault location can be calculated by finding the intersection point of the straight line obtained from the roll and pitch values ​​in step S37 and the vertical plane calculated in step S36.

[0130] Next, CPU60A calculates the distance between the near-distance reference position and the faulty part (step S39). The distance between the near-distance reference position and the faulty part is calculated based on the spatial coordinates of the near-distance reference position calculated in step S33 and the spatial coordinates of the faulty part calculated in step S38.

[0131] Next, the CPU 60A outputs the distance between the near-distance reference position calculated in step S39 and the fault location to the display 13a (step S40).

[0132] Figure 10 This is an explanation Figure 9 A diagram showing the calculation of the spatial coordinates of the distant reference position in the first variation. (See diagram.) Figure 10 As shown, a power transmission line 91 is connected between tower 81 and tower 82. End A of the power transmission line 91 is held by the power transmission line holding part of tower 81, and end B of the power transmission line 91 is held by the power transmission line holding part of tower 82. The power transmission line holding part of tower 81, where end A of the power transmission line 91 is connected, is a distant reference position, and the power transmission line holding part of tower 82, where end B of the power transmission line 91 is connected, is a near-distance reference position.

[0133] For example, suppose that the height hA of the distant reference position is the same as the height hB of the near reference position (hA=hB=h). In this case, the distance (distA) from the camera device 10 to the distant reference position can be calculated by the following formula.

[0134] hA=hB h = distB × sin(tiltB) distA = h / sin(tiltA) =distB×sin(tiltB) / sin(tiltA) Furthermore, for example, assume that the height hA of the distant reference position and the height hB of the near reference position are a predetermined height ratio (hA = hB × r). In this case, the distance (distA) from the camera device 10 to the distant reference position can be calculated by the following formula.

[0135] hA=hB×r hB = distB × sin(tiltB) distA = hA / sin(tiltA) =distB×sin(tiltB)×r / sin(tiltA) Thus, the distance (distA) from the camera device 10 to the distant reference position can be calculated without using the focus information of the distant reference position. Consequently, the spatial coordinates of the distant reference position can be calculated as the distance (distA) by moving the distant reference position from the spatial coordinates of the camera device 10 in a camera direction determined by pan and tilt values ​​that are correlated with the segmented camera image including the distant reference position. Furthermore, information indicating the height relationship between the distant and near reference positions (same height, height ratio) can be specified, for example, through user input.

[0136] Furthermore, in Figure 10 In the diagram, the vertical plane 110, shown by the oblique lines at ends A and B of the transmission line 91, is in... Figure 9 The vertical plane calculated in step S36. A near-distance reference position, a far-distance reference position, and a fault location 92 of the transmission line 91 exist on the vertical plane 110. Therefore, the spatial coordinates of the fault location 92 can be calculated by intersecting the vertical plane 110 with a straight line extending from the spatial coordinates of the imaging device 10 into an imaging direction determined by pan and pitch values ​​corresponding to the segmented imaging image including the fault location 92. Then, the distance from the near-distance reference position to the fault location 92 of the transmission line 91 can be calculated based on the calculated spatial coordinates.

[0137] As described above, according to the first modification of the CPU60A-based processing, the position information of the distant reference position can be obtained based on the rotation conditions (pan and pitch values) of the rotation device 16 relative to the near reference position, the imaging conditions (focus information) of the imaging device 10, and the rotation conditions of the rotation device 16 relative to the distant reference position. Therefore, even when the distance from the imaging device 10 to the distant reference position is large and the accuracy of the distance information based on the imaging conditions of the distant reference position is low, the distance information of the distant reference position can be obtained with high accuracy even without using the imaging conditions of the distant reference position. Furthermore, the position information of the faulty part 92 can be obtained based on the position information of the near reference position, the position information of the distant reference position, and the rotation conditions of the rotation device 16 relative to the faulty part 92. Therefore, even when the distance from the imaging device 10 to the faulty part 92 is large and the accuracy of the distance information based on the imaging conditions of the faulty part 92 is low, the distance information of the faulty part 92 can be obtained with high accuracy even without using the imaging conditions of the faulty part 92. Therefore, the distance from the fault location 92 to the nearest reference position can be calculated, making it easy to determine the actual location of the fault location 92 in the transmission line 91. This allows for smooth maintenance of the fault location 92.

[0138] Figure 11 This is a flowchart illustrating a second variation of the processing based on the CPU 60A of the information processing device 60. Similar to the first variation, the second variation calculates the position information of a distant subject without using focus information of that subject that is far from the imaging device 10 (camera). However, unlike the first variation, it uses the distance between a nearby subject and the distant subject to calculate the position information of the distant subject.

[0139] like Figure 11 As shown, in the second variation, the processing of steps S31 to S34 is the same as... Figure 9 The processes in steps S31 to S34 of the first variant described herein are the same.

[0140] Next, the CPU 60A obtains the distance L between tower 81, which is connected to end A of the power transmission line 91, and tower 82, which is connected to end B of the power transmission line 91 (step S35A). The distance between tower 81 and tower 82 is, for example, a distance specified by the user operating the personal computer 11.

[0141] Next, CPU60A calculates the spatial coordinates of the distant reference position based on the distance L between towers 81 and 82 obtained in step S35A and the roll and pitch values ​​related to the distant reference position obtained in step S34 (step S35B). The calculation of the spatial coordinates of the distant reference position uses... Figure 12 This will be explained later.

[0142] The processing of steps S36 to S40 is as follows: Figure 9 The processes in steps S36 to S40 of the first variant described herein are the same.

[0143] Figure 12 This is an explanation Figure 11 A diagram showing the calculation of the spatial coordinates of the distant reference position in the second variation. Figure 12 In the image, from above, the end A of the transmission line 91 held by the transmission line holding part of tower 81, the end B of the transmission line 91 held by the transmission line holding part of tower 82, and the camera device position 10a where the camera device 10 (camera) is installed are viewed. On a horizontal plane represented by the x-axis and y-axis, the intersection of the x-axis and y-axis is defined as the camera device position 10a, and the y-axis direction is defined as the reference camera direction determined by the panning value when the camera's rotation is 0 (zero). Furthermore, the distance between end A of the transmission line 91 connected to tower 81 and end B of the transmission line 91 connected to tower 82 is defined as the distance L between the towers. Additionally, end B of the transmission line 91 is... Figure 11 The near-distance reference position is the end A of the transmission line 91. Figure 11 The long-distance reference position in the middle.

[0144] The distance from the camera position 10a to the end B of the transmission line 91 on the xy horizontal plane can be expressed as distB × cos(tiltB) (refer to...). Figure 10 Therefore, the position of end B of transmission line 91 is represented as (distB×cos(tiltB)×sin(PanB), distB×cos(tiltB)×cos(PanB)).

[0145] Therefore, the position of end A of the transmission line 91 on the xy horizontal plane can be determined by drawing a circle 120 with a radius equal to the distance L centered on end B of the transmission line 91 and calculating the intersection of this circle 120 with the imaging direction of end A of the transmission line 91 determined by PanA. Thus, the distance from the camera device position 10a to end A of the transmission line 91 can be calculated.

[0146] Thus, by using the distance L between the towers, the distance from the camera device 10 to the distant reference position can be calculated without using the focusing information of the distant reference position. Therefore, the spatial coordinates of the distant reference position can be calculated as the coordinates after moving the position from the end A of the transmission line 91 on the xy horizontal plane to a height hA in the vertical direction relative to the distant reference position.

[0147] As described above, according to the second modification of the CPU 60A-based processing, the position information of the distant reference position can be obtained based on the position information obtained relative to the near reference position based on the rotation conditions (pan and pitch values) of the rotation device 16 and the imaging conditions (focus information) of the imaging device 10, the distance L between the near and distant reference positions, and the rotation conditions of the rotation device 16 relative to the distant reference position. Therefore, similar to the first modification described above, even if the distance from the imaging device 10 to the distant reference position is large, the distance information of the distant reference position can be obtained with high accuracy. Furthermore, even if the distance from the imaging device 10 to the fault location 92 is large, the distance information of the fault location 92 can be obtained with high accuracy. As a result, it is easy to determine the location of the fault location 92 of the transmission line 91 in actual space, thereby enabling smooth maintenance work on the fault location 92.

[0148] Furthermore, while the above description addresses the case where the fault location is on the power transmission line, it is not a limitation. The fault location can, for example, be on the wall of a building. That is, the invention can be applied even when inspecting and maintaining the walls of a building.

[0149] <Storage media for information processing programs> In the above-described camera control, examples have been given in which the information processing program of each embodiment is stored in the storage device 60B of the information processing device 60 and the CPU 60A of the information processing device 60 executes the information processing program in the memory 60C, but the technology of the present invention is not limited thereto.

[0150] Figure 13 This diagram illustrates an example of a method of installing an information processing program from a storage medium containing an information processing program for camera control onto an information processing device 60 of a personal computer 11. For example... Figure 13 As shown, as an example, the information processing program 221 can be stored in a non-temporary storage medium, namely storage medium 220. Figure 13 In the example shown, the information processing program 221 stored in the storage medium 220 is installed in the information processing device 60, and the CPU 60A executes the above-described processes according to the information processing program 221.

[0151] The various embodiments have been described above, but the present invention is not limited to this example. Obviously, those skilled in the art will be able to conceive of various modifications or alterations within the scope described in the technical solution, and it should be understood that these modifications or alterations naturally fall within the technical scope of the present invention. Furthermore, the constituent elements of the above embodiments can be combined arbitrarily without departing from the spirit of the invention.

[0152] Furthermore, this application is based on Japanese patent application filed on October 30, 2023 (Japanese Patent Application No. 2023-185446), the contents of which are incorporated herein by reference.

[0153] Symbol Explanation 1-Camera system, 10-Camera device, 10a-Camera device location, 11-Personal computer, 12-Communication line, 13a, 43B-Display, 13b-Keyboard, 13c-Mouse, 14-Secondary storage device, 15-Optical system, 15B-Lens group, 15B1-Anti-vibration lens, 15B2-Zoom lens, 16-Rotation device, 17, 21-Lens actuator, 19-Computer, 22-BIS driver, 23-OIS driver, 25-Imaging element, 25A-Light receiving surface, 27-Imaging element actuator, 28-Lens driver, 29, 45-Correction mechanism, 31-DSP, 32-Image memory, 33-Correction unit, 34, 66, 68, 78-Communication I / F, 35, 60C-Storage Storage device, 36, 60B - storage device, 37, 60A - CPU, 38, 69 - bus, 39, 47 - position sensor, 40 - quantity detection sensor, 43 - UI system device, 43A, 62 - receiving device, 60 - information processing device, 71 - yaw axis rotation mechanism, 72 - pitch axis rotation mechanism, 73, 74 - motor, 75, 76 - driver, 81, 82 - tower, 82a - transmission line holding part, 91 - transmission line, 92 - fault location, 100 - distance information, 101a, 101b, 101c, 101d, 101e - segmented camera image, 110 - vertical plane, 120 - circle, 220 - storage medium, 221 - information processing program, D1, D2 - distance, A1, A2 - end.

Claims

1. An information processing apparatus comprising a processor capable of communicating with a camera device and a rotating device for rotating the camera device, wherein the information processing apparatus, The processor performs the following processing: The first position information is obtained based on the rotation conditions of the rotating device corresponding to the first subject; The second position information is obtained based on the rotation conditions of the rotating device corresponding to the second subject; The third position information is obtained based on the rotation conditions of the rotating device corresponding to the third subject; Based on the first location information, the second location information, and the third location information, output the distance information between the first subject and either the second or third subject.

2. The information processing apparatus according to claim 1, wherein, The distance information refers to the distance between the first subject and the second subject, which is closer to the camera device than the third subject.

3. The information processing apparatus according to claim 1, wherein, The first, second, and third subjects are located on the first surface.

4. The information processing apparatus according to claim 1, wherein, The processor performs the following processing: The first position information is obtained based on the rotation conditions of the rotating device corresponding to the first subject and the imaging conditions of the imaging device. The second position information is obtained based on the rotation conditions of the rotating device corresponding to the second subject and the imaging conditions of the imaging device. The third position information is obtained based on the rotation conditions of the rotating device corresponding to the third subject and the imaging conditions of the imaging device.

5. The information processing apparatus according to claim 1, wherein, The second subject is the subject that is closer to the camera device than the third subject. The processor performs the following processing: The second position information is obtained based on the rotation conditions of the rotating device corresponding to the second subject and the imaging conditions of the imaging device. The third position information is obtained based on the rotation conditions of the rotating device corresponding to the second subject, the imaging conditions of the imaging device, and the rotation conditions of the rotating device corresponding to the third subject.

6. The information processing apparatus according to claim 1, wherein, The second subject is the subject that is closer to the camera device than the third subject. The processor performs the following processing: The second position information is obtained based on the rotation conditions of the rotating device corresponding to the second subject and the imaging conditions of the imaging device. The third position information is obtained based on the second position information, the distance between the second subject and the third subject, and the rotation conditions of the rotating device corresponding to the third subject.

7. The information processing apparatus according to claim 5, wherein, The first, second, and third subjects are located on the first surface. The processor performs the following processing: The first position information is obtained based on the second position information, the third position information, and the rotation conditions of the rotating device corresponding to the first subject.

8. The information processing apparatus according to claim 5, wherein, The processor performs the following processing: The second subject is determined to be closer to the camera device than the third subject based on the camera conditions of the camera device corresponding to the second subject and the camera conditions of the camera device corresponding to the third subject.

9. The information processing apparatus according to claim 5, wherein, The processor performs the following processing: obtaining the third position information based on information representing the relative height relationship between the second subject and the third subject.

10. The information processing apparatus according to claim 1, wherein, The rotation conditions of the slewing device include at least one of the slewing device's panning state and pitch state.

11. The information processing apparatus according to claim 1, wherein, The camera conditions of the camera device include the focus information of the camera device.

12. A control device that controls the camera device and / or the rotary device based on information generated by the information processing device according to any one of claims 1 to 11.

13. An information processing method, executed by an information processing device having a processor, the processor being capable of communicating with a camera device and a rotating device for rotating the camera device, wherein in the information processing method, The processor performs the following processing: The first position information is obtained based on the rotation conditions of the rotating device corresponding to the first subject; The second position information is obtained based on the rotation conditions of the rotating device corresponding to the second subject; The third position information is obtained based on the rotation conditions of the rotating device corresponding to the third subject; Based on the first location information, the second location information, and the third location information, output the distance information between the first subject and either the second or third subject.

14. An information processing program executed by an information processing device having a processor, the processor being capable of communicating with a camera device and a rotating device for rotating the camera device, the information processing program causing the processor to perform the following processing: The first position information is obtained based on the rotation conditions of the rotating device corresponding to the first subject; The second position information is obtained based on the rotation conditions of the rotating device corresponding to the second subject; The third position information is obtained based on the rotation conditions of the rotating device corresponding to the third subject; Based on the first location information, the second location information, and the third location information, output the distance information between the first subject and either the second or third subject.

Images