Imaging apparatus, method for controlling imaging apparatus, and program

Abstract

To provide an imaging apparatus capable of generating an image capable of grasping a moving body, even when the moving body having a higher speed is detected.SOLUTION: The imaging apparatus includes: acquisition means for acquiring a first image and a second image in the same imaging range as the first image but temporally different therefrom; comparison means for comparing a speed in a moving body detected from the second image with a predetermined threshold value; generation means for generating a third image on the basis of the first image and an area corresponding to the moving body in the second image; and distribution means for distributing the third image to an external device according to a comparison result by the comparison means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an imaging device, a control method for an imaging device, and a program. [Background technology]

[0002] Some ships are equipped with imaging devices such as surveillance cameras. These are used for a variety of purposes, including watchkeeping, obtaining evidence in the event of an accident, and searching for violating vessels, suspicious vessels, and people in distress. Some ships are even able to streamline operations by transmitting the images they obtain to shore in real time and sharing the information with shore-based staff, thereby reducing the number of personnel on board and improving operational efficiency.

[0003] Ships sailing on the open seas sometimes use satellite communications as a means of communication between ship and shore, and information must be exchanged within limited communication speeds. Conventionally, technologies aimed at improving information exchange between ship and shore in such situations where communication speeds are limited have been disclosed. For example, Patent Document 1 discloses a technology in which color video data obtained from an imaging device is sent separately as compressed color still image data and black-and-white video data. The receiving side then restores the black-and-white video data as color video data based on color information obtained from the color still image data. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2001-37979 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the conventional technology disclosed in Patent Document 1 attempts to reduce the amount of communication data by converting color video data into black and white video data, but it is still unable to obtain a sufficient frame rate in an environment with limited communication speed. Therefore, for example, if a fast moving object (subject) enters the imaging range, there is a risk that important scenes of the moving object will not be delivered or that the moving object will be lost.

[0006] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide an imaging device that can generate an image that makes it possible to grasp a moving object even when the moving object is detected moving at a high speed. [Means for solving the problem]

[0007] In order to achieve the above object, an imaging device according to one aspect of the present invention is characterized by having an acquisition means for acquiring a first image and a second image having the same imaging range as the first image but different in time, a comparison means for comparing the speed of a moving object detected from the second image with a predetermined threshold, a generation means for generating a third image based on the first image and an area in the second image corresponding to the moving object, and a distribution means for distributing the third image to an external device according to the comparison result of the comparison means. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide an imaging device that makes it possible to grasp the situation without losing sight of a fast moving object even when the fast moving object is detected. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram of an imaging system according to an embodiment. [Figure 2] 1 is a hardware configuration diagram of an imaging device 100 according to an embodiment. [Figure 3] 3A and 3B are diagrams illustrating an example of an installation state of an imaging device according to an embodiment and output image data. [Figure 4]10A and 10B are diagrams illustrating an example of image data output by the imaging device according to the embodiment and image data for distribution when image synthesis processing is not performed. [Figure 5] 4 is a flowchart illustrating processing of the imaging device according to the embodiment. [Figure 6] 10 is a flowchart illustrating another process of the imaging apparatus according to the present embodiment. [Figure 7] FIG. 7 is a diagram supplementing the processing of the flowchart shown in FIG. 6. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. The embodiments described below are examples of means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions, and the present invention is not limited to the following embodiments.

[0011] <Embodiment 1> The configuration of an imaging system including an imaging device 100 and an external device 111 according to this embodiment will be described with reference to Fig. 1 and Fig. 2. Fig. 1 is a block diagram of the imaging system according to this embodiment. Fig. 2 is a hardware configuration diagram of the imaging device 100 according to this embodiment.

[0012] The imaging device 100 and the external device 111 are connected to each other so that they can communicate with each other via a network 110. In the imaging system of this embodiment, image synthesis processing is performed by the imaging device 100 installed on a ship. In this embodiment, it is assumed that the imaging device 100 is installed on the ship and the external device 111 is installed on land.

[0013] The imaging device 100 of this embodiment includes an imaging unit 101, a storage unit 102, an image processing unit 103, a communication unit 104, an image analysis unit 105, and a control unit .

[0014] The imaging unit 101 functions as an imaging means composed of an imaging optical system including optical elements such as multiple lenses and holding members, and an imaging element. The imaging optical system may be configured to be able to adjust the zoom magnification and focus position using a lens drive motor or the like, and may further be configured to be able to insert and remove a filter that transmits or attenuates specific wavelengths. The imaging element has semiconductor elements such as a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor. Light that passes through the imaging optical system is imaged by the imaging element and converted into an electrical signal. Furthermore, the imaging element is sensitive to light in the visible light range, but is not limited thereto, and may be sensitive to non-visible light ranges such as infrared light, for example.

[0015] The storage unit 102 functions as a storage means capable of temporarily saving (storing) and reading out one or more pieces of image data that have been subjected to predetermined image processing by the image processing unit 103. Furthermore, the storage unit 102 is also used as a storage area for programs executed by the control unit 106, a storage area for various parameters, and a work area during program execution.

[0016] The image processing unit 103 generates image data by performing various image processing such as development processing, color balance processing, gamma processing, and noise reduction processing on the image data output by the imaging unit 101. Furthermore, it is also possible to perform a cropping process to extract a portion of the image data stored in the storage unit 102, and a compositing process to overwrite the cropped image onto another image.

[0017] The communication unit 104 converts the image data stored in the storage unit 102 into data that conforms to a communication protocol, and then distributes the data to the external device 111. However, the communication unit 104 is not limited to sending still image data, and may generate video data from the stored image data, perform compression encoding processing such as H.264, and distribute the video data.

[0018] The image analysis unit 105 detects the speed or vector of a moving object in the image data. The detection method involves first extracting the moving object by taking the difference between multiple pieces of image data, and then calculating the speed or vector from the coordinate difference of the moving object's feature points and the acquisition time difference of the compared image data. Note that the method for calculating the speed of a moving object within the angle of view is not limited to this. For example, the speed may be calculated using a stereo camera, or a mechanism may be used in which speed information calculated by an external device such as a radar is received and the received speed information is linked to the moving object (subject) within the angle of view.

[0019] The control unit 106 is configured as at least one computer including a CPU, memory, etc., and is connected to each component of the exposure apparatus EX via a line. The control unit 106 also functions as a system control unit that comprehensively controls each component of the imaging apparatus 100 in accordance with a program stored in the memory, and sets various parameters and issues instructions for data transmission and reception. The control unit 106 may be configured integrally with the other components of the imaging apparatus 100 (in a common housing), or may be configured separately from the other components of the imaging apparatus 100 (in a different housing), or may be installed in a location separate from the imaging apparatus 100 and controlled remotely.

[0020] The network 110 is a WAN (Wide Area Network) and is made up of routers, switches, cables, and even artificial satellites that comply with communication standards such as Ethernet (registered trademark). The image capture device 100 can be connected to an external device 111 via the network 110. Note that the network 110 can be configured to communicate between the image capture device 100 and the external device 111 regardless of its communication standard, scale, or configuration, and may be configured, for example, via a cloud.

[0021] The external device 111 is a client device consisting of a PC, a mobile terminal, etc., and is connected to the imaging device 100 via the network 110 in a state where they can communicate with each other, and can receive, display, and record image data sent from the imaging device 100.

[0022] The imaging device 100 is configured to include a CPU 10 , a ROM 11 , a RAM 12 , an imaging unit 13 , a storage device 14 , and a communication unit 15 .

[0023] The CPU (processor) 10 is a central processing unit that reads out control programs stored in a ROM (read-only memory) 11 and executes various processes. The ROM 11 is a non-volatile memory that stores the programs of the respective embodiments, programs (control programs) required for other controls, and various data. The ROM 11 constitutes a storage unit 102, and therefore a detailed description thereof will be omitted.

[0024] The RAM (Random Access Memory) 12 is a volatile memory and is used as a temporary storage area such as the main memory or work area of ​​the CPU 10. The RAM 12 constitutes the storage unit 102, and therefore a detailed description thereof will be omitted. The imaging unit 13 is an imaging means that constitutes the imaging unit 101 in FIG. 1, and therefore a detailed description thereof will be omitted.

[0025] The storage device 14 stores various data, various programs, etc. The storage device 14 is, for example, a non-volatile storage device such as an HDD, flash memory, or SD card. The storage device 14 is used as a permanent storage area for the OS, various programs, various data, etc., as well as a short-term storage area for various data, etc. The storage device 14 constitutes the storage unit 102 in FIG. 1, and therefore a detailed description thereof will be omitted. The communication unit 15 performs communication processing with an external device 111, such as a client device or a server, via a wired or wireless network. The communication unit 15 constitutes the storage unit 102 in FIG. 1, and therefore a detailed description thereof will be omitted.

[0026] The functions and processes of the imaging device 100 are realized by the CPU 10 reading and executing a program stored in the ROM 11 or the storage device 14. As another example, the CPU 10 may read a program stored in a recording medium such as an SD card instead of the ROM 11.

[0027] In this embodiment, the imaging device 100 is configured such that one processor (CPU 10) uses one memory (ROM 11) to execute each process shown in the flowcharts described below, but other configurations are also possible. For example, multiple processors, multiple RAMs 12, ROMs 11, and storages (storage devices 14) can cooperate to execute each process shown in the flowcharts described below. Also, some of the processes may be executed using hardware circuits. Furthermore, functions and processes of the imaging device 100 described below may be realized using a processor other than a CPU. Furthermore, for example, a GPU (Graphics Processing Unit) may be used instead of a CPU.

[0028] Although not shown, the external device 111 is configured to have a CPU, ROM, RAM, and a communication unit. These can be similar to the CPU 10, ROM 11, RAM 12, imaging unit 13, and communication unit 15 of the imaging device 100, and therefore a description thereof will be omitted. The external device 111 may also have a display unit such as a monitor or display, and input devices such as a mouse or keyboard.

[0029] 3A and 3B are diagrams showing an example of the installation state of the imaging device 100 according to this embodiment and output image data. Fig. 3A is a diagram simply showing a ship 200 on which the imaging device 100 is installed and its surroundings. Fig. 3B is a diagram showing image data 207, which is an example of image data obtained by performing predetermined image processing on imaging data.

[0030] The imaging device 100 is installed at the tip of a mounting base 201 provided on a ship 200 at an installation height h from the sea surface 204, with a center of angle of view 203 at an installation angle θ with respect to the vertical axis of the sea surface 204. Furthermore, within the imaging range 202 of the imaging device 100, which has an angle of view Φ, there are a first ship 205 and a second ship 206, which are subjects (moving bodies). Hereinafter, moving subjects such as the first ship 205 and the second ship 206 shown in FIG. 3 will be referred to as moving bodies. In this embodiment, the distance between the imaging device 100 and the first ship 205 is defined as a1, and the distance between the imaging device 100 and the second ship 206 is defined as a2.

[0031] In this embodiment, the distances a1 and a2 are calculated based on the height h, the installation angle θ, the angle of view Φ, and the subject coordinates in the image data, but are not limited to this. For example, the distances may be measured using a distance measuring sensor or radar (not shown) provided on the imaging device 100 or the ship 200. Here, as an example, the distance a1 is 100 m and the distance a2 is 1000 m.

[0032] The image data 207 shown in Figure 3(B) includes images of the bow of the ship 200, the first ship 205, and the second ship 206. Here, the number of horizontal pixels of the image data 207 is indicated by X. In this embodiment, it is set to 2000, but this is just an example and is not limiting. The number of horizontal pixels of the first ship 205 is set to X1, and the number of horizontal pixels of the second ship 206 is set to X2. In this embodiment, X1 and X2 are both set to 200, for example.

[0033] Fig. 4 is a diagram for explaining an example of image data output by the imaging device 100 in this embodiment and image data for distribution to be distributed to the external device 111. Note that in this embodiment, a composite image generated by the control unit 106 performing the processes shown in the flowcharts of Figs. 5 and 6, which will be described later, is used as image data for distribution, but Fig. 4 shows a case where a composite image such as that generated in this embodiment is not used.

[0034] The imaging device 100 of this embodiment is capable of acquiring image data at a frame rate of 30 fps. A display device (not shown) arranged inside the ship 200 is capable of observing the image data 208 acquired at a frame rate of 30 fps in real time. When image data for distribution is distributed to an external device 111 installed on land via satellite communication or the like, the communication speed is approximately several hundred kbps, so not all image data is distributed, but one image data for distribution is distributed per second. Note that the frame rate of the imaging device 100 and the distribution time interval to the external device 111 are not limited to the values ​​described above.

[0035] Here, to simplify the diagram, the image data 208 is displayed in groups of five overlapping frames, clearly showing image data from times T0, T0.2, T0.4, T0.6, and T0.8. The image data 208 at each time point makes it possible to capture the intrusion position and path of the first vessel 205. However, in the image data for distribution 209, the first vessel 205 is not displayed in the image data for distribution 209a and the image data for distribution 209c, and only the image data for distribution 209b is displayed. Therefore, it may be difficult to determine where the first vessel 205 entered and where it left off.

[0036] In this way, if the speed (moving speed) of the moving object is fast compared to the time interval (distribution rate Ts) of the image data to be distributed, the path of the moving object may not be clear from the image data to be distributed, or the moving object itself may not be displayed. In such cases, there is a risk that the moving object may not be recognizable by users (staff) on land.

[0037] In this embodiment, the control unit 106 performs the processes shown in FIGS. 5 and 6, which will be described later, to generate image data for distribution of fast-moving objects so that the route of the object can be determined.

[0038] Fig. 5 is a flowchart showing the processing of the imaging device 100 according to this embodiment. Hereinafter, processing related to the detection of a moving object present in image data will be described with reference to Fig. 5. Note that each operation (process) shown in the flowchart of Fig. 5 is realized by the control unit 106 of the imaging device 100 executing a program stored in the ROM 11 or the like. Furthermore, by adding an S to the beginning of each process (step), the notation of the process (step) is omitted.

[0039] In S501, the control unit 106 generates image data. Specifically, the control unit 106 controls the image processing unit 103 to acquire imaging data captured by the imaging unit 101, and generates image data by performing predetermined image processing on the acquired imaging data.

[0040] In S502, the control unit 106 determines whether or not a moving object is present in the image data generated in S501. If it is determined that a moving object is present in the image data, the process proceeds to S503. On the other hand, if it is determined that a moving object is not present in the image data, the process proceeds to S507. The control unit 106 controls the image analysis unit 105 to perform the process of detecting whether or not a moving object is present in the image data.

[0041] In this embodiment, the image analysis unit 105 compares the difference with past image data and detects moving objects from changes in the image, but any detection method may be used. For example, a method may be used in which the image is compared with pre-registered sample images of potential moving objects, such as ships. Furthermore, processing may be performed to exclude moving objects that should not be noted, such as drifting objects or flying objects. Taking the image data 207 in FIG. 3(B) as an example, a first ship 205 and a second ship 206 are detected as moving objects.

[0042] In S503, the control unit 106 calculates the upper speed limit (threshold) Vm of the detected moving object. Note that, if multiple moving objects are detected, the control unit 106 calculates the upper speed limit Vm for each moving object. Using the image data of FIG. 3(B) as an example, the control unit 106 calculates the upper speed limit Vm for each of the first ship 205 and the second ship 206. The upper speed limit Vm is used as a speed threshold (predetermined threshold) for determining whether the moving object is moving at a speed faster than the distribution rate Ts. Note that the upper speed limit Vm of the moving object is calculated by the control unit 106, for example, controlling the image analysis unit 105. With reference to FIG. 3, a method for calculating the upper speed limit Vm1 of the first ship 205 will be described below.

[0043] If a moving object moves more than the number of pixels occupied by the moving object during the distribution rate Ts, there is a risk that the moving object will be lost when the next image data is distributed. Therefore, the first ship 205 calculates the upper speed limit Vm1 using the following equations (1) and (2) to determine whether the moving object will move X1 pixels during Ts.

number

number

[0044] Equation (1) is an equation for estimating the horizontal length of the first vessel 205, and when the angle of view Φ is 62 degrees, y1 is calculated to be approximately 12.0 m. Next, when the distribution rate Ts is 1 second, equation (2) calculates the upper speed limit Vm1 for the first vessel 205 to be 12.0 m / s. As in the case of the first vessel 205, the upper speed limit Vm2 for the second vessel 206 is calculated to be 120 m / s. As can be seen from the above equation, the smaller the moving body and the closer the imaging distance, the lower the calculated upper speed limit Vm. In this way, the control unit 106 calculates the upper speed limit Vm for each moving body based on the time interval for distributing image data to the external device 111 and the distance from the imaging device 100 to the moving body. The control unit 106 also functions as a calculation unit when calculating the upper speed limit Vm.

[0045] In S504, the control unit 106 calculates the speed Vn of the detected moving object. If there are multiple moving objects (if multiple moving objects are detected), the control unit 106 calculates the speed Vn for each moving object. The speed Vn of the moving object is calculated by the control unit 106 controlling the image analysis unit 105. Using the image data of FIG. 3(B) as an example, the speed Vn1 of the first ship 205 is set to 15.0 m / s. Furthermore, the speed Vn2 of the second ship 206 is set to 10.0 m / s.

[0046] In S505, the control unit 106 compares the speed Vn of the moving object with the upper speed limit Vm for the moving object to determine whether the speed Vn of the moving object exceeds the upper speed limit Vm for the moving object. If the speed Vn of the moving object does not exceed the upper speed limit Vm for the moving object (if it is less than the predetermined threshold), the process proceeds to S507. On the other hand, if the speed Vn of the moving object exceeds the upper speed limit Vm for the moving object (if it is equal to or greater than the predetermined threshold), the process proceeds to S506. Note that if there are multiple moving objects, the process of S505 is performed for each moving object. That is, when multiple moving objects are detected and the process of S505 is performed, if the speed Vn of one or more of the moving objects exceeds the upper speed limit Vm for the moving object, the process proceeds to S506. As described above, in this embodiment, the control unit 106 also functions as a comparison unit that detects moving objects from image data and compares the speed Vn of the detected moving object with the upper speed limit Vm, which is a predetermined threshold set for each moving object. Then, the control unit 106, which also functions as a comparison means, determines whether the speed Vn of the moving object exceeds the upper speed limit Vm for the moving object.

[0047] Using the image data of Figure 3(B) as an example, the second vessel 206 has a speed Vn of 10.0 m / s and an upper speed limit Vm of 120 m / s, so the control unit 106 determines that the speed Vn of the moving body does not exceed the upper speed limit Vm for the moving body. On the other hand, the first vessel 205 has a speed Vn of 15.0 m / s and an upper speed limit Vm of 12.0 m / s, so the control unit 106 determines that the speed Vn of the moving body exceeds the upper speed limit Vm for the moving body. In this case, the speed Vn of the first vessel 205 exceeds the upper speed limit Vm for the first vessel 205, so the process proceeds to S506.

[0048] In S506, the control unit 106 registers the target moving body (subject) in the storage unit 102 as a moving body to be subjected to the compositing process described below. That is, the moving body whose speed Vn exceeded the upper speed limit Vm in S505 is stored in the storage unit 102 as a moving body to be subjected to the compositing process. Taking the image data of FIG. 3(B) as an example, since it was determined in S505 that the speed Vn of the first ship 205 exceeds the upper speed limit Vm for the first ship 205, the first ship 205 becomes a moving body to be subjected to the processing of S506. That is, the control unit 106 stores the first ship 205 in the storage unit 102 as a moving body to be subjected to the compositing process. On the other hand, since it was determined that the speed Vn of the second ship 206 does not exceed the upper speed limit Vm for the second ship 206, the second ship 206 is not subjected to the processing of S506. That is, the control unit 106 does not treat the second ship 206 as a moving body to be subjected to the synthesis process, and does not store it in the storage unit 102 either.

[0049] In S507, the control unit 106 records the image data generated in S501 in the storage unit 102. Thereafter, the processing in Fig. 5 ends. Note that in this embodiment, the processing of the flowchart shown in Fig. 5 is performed for each frame of the imaging device 100 set to a frame rate of 30 fps, but this is not limited to this. For example, a configuration may be adopted in which the processing is performed every few frames, and the frames for which moving object detection is performed are limited.

[0050] Next, the image synthesis process of the imaging device 100 and the process related to the distribution of image data will be described with reference to Fig. 6 and Fig. 7. Fig. 6 is a flowchart showing the process of the imaging device 100 according to this embodiment. Fig. 7 is a diagram supplementing the process of the flowchart shown in Fig. 6. Note that each operation (process) shown in the flowchart of Fig. 6 is realized by the control unit 106 of the imaging device 100 executing a program stored in the ROM 11 or the like. Furthermore, by adding an S to the beginning of each process (step), the notation of the process (step) is omitted.

[0051] This flowchart is executed at each predetermined distribution time, but may be configured to omit processing depending on the communication status with the external device 111. For example, it may be executed only when distribution is performed using satellite communication while navigating the open ocean, and may be switched to distributing all of the obtained image data to land without processing in an environment where high-speed wireless communication is possible, such as a coastal area.

[0052] In S601, the control unit 106 determines whether the moving object to be subjected to the synthesis process is stored in the storage unit 102. If it is not stored in the storage unit 102, the process proceeds to S602. On the other hand, if it is stored in the storage unit 102, the process proceeds to S603.

[0053] In S602, the control unit 106 selects (selects) image data to be distributed. The control unit 106 selects one image from the image data most recently stored in the storage unit 102 as the image data to be distributed. However, the present invention is not limited to this, and any one image may be selected from the image data stored in the storage unit 102 between the most recent distribution time and the current time. For example, in a situation where there is rippling water, data that is highly similar to the image data to be distributed the previous time may be selected to reduce image blur.

[0054] Here, an example of the processes of S603, S604, S605, and S606 in Fig. 6 will be described with reference to Fig. 7. All image data 220 in Fig. 7 indicates all image data recorded in one interval of a delivery rate Ts. The imaging device 100 in this embodiment acquires data at 30 fps, and the delivery rate Ts is 1 second. Therefore, the total image data 220 includes 30 frames.

[0055] In S603, the control unit 106 selects reference image data (first image) 221 from the image data stored in the storage unit 102. Specifically, the control unit 106 selects and extracts reference image data 221 for synthesis processing from all image data 220 stored in the storage unit 102. In this embodiment, the reference image data 221 is selected as the oldest data among all image data 220. However, this is not limited thereto, and the reference image data 221 may be selected as any one image specified by the user or automatically by the control unit 106 depending on the processing status, for example.

[0056] In S604, the control unit 106 performs a cropping process of the moving object to be subjected to the composition process from the image. Specifically, the control unit 106 selects at least one image data (second image) from the entire image data 220 shown in FIG. 7, in which the area occupied by the first ship 205, which is a moving object, does not overlap. In the present embodiment, as an example, two image data (second images) are selected as images in which the area occupied by the first ship 205 does not overlap. Then, the area occupied by the first ship 205 is extracted (cropped) from each of the two image data. For example, from the tenth oldest image data 222 in the entire image data 220, the area 224a occupied by the first ship 205 (the area where the moving object exists) is extracted as a cropped image. Furthermore, from the twentieth oldest image data 223 in the entire image data 220, the area 224b occupied by the first ship 205 (the area where the moving object exists) is extracted as a cropped image. In this way, the two pieces of image data (second images) have different acquisition times. It is preferable that the second image is an image that was stored in the storage unit 102 before the reference image data (first image) 221. In other words, the reference image data (first image) 221 and the image data (second image) have different acquisition times (photographing times).

[0057] In S605, the control unit 106 generates a composite image (third image) 225 based on the reference image data 221 and the region corresponding to the moving object in the image data selected in S604. That is, in this embodiment, the control unit 106 performs a compositing process to superimpose regions 224a and 224b, which are regions respectively cropped from the two second images in the process of S604, onto the reference image data 221, thereby generating the composite image 225 as shown in FIG. 9. In other words, the control unit 106 generates the composite image 225 by superimposing the cropped images of regions 224a and 224b, which are images cropped from the reference image data 221. The control unit 106 controls the image processing unit 103 to generate the composite image. As shown in FIG. 7, the regions 224a and 224b are regions that do not overlap with the regions occupied by the first ship 205, and therefore the first ship 205 does not overlap even when combined.

[0058] In this way, the control unit 106 generates a composite image, which is the third image, in accordance with the comparison result in S505. That is, the control unit 106 generates a composite image, which is the third image, when it is determined in S505 that the speed Vn of the moving object exceeds the upper speed limit Vm for the moving object. In this process, the control unit 106 also functions as a generation unit that generates a composite image.

[0059] The generation of a composite image is not limited to the above method, and may be, for example, a process of expressing the moving path of the moving object with a line or the like so that the trajectory of the moving object can be seen. Furthermore, to make it clear that a composite process has been performed on the image, the composited area may be surrounded by a frame, or the image capture time may be displayed for each extracted image, making the image visually easier to understand. Furthermore, if there is important image data that captures an event that should be highlighted, such as the moment of an accident, the image data from that time may be selected preferentially.

[0060] Furthermore, it is assumed that the photographed area in the reference image data and the image to be cropped may be misaligned, for example, when the ship 200 is rocking due to waves. In this case, the amount of misalignment may be calculated from the image change, and when the cropped area is overwritten on the reference image data, the coordinates may be overwritten taking into account the calculated amount of misalignment. In other words, if there is a change in the angle of view of the imaging device 100 between the reference image data and the image data from which the moving object was extracted that is equal to or exceeds a threshold, a composite image may be generated based on the amount of change in the angle of view, the area containing the cropped moving object, and the reference image data.

[0061] In addition, the control unit 106 may generate a composite image when the second time interval, which is the time interval for delivering image data to the external device 111, is longer than a predetermined value set in advance, which is longer than the first time interval, which is the time interval for obtaining image data from the imaging unit 101.

[0062] In S606, the control unit 106 determines the composite image 225 generated in S605 as the image to be distributed (sets it as the image to be distributed).

[0063] In S607, the control unit 106 distributes the image data to the external device 111. Note that if the process proceeds from S601 to S602, the image data selected in S602 is distributed to the external device 111 via the communication unit 104. Also, if a composite image is set as an image to be distributed in S605, the composite image is distributed to the external device 111 via the communication unit 104. In this process, the control unit 106 also functions as a distribution unit that distributes image data of the image to be distributed to the external device 111 as described above. In this way, in this process, when a composite image, which is the third image, is generated, the composite image is distributed to the external device 111 as image data to be distributed.

[0064] As described above, in the imaging device 100 of this embodiment, when a fast moving object appears within the angle of view during the distribution rate Ts, the imaging device 100 registers the moving object, generates a composite image that shows the movement path of the registered moving object, and distributes the composite image. This allows a person viewing the image data for distribution on the external device 111 to grasp the situation without losing track of the direction from which the moving object came and the direction to which it departed, even if the moving object is moving at a fast speed.

[0065] Although the calculation method for the horizontal direction has been described in this embodiment, it can also be applied to the vertical direction by performing a similar calculation. Furthermore, although it is assumed that the imaging device 100 of this embodiment is installed on a ship and the external device 111 is installed on land, this is not limited to this. For example, the imaging device 100 may be installed on land. In such a case, the moving object is not limited to a ship, but may be any other vehicle, animal, or other moving object. Furthermore, the imaging device 100 of this embodiment makes it possible to grasp the situation without losing sight of the moving object even in an environment with limited communication speed. Therefore, when the imaging device 100 is installed on land, the technology of this embodiment can be applied in a similar manner to the above, even in remote areas such as mountainous regions or deep jungles. Furthermore, when the imaging device 100 is installed on land, it is also assumed that the external device 111 is installed on land.

[0066] Although the preferred embodiments of the present invention have been described above using examples and drawings, the present invention is not limited to these embodiments and various modifications and changes are possible within the scope of the gist of the present invention.

[0067] The disclosure of this embodiment includes the following configuration, method, and program.

[0068] (Configuration 1) an acquisition means for acquiring a first image and a second image having the same imaging range as the first image but different in time; a comparison means for comparing the speed of the moving object detected from the second image with a predetermined threshold value; a generating means for generating a third image based on the first image and a region in the second image corresponding to the moving object; and a distribution means for distributing the third image to an external device in accordance with a comparison result of the comparison means. An imaging device characterized by:

[0069] (Configuration 2) The imaging device according to configuration 1, further comprising a calculation means for calculating the predetermined threshold value for the moving body based on a time interval for delivering image data to the external device and a distance from the imaging device to the moving body.

[0070] (Configuration 3) 3. The imaging device according to claim 1, wherein the generating means generates the third image when the comparing means determines that the speed of the moving object is equal to or greater than a predetermined threshold.

[0071] (Configuration 4) The imaging device according to any one of configurations 1 to 3, wherein the generating means generates the third image by cutting out an area corresponding to the moving object from the second image and superimposing the cut-out area on the first image.

[0072] (Configuration 5) the acquisition means acquires a plurality of second images of the same imaging range but at different times; The imaging device described in any one of configurations 1 to 4, characterized in that the generation means generates the third image by cutting out areas corresponding to the moving object from multiple second images and superimposing the multiple cut-out areas on the first image.

[0073] (Configuration 6) The imaging device according to any one of configurations 1 to 5, wherein the generating means selects the first image from a group of images recorded during one time interval during which image data is delivered to the external device.

[0074] (Configuration 7) The imaging device according to any one of configurations 1 to 6, wherein the generating means selects the second image from a group of images recorded during one time interval during which image data is delivered to the external device.

[0075] (Configuration 8) a storage means for storing the first image and the second image; 8. The imaging device according to any one of configurations 1 to 7, wherein the first image is an image stored in the storage means before the second image is stored.

[0076] (Configuration 9) 9. The imaging device according to any one of configurations 1 to 8, wherein the second image is an image acquired by the acquisition means earlier in time than the first image.

[0077] (Configuration 10) The imaging device according to configuration 8, wherein the distribution means selects image data to be distributed to the external device from the storage means when the moving object is not detected in the second image.

[0078] (Configuration 11) The imaging device described in any one of configurations 1 to 10, characterized in that the generation means generates the third image when a second time interval, which is a time interval for delivering image data to the external device, is longer than a first time interval, which is a time interval for acquiring an image by the acquisition means.

[0079] (Configuration 12) a detection means for detecting a subject from the second image; 12. The imaging device according to any one of configurations 1 to 11, wherein when a plurality of subjects are detected by the detecting means, the comparing means performs the comparison for each detected moving object.

[0080] (Configuration 13) 13. The imaging device described in any one of configurations 1 to 12, wherein, when there is a change in the angle of view of the imaging device between the first image and the second image that is equal to or greater than a threshold, the generation means generates the third image based on the amount of change in the angle of view, the cut-out area, and the first image.

[0081] (Configuration 14) A control method for an imaging device, comprising: an acquisition step of acquiring a first image and a second image having the same imaging range as the first image but different in time; a comparison step of comparing the speed of the moving object detected from the second image with a predetermined threshold; a generating step of generating a third image based on the first image and a region in the second image corresponding to the moving object; a delivery step of delivering the third image to an external device according to a comparison result in the comparison step. 10. A method for controlling an imaging device, comprising:

[0082] (Configuration 15) A program for causing a computer to execute a control method for an imaging device, the program including: an acquisition step of acquiring a first image and a second image having the same imaging range as the first image but different in time; a comparison step of comparing the speed of the moving object detected from the second image with a predetermined threshold; a generating step of generating a third image based on the first image and a region in the second image corresponding to the moving object; a delivery step of delivering the third image to an external device according to a comparison result in the comparison step. A program characterized by: [Explanation of symbols]

[0083] 100 Imaging device 101 Imaging unit 102 Storage section 103 Image processing section 104 Communications Department 105 Image Analysis Unit 106 Control Unit 110 Network 111 External device 200 ships

Claims

1. an acquisition means for acquiring a first image and a second image having the same imaging range as the first image but different in time; a comparison means for comparing the speed of the moving object detected from the second image with a predetermined threshold value; a generating means for generating a third image based on the first image and a region in the second image corresponding to the moving object; and a distribution means for distributing the third image to an external device in accordance with a comparison result of the comparison means. An imaging device characterized by:

2. 2. The imaging device according to claim 1, further comprising a calculation means for calculating the predetermined threshold value for the moving body based on a time interval for distributing image data to the external device and a distance from the imaging device to the moving body.

3. 2. The imaging device according to claim 1, wherein the generating means generates the third image when the comparing means determines that the speed of the moving object is equal to or greater than a predetermined threshold value.

4. The imaging device according to claim 1, characterized in that the generating means generates the third image by cutting out an area corresponding to the moving object from the second image and superimposing the cut-out area on the first image.

5. the acquisition means acquires a plurality of second images of the same imaging range at different times, The imaging device according to claim 1, characterized in that the generating means generates the third image by cutting out areas corresponding to the moving object from each of the second images and superimposing the cut-out areas on the first image.

6. 2. The imaging device according to claim 1, wherein the generating means selects the first image from a group of images recorded during one time interval during which image data is distributed to the external device.

7. 2. The imaging device according to claim 1, wherein the generating means selects the second image from a group of images recorded during one time interval during which image data is distributed to the external device.

8. a storage means for storing the first image and the second image; 2. The imaging device according to claim 1, wherein the first image is an image stored in the storage means before the second image is stored.

9. 2. The imaging device according to claim 1, wherein the second image is an image acquired by the acquisition means earlier in time than the first image.

10. 9. The imaging device according to claim 8, wherein the distribution means selects image data to be distributed to the external device from the storage means when the moving object is not detected in the second image.

11. The imaging device according to claim 1, characterized in that the generation means generates the third image when a second time interval, which is the time interval for delivering image data to the external device, is longer than a first time interval, which is the time interval for acquiring an image by the acquisition means.

12. a detection means for detecting a subject from the second image; 2. The imaging apparatus according to claim 1, wherein when a plurality of subjects are detected by said detecting means, said comparison by said comparing means is performed for each detected moving object.

13. The imaging device according to claim 1, characterized in that, when there is a change in the angle of view of the imaging device between the first image and the second image that is equal to or greater than a threshold, the generation means generates the third image based on the amount of change in the angle of view, the cut-out area, and the first image.

14. A control method for an imaging device, comprising: an acquisition step of acquiring a first image and a second image having the same imaging range as the first image but different in time; a comparison step of comparing the velocity of the moving object detected from the second image with a predetermined threshold value; a generating step of generating a third image based on the first image and a region in the second image corresponding to the moving object; a delivery step of delivering the third image to an external device in accordance with a comparison result in the comparison step.

10. A method for controlling an imaging device, comprising:

15. A program for causing a computer to execute a control method for an imaging device, the program including: an acquisition step of acquiring a first image and a second image having the same imaging range as the first image but different in time; a comparison step of comparing the velocity of the moving object detected from the second image with a predetermined threshold value; a generating step of generating a third image based on the first image and a region in the second image corresponding to the moving object; a distribution step of distributing the third image to an external device according to a comparison result in the comparison step. A program characterized by:

Images