Control device, program, and monitoring system

Abstract

A control device (20) for controlling an imaging device (30) in which point cloud data of a monitoring area AR output by a LiDAR device (10) is input and which is capable of moving the monitoring area AR, the control device (20) comprising: an extraction unit (21) for extracting a moving body M from the point group data; a position identification unit (22) for identifying the position of the extracted moving body (M); and an instruction unit (24) for outputting, to the imaging device (30), an instruction to move the position of the moving body M to a position at which the moving body M can be captured.

Description

Control device, program, and monitoring system

[0001] The present invention relates to a control device, a program, and a monitoring system.

[0002] Surveillance systems that use surveillance cameras to monitor people in a surveillance area are in practical use. Patent Document 1 below describes such a surveillance system. In this surveillance system, a LiDAR (Light Detection and Ranging) device measures the surveillance area, and people photographed by the camera are associated with moving objects measured by the LiDAR device. Therefore, the movements of people identified by the camera can be identified by the LiDAR device. The surveillance system of Patent Document 1 uses one or more cameras.

[0003] Japanese Patent Application Laid-Open No. 2022-154335

[0004] The surveillance system described in the above patent document cannot capture an image of a person if a fixed object is located between the camera and the person. Therefore, there is a need to capture an image of the subject even when there is a fixed object other than the person. In addition, the subject may be a moving object such as a car or an animal, in addition to a person.

[0005] Therefore, the present invention aims to provide a control device, program, and monitoring system that can realize a monitoring system that can photograph a moving object even if there are objects other than the moving object being photographed in the monitoring area.

[0006] A control device that receives point cloud data of a monitoring area output by a LiDAR device and controls an imaging device that can move within the monitoring area, and is characterized by comprising: an extraction unit that extracts a moving object from the point cloud data; a position identification unit that identifies the position of the extracted moving object; and an instruction unit that outputs an instruction to the imaging device to move the position of the moving object to a position where it can be photographed.

[0007] In addition, the program of the present invention is a program executed by a control device that controls an imaging device that can move within a monitoring area based on point cloud data of the monitoring area output by a LiDAR device, and is characterized by comprising: an extraction step that extracts a moving object from the point cloud data; a position identification step that identifies the position of the extracted moving object; and an instruction step that outputs an instruction to the imaging device to move the position of the moving object to a position where it can be photographed.

[0008] In addition, the monitoring system of the present invention comprises a LiDAR device, an imaging device capable of moving within the monitoring area of ​​the LiDAR device, and a control device that controls the imaging device based on point cloud data of the monitoring area output by the LiDAR device, and is characterized in that the control device has an extraction unit that extracts a moving object from the point cloud data, a position identification unit that identifies the position of the extracted moving object, and an instruction unit that outputs an instruction to the imaging device to move the position of the moving object to a position where it can be photographed.

[0009] In the present invention, the control device moves the image capture device to a position where it can capture the position of the moving object based on point cloud data from the LiDAR device. Therefore, even if there is a fixed object other than the moving object to be captured in the monitoring area, the image capture device can capture the moving object while avoiding the fixed object.

[0010] The control device and monitoring system preferably further include an orientation specifying unit that specifies a predetermined orientation of the extracted moving object, and the instruction unit outputs an instruction to the imaging device, which is movable within the monitoring area and capable of changing its imaging direction, to move from the position of the moving object to a position moved toward the predetermined orientation, and to direct the imaging direction toward the position of the moving object.Furthermore, the program preferably further includes an orientation specifying step that specifies a predetermined orientation of the extracted moving object, and the instruction step outputs an instruction to the imaging device, which is movable within the monitoring area and capable of changing its imaging direction, to move from the position of the moving object to a position moved toward the predetermined orientation, and to direct the imaging direction toward the position of the moving object.

[0011] With this configuration, the moving object can be photographed from a predetermined direction of the moving object. In this case, if the predetermined direction is, for example, the front of the moving object, the moving object can be photographed from the front side.

[0012] In the control device and monitoring system, the orientation identifying unit preferably identifies the predetermined orientation of the moving object from the point cloud data. Also, in the program, the orientation identifying step preferably identifies the predetermined orientation of the moving object from the point cloud data.

[0013] In this case, the predetermined orientation of the moving body can be identified almost simultaneously with the extraction of the moving body, so that the moving body can be extracted and the predetermined orientation can be identified in a short time.

[0014] In the control device and the monitoring system, it is preferable that the direction identification unit sets the moving direction of the moving object to the predetermined direction. Also, in the program, it is preferable that the direction identification unit sets the moving direction of the moving object to the predetermined direction.

[0015] In most cases, the moving object moves in a direction facing the front of the moving object, and therefore, with the above configuration, the moving object can be photographed from the front side of the moving object.

[0016] Alternatively, in the above-mentioned control device and monitoring system, it is preferable that data related to a video signal from the imaging device is input to the orientation identification unit, and the orientation identification unit identifies the predetermined orientation of the moving object from the data related to the video signal. Also, in the above-mentioned program, it is preferable that data related to a video signal from the imaging device is input to the above-mentioned control device, and the orientation identification step identifies the predetermined orientation of the moving object from the data related to the video signal.

[0017] Generally, an image capture device can capture images with a higher pixel count than a LiDAR device, allowing for more detailed images of a moving object. Therefore, by determining the predetermined orientation of a moving object from data related to the video signal, the predetermined orientation can be accurately determined.

[0018] In the above control device and monitoring system, it is preferable that the extraction unit extracts a predetermined part of the moving object, the position identification unit identifies the position of the predetermined part, and the instruction unit outputs an instruction to the imaging device to move the position of the predetermined part to a position where it can be imaged. Also, in the above program, it is preferable that the extraction step extracts a predetermined part of the moving object, the position identification step identifies the position of the predetermined part, and the instruction step outputs an instruction to the imaging device to move the position of the predetermined part to a position where it can be imaged.

[0019] With this configuration, it is possible to capture an image of a predetermined part of the moving object that is identified based on the point cloud data. For example, if the moving object is a person, the predetermined part can be the person's hands.

[0020] In the control device and monitoring system, the extraction unit preferably extracts the predetermined portion from the point cloud data. Also, in the program, the extraction step preferably extracts the predetermined portion from the point cloud data.

[0021] In this case, since the extraction of the moving body and the extraction of the predetermined portion are obtained almost simultaneously, the extraction of the moving body and the extraction of the predetermined portion can be performed in a short time.

[0022] In the control device and monitoring system, the extraction unit preferably determines the part of the moving object that moves the most as the predetermined part. Also, in the program, the extraction step preferably determines the part of the moving object that moves the most as the predetermined part.

[0023] In a surveillance device, a part that moves a lot, such as a person's hand, may be a part that is highly valuable to monitor. Therefore, with the above configuration, it is possible to capture an image of a part that is highly valuable to monitor.

[0024] In the above control device and monitoring system, it is preferable that the extraction unit receives data relating to the video signal from the imaging device, and the extraction unit extracts the predetermined part from the data relating to the video signal. Also, in the above program, it is preferable that the control device receives data relating to the video signal from the imaging device, and the extraction step extracts the predetermined part from the data relating to the video signal.

[0025] As described above, an imaging device can generally capture a moving object in more detail than a LiDAR device, and therefore, by extracting a specific part of the moving object from the data related to the video signal, the specific part can be accurately extracted.

[0026] In the control device and the monitoring system, it is preferable that the instruction unit outputs instructions to the plurality of image capturing devices to move the position of the identified moving object to positions where the position can be photographed from different directions. Also, in the program, it is preferable that the instruction step outputs instructions to the plurality of image capturing devices to move the position of the identified moving object to positions where the position can be photographed from different directions.

[0027] In this case, a multi-angle image can be obtained.

[0028] As described above, according to the present invention, a control device, a program, and a monitoring system are provided that can realize a monitoring system that can photograph a moving object even if there are objects other than the moving object being photographed in the monitoring area.

[0029] FIG. 1 is a conceptual diagram showing a monitoring system in a first embodiment of the present invention. FIG. 2 is a block diagram mainly showing a LiDAR device and a control device. FIG. 3 is a block diagram showing an imaging device. FIG. 4 is a flowchart showing a first operation of the control device in the first embodiment. FIG. 5 is a flowchart showing a second operation of the control device in the first embodiment. FIG. 6 is a flowchart showing a first operation of the control device in a second embodiment of the present invention. FIG. 7 is a flowchart showing a second operation of the control device in the second embodiment of the present invention. FIG. 8 is a conceptual diagram showing a monitoring system in a third embodiment of the present invention.

[0030] Below, embodiments for implementing a control device, a program, and a monitoring system according to the present invention are illustrated with the accompanying drawings. The embodiments illustrated below are intended to facilitate understanding of the present invention and are not intended to limit the present invention. The present invention can be modified and improved within the scope of the claims without departing from the spirit thereof. Furthermore, the present invention may also be realized by appropriately combining the components in the embodiments illustrated below. Note that in the drawings referred to below, the dimensions of each component may be changed to facilitate understanding.

[0031] First Embodiment Fig. 1 is a diagram showing a monitoring system according to the present embodiment. As shown in Fig. 1, the monitoring system 1 according to the present embodiment mainly includes a LiDAR device 10, a control device 20, and an imaging device 30.

[0032] The LiDAR device 10 will now be described.

[0033] As shown in FIG. 1 , the LiDAR device 10 is installed at a slightly elevated position so as to be able to detect objects in a monitoring area AR. For example, if the monitoring area AR is a road, the LiDAR device 10 is installed on a pole near the road. If the monitoring area AR is indoors, the LiDAR device 10 is installed on a pillar or the like. The LiDAR device 10 detects objects by emitting laser light into the monitoring area AR and detecting the light reflected from the laser light. Examples of objects that the LiDAR device 10 detects include moving objects such as people and cars, and fixed objects such as buildings and shelves. FIG. 1 illustrates an example in which a rectangular object such as a shelf is the fixed object F and a person is the moving object M. The arrow indicates the direction of movement of the moving object M. Through this detection, the LiDAR device 10 can detect, for example, the shapes of the fixed object F and the moving object M. Note that the fixed object F and the moving object M shown in this example are merely illustrative. For example, the fixed object F may have a shape other than a rectangle, and the moving object M may be a moving object other than a person.

[0034] FIG. 2 is a block diagram mainly showing the LiDAR device 10 and the control device 20. The LiDAR device 10 of this embodiment is, for example, a raster scan type LiDAR device. The LiDAR device 10 of this embodiment mainly comprises a cover 19, a driver circuit 11 housed inside the cover 19, a laser light source 12, an H-direction scanning drive mirror 13, a V-direction scanning drive mirror 14, a light receiving element 15, and a point cloud data generator 16. In the example of FIG. 2, the LiDAR device 10 is a mechanical type LiDAR device, but it may also be a phased array type LiDAR device that does not include a drive unit (not shown) described below.

[0035] The cover 19 has a storage space for housing the driver circuit 11, the laser light source 12, the driving mirror 13 for H-direction scanning, the driving mirror 14 for V-direction scanning, the light receiving element 15, and the point cloud data generation unit 16, and transmits the laser light Lb emitted from the laser light source 12 and the reflected light Lr that is reflected by an object within the monitoring area AR.

[0036] The driver circuit 11 is composed of, for example, a plurality of logic circuits, and is electrically connected to the laser light source 12, the H-direction scanning drive mirror 13, and the V-direction scanning drive mirror 14 to control them.

[0037] The laser light source 12 emits laser light Lb of a predetermined wavelength. This laser light Lb is near-infrared light with a wavelength of, for example, 905 nm or 1550 nm. The timing at which the laser light source 12 emits the laser light Lb is controlled by a driver circuit 11, and the laser light source 12 emits the laser light Lb in response to a signal from the driver circuit 11. The driver circuit 11 is electrically connected to a point cloud data generator 16, and outputs data including the timing at which the laser light Lb is emitted from the laser light source 12 to the point cloud data generator 16.

[0038] The H-direction scanning drive mirror 13 includes a mirror that reflects the laser light Lb emitted from the laser light source 12 and a drive unit (not shown) that is controlled by the driver circuit 11. When reflecting the laser light Lb, the H-direction scanning drive mirror 13 reflects the laser light Lb while changing the reflection angle in the horizontal direction using the drive unit. By changing the reflection angle of the H-direction scanning drive mirror 13, the LiDAR device 10 performs horizontal scanning.

[0039] The V-direction scanning drive mirror 14 includes a mirror that reflects the laser light Lb reflected by the H-direction scanning drive mirror 13 and a drive unit (not shown) controlled by the driver circuit 11. When reflecting the laser light Lb, the V-direction scanning drive mirror 14 reflects the laser light Lb while changing the reflection angle in the vertical direction using the drive unit. This change in the reflection angle of the V-direction scanning drive mirror 14 changes the horizontal scanning position performed by the LiDAR device 10 in the vertical direction. The laser light reflected by the V-direction scanning drive mirror 14 passes through the cover 19 and is irradiated forward of the LiDAR device 10.

[0040] The H-direction scanning drive mirror 13 and the V-direction scanning drive mirror 14 are configured to include, for example, a polygon mirror or a galvanometer mirror. The H-direction scanning drive mirror 13 and the V-direction scanning drive mirror 14 may each be configured as a MEMS mirror. The H-direction scanning drive mirror 13 and the V-direction scanning drive mirror 14 may be combined into two mirrors with biaxial scanning, and the order in which the laser light Lb is reflected by the H-direction scanning drive mirror 13 and the V-direction scanning drive mirror 14 may be reversed.

[0041] The light receiving element 15 is an element that receives reflected light Lr, which is laser light Lb reflected by an object in the monitoring area AR. The reflected light Lr received by the light receiving element 15 contains information related to the object located in the monitoring area AR. The light receiving element 15 outputs a signal that changes depending on the intensity of the received light. The light receiving element 15 is electrically connected to the point cloud data generation unit 16, and the information is input to the point cloud data generation unit 16 as an electrical signal.

[0042] The point cloud data generator 16 generates point data for each reflection position based on the direction of the reflection position where the laser beam Lb is reflected, the distance to the reflection position, and the intensity of the light received by the light receiving element 15, based on data related to the emission timing of the laser beam input from the driver circuit 11, information input from the light receiving element 15, and timing data of the information input from the light receiving element 15. The point data includes information related to the coordinates of the point and the intensity of the light reflected at the position that becomes the point data. Therefore, the point cloud data generator 16 generates point cloud data, which is a collection of point data. The point cloud data generator 16 is electrically connected to the control device 20, and the point cloud data is input to the control device 20.

[0043] Next, the control device 20 will be described.

[0044] The control device 20 is composed of, for example, an integrated circuit such as a microcontroller, an integrated circuit (IC), a large-scale integrated circuit (LSI), or an application-specific integrated circuit (ASIC), or an NC (numerical control) device. The control device 20 may or may not use a machine learning device. In this embodiment, the control device 20 is electrically connected to the LiDAR device 10, the memory 40, and the monitor 60, and is configured to be able to wirelessly input and output signals to and from the imaging device 30.

[0045] The memory 40 is configured to store information and to be able to read the stored information. The memory 40 is, for example, a non-transitory recording medium, and is preferably a semiconductor recording medium such as a random access memory (RAM) or a read-only memory (ROM), but may also include any type of recording medium, such as an optical recording medium or a magnetic recording medium. Note that "non-transitory" recording medium includes all computer-readable recording media except for transient, propagating signals, and does not exclude volatile recording media. Note that the memory 40 and the control device 20 may be provided in an integrated package. The memory 40 stores data necessary for generating programs and information executed by the control device 20. The memory 40 also stores data related to the shape of the moving object M and background data as needed. The control device 20 reads out the programs and information stored in the memory 40. The memory 40 also stores information and the like in response to instructions from the control device 20.

[0046] The control device 20 has an extraction unit 21, a position identification unit 22, an orientation identification unit 23, an instruction unit 24, and an antenna 29, each of which is electrically connected via a bus line. When the control device 20 reads out a program stored in the memory 40, parts of the control device 20 become the extraction unit 21, the position identification unit 22, the orientation identification unit 23, and the instruction unit 24, and perform the respective operations. Note that the control device 20 may be configured so that the extraction unit 21, the position identification unit 22, the orientation identification unit 23, and the instruction unit 24 operate without reading out a program from the memory 25.

[0047] The extraction unit 21 is a component that extracts the moving body M from the point cloud data input from the LiDAR device 10. For example, background data, i.e., point cloud data when the moving body M does not exist, is stored in the memory 40, and the extraction unit 21 compares the background data with the point cloud data input from the LiDAR device 10 and extracts the moving body M from the difference. Alternatively, the extraction unit 21 compares the point cloud data of multiple frames input from the LiDAR device 10 and extracts the changed portion as the moving body M. Note that the method by which the extraction unit 21 extracts the moving body M is not limited to the above example.

[0048] In this embodiment, in addition to extracting the moving object M, the extraction unit 21 also extracts a predetermined part of the moving object M based on the point cloud data. Extracting the predetermined part in this manner allows the extraction of the moving object M and the extraction of the predetermined part to be performed in a short time. For example, if the moving object is a person, the person's hands can be cited as an example of the predetermined part. In this case, the extraction unit 21 extracts the predetermined part from the shape of the moving object M extracted from the point cloud data. For example, if the moving object M is a person as shown in FIG. 1 , the extraction unit 21 extracts, for example, hands as a predetermined part from the overall shape of the moving object M. In this case, for example, data representing the shape of a person and the shape of the person's hands is stored in the memory 40, and the extraction unit 21 reads the data from the memory 40, compares the data with the point cloud data, and extracts the hands as a predetermined part. Alternatively, the extraction unit 21 may extract a part of the moving object M that moves the most as a predetermined part. In this case, for example, if the moving object M is a person, the part with the greatest movement may be the person's hand. By extracting the person's hand as the predetermined part and photographing the predetermined part as described below, it may be possible to monitor theft, etc. In the surveillance system 1, a part with the greatest movement, such as a person's hand, may be highly valuable for surveillance. By designating the part with the greatest movement as the predetermined part, it is possible to photograph the part with the greatest monitoring value as described below. The extraction unit 21 may extract the predetermined part from the point cloud data by, for example, machine learning. Note that the predetermined part extracted by the extraction unit 21 and the method by which the extraction unit 21 extracts the predetermined part are not limited to the above examples.

[0049] The position identifying unit 22 identifies the position of the moving object M extracted by the extracting unit 21. That is, for example, when the monitoring area AR is represented by a three-axis coordinate system, the position identifying unit 22 indicates the position of the moving object M in that coordinate system. Note that the coordinates of the position of the moving object M may be, for example, the coordinates of a position that is approximately the center of the moving object M or the coordinates of the highest position of the moving object M, but are not limited to these.

[0050] Furthermore, in this embodiment, the position identifying unit 22 identifies the position of the moving object M, as well as the position of the predetermined part of the moving object M extracted by the extraction unit 21. In this case, the position identifying unit 22 indicates, for example, the position of the predetermined part of the moving object M using coordinates. Generally, the position range of the predetermined part is small, so the coordinates of the position of the predetermined part are, for example, the coordinates of a position that is approximately the center of the predetermined part. However, for example, the coordinates of a position other than the center of the predetermined part may be used as the coordinates of the position of the predetermined part.

[0051] The orientation identification unit 23 identifies the predetermined orientation of the moving body M extracted by the extraction unit 21. In this embodiment, the orientation identification unit 23 identifies the predetermined orientation of the moving body M from the point cloud data. In this case, the predetermined orientation of the moving body M can be identified almost simultaneously with the extraction of the moving body M, so that the moving body M can be extracted and the predetermined orientation can be identified in a short time. The orientation identification unit 23 identifies the predetermined orientation of the moving body M, for example, from the shape of the moving body M extracted from the point cloud data. In this case, for example, data indicating the relationship between the shape of the moving body M and the predetermined orientation is stored in the memory 40, and the orientation identification unit 23 compares the relationship data stored in the memory 40 with the point cloud data to identify the predetermined orientation of the moving body M. Furthermore, when the moving body M is moving based on multiple frames of point cloud data, the orientation identification unit 23 may identify the movement direction of the moving body M as the predetermined orientation of the moving body M. Therefore, in FIG. 1, the movement direction indicated by the arrow of the moving body M is the predetermined orientation. This case is suitable for the case where, as described above, the extraction unit 21 compares point cloud data of multiple frames and extracts a changing portion as the moving body M. Generally, the moving direction of the moving body M is the front of the moving body M. Therefore, by setting the moving direction to a predetermined orientation, it is possible to photograph the moving body M from the front side of the moving body M by photographing the moving body M from the predetermined orientation side as described below. Note that the method by which the orientation identification unit 23 identifies the orientation of the moving body M is not limited to the above.

[0052] The instructing unit 24 outputs an instruction to the image capturing device 30 to move the position of the moving object M identified by the position identifying unit 22 to a position where image capturing is possible. In this embodiment, the instructing unit 24 outputs an instruction to the image capturing device 30 to move from the position of the moving object M to a position toward the predetermined direction identified by the orientation identifying unit 23, and also outputs an instruction to orient the image capturing direction toward the position of the moving object M. In this case, this image capturing direction is the direction opposite to the predetermined direction. When the moving direction of the moving object M is the predetermined direction, in FIG. 1 , the direction opposite to the direction of the arrow becomes the image capturing direction. For example, assume that the coordinates of the moving object M are (x, y) = (5, 6) and the predetermined orientation of the moving object M is the (x, y) = (1, 1) direction. In this case, the instruction unit 24 outputs an instruction to the image capture device 30 to move to, for example, (x, y) = (5 + 1, 6 + 1) = (6, 7), and further outputs an instruction to the image capture device 30 to turn the image capture direction of the image capture device 30 in the (x, y) = (-1, -1) direction. Note that the above is an example where the position of the moving object is represented in a two-axis coordinate system. When the position of the moving object is represented in a three-axis coordinate system, the instruction unit 24 outputs an instruction in the three-axis coordinate system.

[0053] The antenna 29 is a component that transmits control signals from the control device 20 to the image capturing device 30. Examples of these control signals include signals related to movement instructions from the instruction unit 24 and control signals related to image capturing, such as the start / end of image capturing. A predetermined protocol for wireless communication is assigned to the control signal by a protocol assigning unit (not shown). In this embodiment, the antenna 29 receives video signals captured by the image capturing device 30 and imports the video signals into the control device 20.

[0054] In this embodiment, the monitor 60 is electrically connected to the control device 20. Therefore, the video signal input from the imaging device 30 is output to the monitor 60 via the antenna 29, and the video is displayed on the monitor 60.

[0055] Next, the photographing device 30 will be described.

[0056] Fig. 3 is a block diagram showing the image capture device 30. As shown in Figs. 1 and 3, the image capture device 30 is movably supported on rails 50. The configuration of the rails 50 is not particularly limited as long as the image capture device 30 is movable. In the example of Fig. 1, the rails 50 are formed in a square lattice pattern above the monitoring area AR. However, the rails 50 may also be formed in a triangular lattice pattern, a combination of a square lattice and a triangular lattice, or a Mondrian pattern.

[0057] 3, the imaging device 30 of this embodiment includes a main body 30M and a support unit 30S. The support unit 30S has a drive unit 31 and an imaging direction change unit 32, while the main body 30M mainly includes a drive control unit 33, an imaging unit 34, an image processing unit 35, an imaging control unit 36, and an antenna 39. These components are electrically connected via, for example, a bus line.

[0058] The antenna 39 receives control signals from the control device 20 and inputs the control signals into the image capturing device 30. These control signals include, as described above, signals for moving the image capturing device 30, the image capturing direction, and instructions for starting and stopping image capturing. The input control signals are sent to the drive control unit 33, the image capturing control unit 36, etc. In this embodiment, the antenna 39 also transmits video signals captured by the image capturing device 30.

[0059] The drive unit 31 is configured to allow the imaging device 30 to move on the rails 50. Therefore, when the rails 50 are configured in a square lattice pattern as described above, the drive unit 31 is configured to be movable in the x and y directions. In this case, the drive unit 31 has wheels that can switch the movement direction between the x and y directions, for example, and roll on the rails 50. Note that the wheels are omitted in the figure. When the drive unit 31 has wheels in this way, it has, for example, a stepping motor that rotates the wheels.

[0060] The shooting direction change unit 32 is configured to be able to change the orientation of the main body 30M of the imaging device 30. In other words, the shooting direction change unit 32 can change the shooting direction of the imaging device 30. The shooting direction change unit 32 is configured, for example, to include a biaxial hinge or ball joint that changes the rotational direction and vertical orientation of the main body 30M, and is equipped with a stepping motor that drives the main body 30M to change the orientation. Note that the main body 30M may have a sensor that detects the roll angle, pitch angle, yaw angle, etc. relative to the x, y, and z coordinates of the main body 30M, i.e., a sensor that detects the orientation of the main body 30M. The sensor is, for example, a gyro sensor. If such a sensor is included, a signal from the sensor is input to the drive control unit 33.

[0061] The drive control unit 33 receives signals related to movement instructions and imaging direction instructions from the instruction unit 24 of the control device 20 via the antenna 39 and controls the drive unit 31 and the imaging direction change unit 32. The drive control unit 33, for example, knows the position of the imaging device 30 on the rail 50 and controls the drive unit 31 so that the imaging device 30 moves to the position of coordinates specified in the movement instruction from the instruction unit 24. As described above, if the drive unit 31 has a stepping motor for moving the rail 50, the drive control unit 33 controls the number of rotations of the stepping motor so that the imaging device 30 moves to the position of coordinates specified in the movement instruction. Note that, for example, if the rail 50 has a coordinate assigning unit that can identify a position, the imaging device 30 may acquire the position identified by the coordinate assigning unit, and the drive control unit 33 may compare the movement instruction from the control device 20 with the position identified by the coordinate assigning unit and control the drive unit 31 based on the result of this comparison.

[0062] Furthermore, if the imaging direction change unit 32 has a stepping motor that changes the imaging direction of the imaging device 30, the drive control unit 33 rotates the stepping motor a predetermined number of times in response to a signal related to the imaging direction from the instruction unit 24, thereby changing the orientation of the imaging device 30. Furthermore, as described above, if the main body 30M has a sensor that detects a roll angle, a pitch angle, a yaw angle, etc., the drive control unit 33 compares the imaging direction of the imaging device 30 specified by the movement instruction from the instruction unit 24 with the orientation of the main body 30M specified by the output from the sensor, and controls the drive unit 31 so that the imaging direction of the imaging device 30 becomes the one based on the movement instruction from the instruction unit 24.

[0063] The image capturing unit 34 is composed of, for example, a matrix-type light receiving element such as a CMOS (Complementary Metal-Oxide-Semiconductor), a lens, etc. In this embodiment, the image capturing direction of the image capturing unit 34 is the direction of the main body 30M. The image capturing unit 34 may also have a zoom function, etc. The image capturing unit 34 is electrically connected to the image processing unit 35 and outputs a signal related to the captured image to the image processing unit 35.

[0064] The image processing unit 35 processes the video signal from the image capturing unit 34. The image processing unit 35 processes the video signal in accordance with a predetermined standard such as MPEG (Moving Picture Expert Group). The processed video signal is sent to the control device 20 via an antenna 39.

[0065] The photographing control unit 36 ​​controls the photographing unit 34 and the image processing unit 35. The photographing control unit 36 ​​controls the start and stop of photographing by the photographing unit 34 and the zoom of the photographing unit 34. The photographing control unit 36 ​​also controls the image processing unit 35 to control the start of processing of the video signal input from the photographing unit 34, etc.

[0066] The video signal processed by the image processing unit 35 is output from the antenna 39 as described above, and the control device 20 receives the video signal output from the imaging device 30 as described above and outputs the video signal to the monitor 60.

[0067] Next, a description will be given of the operation of the monitoring system 1 in this embodiment. The operation of the monitoring system 1 is performed by the control device 20 executing a program stored in the memory 40.

[0068] First, a first operation of the monitoring system 1 will be described. Fig. 4 is a flowchart showing the first operation of the control device 20. The first operation is an operation in which the extraction unit 21 extracts a moving object M from point cloud data, but does not extract a predetermined portion of the moving object M. As shown in Fig. 4, the operation of the control device 20 of this embodiment includes steps S11 to S18.

[0069] <Step S11> This step changes the next step depending on whether point cloud data has been input from the LiDAR device 10. If a predetermined number of frames of point cloud data have been input, the control device 20 proceeds to step S12. If the predetermined number of frames of point cloud data have not been input, the control device 20 repeats this step. The predetermined number of frames is, for example, 1, but may be multiple, as described below.

[0070] <Step S12> This step is an extraction step in which a moving body M is extracted from point cloud data. When the extraction unit 21 compares background data with point cloud data input from the LiDAR device 10 and extracts the moving body M from the difference, the extraction unit 21 may extract the moving body M from one frame of point cloud data in this step. Therefore, in this case, the predetermined number of frames in step S11 is 1. Also, when the extraction unit 21 compares point cloud data of multiple frames and extracts a changing portion as the moving body M, the extraction unit 21 extracts the moving body M from the point cloud data of the multiple frames. Therefore, in this case, the predetermined number of frames in step S11 is the multiple frames.

[0071] <Step S13> This step changes the next step depending on whether or not the moving object M has been extracted from the point cloud data. If the control device 20 has been able to extract the moving object M from the point cloud data, the control device 20 proceeds to step S14, and if the extraction has not been possible, the control device 20 returns to step S11. An example of when the extraction cannot be possible is when the moving object M does not exist in the monitoring area AR.

[0072] <Step S14> This step is a position identification step for identifying the position of the moving object M extracted in step S12. In this step, the position identification unit 22 identifies the position of the moving object M extracted by the extraction unit 21, and indicates the position of the moving object M in a predetermined coordinate system, for example. The identified position is output to the indication unit 24.

[0073] <Step S15> This step is a direction identification step for identifying the predetermined direction of the moving body M extracted by the extraction unit 21 in step S12. In this step, as described above, the direction identification unit 23, for example, identifies the predetermined direction of the moving body M from the shape of the extracted moving body M, or identifies the movement direction of the moving body M as the predetermined direction of the moving body M. When identifying the predetermined direction of the moving body M from the shape of the moving body M, the moving body M may be extracted from one frame of point cloud data. Furthermore, when identifying the movement direction of the moving body M as the predetermined direction of the moving body M, the predetermined direction of the moving body M is identified from point cloud data of multiple frames. Therefore, in this case, the predetermined number of frames in step S11 may be multiple. The identified predetermined direction of the moving body M is output to the instruction unit 24.

[0074] <Step S16> This step changes the next step depending on whether or not the predetermined orientation of the moving body M has been identified in step S15. In this step, if the control device 20 has not been able to identify the predetermined orientation of the moving body M in step S15, the control device 20 proceeds to step S17, and if the control device 20 has been able to identify the predetermined orientation of the moving body M in step S15, the control device 20 proceeds to step S18.

[0075] <Step S17> This step is an instruction step in which the imaging device 30 is instructed to move the identified position of the moving object M to a position where imaging can be performed, and to orient the imaging direction toward the position of the moving object M. This step is based on the assumption that a predetermined orientation of the moving object M was not identified in step S15. Therefore, in this step, the instruction unit 24 identifies a position where imaging can be performed for the position of the moving object M identified in step S14. The imaging position is, for example, a position in a predetermined direction from the position of the moving object M and a predetermined distance away from the moving object M. Furthermore, the instruction unit 24 identifies the direction toward the moving object M from the predetermined direction, i.e., the direction opposite to the above-mentioned specific direction. The identified information is output as part of the instruction. The instruction unit 24 outputs to the moving object M an instruction to move to the identified position where imaging can be performed, and an instruction to orient the imaging direction toward the moving object M from the predetermined direction. For example, if the position of the moving object M is (x, y) = (5, 6), the predetermined specific direction is (x, y) = (1, 2), and the predetermined specific distance is the distance (x, y) = (1, 2), i.e., √5, the instruction unit 24 outputs an instruction to the image capture device 30 to move to (x, y) = (5 + 1, 6 + 2) = (6, 8). The instruction unit 24 also outputs an instruction to the image capture device 30 to orient the main body 30M in the direction opposite to the predetermined direction, i.e., (x, y) = (-1, -2). That is, the instruction unit 24 outputs an instruction to set the image capture direction to (x, y) = (-1, -2). Data related to the output instruction is transmitted to the image capture device 30 via the antenna 29. The specific direction does not have to be a fixed direction. For example, the specific direction may be the direction opposite to the direction from the LiDAR device 10 toward the moving object M. The specific distance does not have to be a fixed distance, but may vary within a certain range, for example.

[0076] The instructions and data relating to the position and direction transmitted from the antenna 29 are input to the drive control unit 33 via the antenna 39 of the image capturing device 30. The drive control unit 33 controls the drive unit 31 based on this data. As a result, the image capturing device 30 moves in a specific direction from the position of the moving object M to a position a specific distance away from the moving object M. The drive control unit 33 also controls the image capturing direction change unit 32 based on this data. As a result, the image capturing direction change unit 32 causes the main body 30M to face in a direction opposite to the specific direction. Therefore, the image capturing direction of the image capturing device 30 faces the moving object M.

[0077] <Step S18> Similar to step S17, this step is an instruction step in which the image capture device 30 is instructed to move to a position in a predetermined direction from the position of the moving body M and to orient the image capture direction toward the position of the moving body M. However, this step is premised on the assumption that the predetermined orientation of the moving body M has been identified in step S15. Therefore, this step differs from step S17 in that the specified direction in step S17 is the predetermined orientation of the moving body M. That is, in this step, control is performed to orient the main body 30M of the image capture device 30 toward the moving body M from the predetermined orientation side of the moving body M. In this step, similar to step S17, the instruction unit 24 identifies a position where the position of the moving body M identified in step S14 can be captured. In this case, in this step, the position where the image capture of the moving body M can be performed is, for example, a position a certain distance away from the moving body M in the direction of the predetermined orientation of the moving body M from the position of the moving body M. Therefore, the instruction unit 24 specifies the direction from the predetermined orientation side of the moving object M toward the moving object M, i.e., the direction opposite to the predetermined orientation of the moving object M. The specified information is output as part of the instruction. Note that the specified distance does not need to be a fixed distance and may, for example, vary within a certain range. The instruction unit 24 outputs to the moving object M an instruction to move to the specified position where image capture is possible and an instruction to turn the image capture direction toward the moving object M from the predetermined orientation side. For example, if the position of the moving object M is (x, y) = (5, 6), the predetermined orientation of the moving object M is (x, y) = (2, 1), and the specific distance is the distance of (x, y) = (2, 1), i.e., √5, the instruction unit 24 outputs to the image capture device 30 an instruction to move to (x, y) = (5 + 2, 6 + 1) = (7, 7). Furthermore, the instruction unit 24 outputs an instruction to the image capturing device 30 to orient the main body 30M in the direction (x, y) = (-2, -1), which is the direction opposite to the predetermined orientation of the moving object M. Data related to the output instruction is transmitted to the image capturing device 30 via the antenna 29.

[0078] As in step S17, the data relating to the identified position and direction transmitted from the antenna 29 is input to the drive control unit 33 via the antenna 39 of the imaging device 30, and the imaging direction of the imaging device 30 becomes the direction facing the moving body M.

[0079] After steps S17 and S18, the photographing device 30 performs photographing in accordance with instructions from the control device 20 or instructions from the photographing control unit 36. In this way, according to this operation, the photographing device 30 can photograph the moving object M in a predetermined orientation or from a specific direction. If the predetermined orientation is the direction of movement of the moving object M, the photographing device 30 photographs the moving object M from the direction of movement. The video signal photographed by the photographing unit 34 is processed by the image processing unit 35 and output from the antenna 39. The control device 20 then receives the video signal from the antenna 29 and outputs it to the monitor 60. Upon receiving the video signal, the monitor 60 displays the video.

[0080] Next, a second operation of the monitoring system 1 will be described.

[0081] 5 is a flowchart showing a second operation of the control device 20. The second operation is an operation in which the extraction unit 21 extracts a moving body M from point cloud data and further extracts a predetermined portion of the moving body M. As shown in FIG. 5, the operation of the control device 20 of this embodiment includes steps S21 to S28.

[0082] <Steps S21 to S23> Steps S21 to S23 are the same as steps S11 to S13 in the first operation.

[0083] <Step S24> This step is an extraction step in which the extraction unit 21 extracts a predetermined part of the moving body M from the point cloud data. Steps S21 and S24 may be combined to form the extraction step. In this step, as described in the control device 20, the extraction unit 21 extracts the predetermined part from, for example, the shape of the moving body M extracted from the point cloud data. Also, for example, the extraction unit 21 extracts the part of the moving body M that moves the most as the predetermined part. Data related to the extracted predetermined part is input to the position identification unit 22.

[0084] <Step S25> This step changes the next step depending on whether or not the predetermined part of the moving body M has been extracted from the point cloud data. If the control device 20 cannot extract the predetermined part from the point cloud data, it proceeds to step S26, and if the extraction has been successful, it proceeds to step S27. Examples of when the extraction cannot be performed include when the predetermined part to be extracted is hidden.

[0085] <Step S26> This step includes steps similar to the multiple steps of the first operation A surrounded by the dashed line. In other words, if the control device 20 cannot extract the predetermined part of the moving object M, it proceeds to a step similar to step S14, identifies the position of the moving object M, and outputs an instruction to move the identified position of the moving object M to a position where it can be photographed, and an instruction to orient the photographing direction toward the position of the moving object M.

[0086] <Step S27> This step is a step in which the position identification unit 22 identifies the position of a predetermined part of the moving object M. In this step, as explained by the control device 20, for example, the position of the predetermined part of the moving object M is indicated, for example, in a coordinate system. Data related to the identified position of the predetermined part of the moving object M is output to the instruction unit 24.

[0087] <Step S28> This step is an instruction step that outputs an instruction to the imaging device 30 to move the position of the identified predetermined portion of the moving object M to a position where it can be photographed. In this step, the instruction unit 24 identifies a position where the position of the predetermined portion of the moving object M identified in step S27 can be photographed. In this example, the predetermined portion is extracted from the point cloud data from the LiDAR device 10. Therefore, the predetermined portion can be photographed from the LiDAR device 10 side. Therefore, the instruction unit 24 identifies, for example, a position moved a predetermined distance toward the LiDAR device 10 from the position of the predetermined portion as an imaging position. The instruction unit 24 also identifies the direction from the imaging position toward the predetermined portion. Information regarding the identified imaging position and the direction toward the predetermined portion are output as part of the instruction. The instruction unit 24 outputs an instruction to the moving object M to move to the identified imaging position where it can be photographed and an instruction to orient the imaging direction from the predetermined orientation toward the moving object M. For example, if the position of the moving object M is (x, y) = (5, 6), the direction from the LiDAR device 10 toward a predetermined portion of the moving object M is (x, y) = (1, 1), and the predetermined distance is (x, y) = (2, 2), i.e., √8, the instruction unit 24 outputs an instruction to the image capture device 30 to move to (x, y) = (5-2, 6-2) = (3, 4). The instruction unit 24 also outputs an instruction to the image capture device 30 to orient the main body unit 30M toward (x, y) = (1, 1), which is the direction from the LiDAR device 10 toward the predetermined portion of the moving object M. Data related to the output instruction is transmitted to the image capture device 30 via the antenna 29.

[0088] Similar to steps S17 and S18, the data relating to the identified position and direction transmitted from the antenna 29 is input to the drive control unit 33 via the antenna 39 of the photographing device 30, and the photographing direction of the photographing device 30 becomes a direction facing a specified part of the moving body M.

[0089] After steps S26 and S28, the photographing device 30 photographs in accordance with instructions from the control device 20 or instructions from the photographing control unit 36, in the same manner as in the first operation. Thus, according to this operation, the photographing device 30 can photograph a predetermined part of the moving object M. The video signal photographed by the photographing unit 34 is processed by the image processing unit 35 and output from the antenna 39. The control device 20 then receives the video signal from the antenna 29 and outputs it to the monitor 60. Upon receiving the video signal, the monitor 60 displays the video.

[0090] Second Embodiment Next, a second embodiment of the present invention will be described in detail with reference to Figures 6 and 7. Note that components that are the same as or equivalent to those in the first embodiment will be given the same reference numerals and will not be described again unless otherwise specified.

[0091] The control device 20 of this embodiment has the same configuration as the control device 20 of the first embodiment, but differs from the control device 20 of the first embodiment in that data related to the imaging signal captured by the imaging device 30 is further input to the extraction unit 21 and the orientation identification unit 23.

[0092] The extraction unit 21 extracts the moving body M from the point cloud data input from the LiDAR device 10, as in the first embodiment. In this embodiment, in addition to extracting the moving body M, the extraction unit 21 also extracts a predetermined part of the moving body M based on data related to the image capture signal. Generally, the image capture device 30 can capture the moving body M in more detail than the LiDAR device 10. Therefore, by extracting the predetermined part of the moving body M from data related to the image signal, the predetermined part can be extracted more accurately. When the extraction unit 21 extracts the predetermined part, the moving body M is first captured by the image capture device 30, as described below. The extraction unit 21 extracts the predetermined part from the shape of the moving body M in the data related to the image capture signal input from the image capture device 30. In this embodiment, for example, if the moving body M is a person as shown in FIG. 1 , the extraction unit 21 extracts, for example, a hand as a predetermined part from the overall shape of the moving body M in the data related to the image capture signal. In this case, for example, similar to the first embodiment, data indicating the shape of a person and the shape of the person's hand is stored in the memory 40, and the extraction unit 21 reads out the data from the memory 40, compares the data with the point cloud data, and extracts the hand as the predetermined part. Alternatively, the extraction unit 21 may extract the part of the moving object M that moves the most in the data related to the image capture signal as the predetermined part. The extraction unit 21 may extract the predetermined part from the data in the image capture signal by, for example, machine learning. Note that the predetermined part extracted by the extraction unit 21 and the method by which the extraction unit 21 extracts the predetermined part are not limited to the above example.

[0093] Furthermore, in this embodiment, the orientation identification unit 23 identifies the predetermined orientation of the moving body M based on data related to the imaging signal. Therefore, as described below, when the orientation identification unit 23 identifies the orientation of the moving body M, the imaging device 30 first captures the moving body M, as described below. The orientation identification unit 23 identifies the predetermined orientation of the moving body M, for example, from the shape of the moving body M in the data related to the imaging signal. In this case, for example, similar to the first embodiment, data indicating the relationship between the shape of the moving body M and the predetermined orientation is stored in the memory 40, and the orientation identification unit 23 compares the relationship data stored in the memory 40 with the point cloud data to identify the predetermined orientation of the moving body M. Furthermore, when the moving body M is moving based on data related to multiple frames of the imaging signal, the orientation identification unit 23 may identify the movement direction of the moving body M as the predetermined orientation of the moving body M. As described above, the imaging device 30 can generally capture images with a higher number of pixels than the LiDAR device 10, and can capture the moving body M in more detail. Therefore, by identifying the predetermined orientation of the moving body M from data related to the video signal, the predetermined orientation can be accurately identified.

[0094] Next, the operation of the monitoring system 1 in this embodiment will be described.

[0095] First, a first operation of the monitoring system 1 in this embodiment will be described. Fig. 6 is a flowchart showing the first operation of the control device 20 in this embodiment. The first operation of this embodiment is an operation in which the extraction unit 21 extracts a moving object M from point cloud data and extracts a predetermined direction of the moving object M from data related to an image capture signal. As shown in Fig. 6, the operation of the control device 20 in this embodiment includes steps S31 to S38.

[0096] <Steps S31 to S34> Steps S31 to S34 are the same as steps S11 to S14 in the first operation.

[0097] <Step S35> This step is similar to step S17 in the first embodiment. In this step of the present embodiment, a predetermined orientation of the moving object M is not specified. Therefore, in this step, similar to step S17 in the first embodiment, the instructing unit 24 specifies a position where the position of the moving object M specified in step S34 can be photographed, and specifies a direction toward the moving object M from the specified direction, i.e., a direction opposite to the specified direction. The instructing unit 24 outputs, together with the specified information, instructions to move the image capturing device 30 and change the image capturing direction as at least part of the instructions. Data related to the output instructions is transmitted to the image capturing device 30 via the antenna 29.

[0098] The data relating to the identified position and direction transmitted from the antenna 29 is input to the drive control unit 33 via the antenna 39 of the image capturing device 30, and the image capturing device 30 moves to a position a predetermined distance from the moving object M in a predetermined specific direction from the position of the moving object M. The drive control unit 33 also controls the image capturing direction change unit 32 based on the data. Therefore, the image capturing direction of the image capturing device 30 becomes a direction facing the moving object M. Thereafter, the image capturing device 30 captures an image in accordance with an instruction from the control device 20 or an instruction from the image capturing control unit 36. In this way, the image capturing device 30 captures an image of the moving object M from a specific direction of the moving object M. The video signal captured by the image capturing unit 34 is processed by the image processing unit 35 and output from the antenna 39. The control device 20 then receives the video signal from the antenna 29. In this embodiment, data relating to the video signal is input to the orientation identification unit 23.

[0099] <Step S36> This step is an orientation identification step in which the predetermined orientation of the moving object M is identified from data related to the video signal from the image capture device 30. This step differs from step S15 in the first embodiment in that the orientation identification unit 23 identifies the predetermined orientation of the moving object M from data related to the video signal. Therefore, similar to step S15 in the first embodiment, the orientation identification unit 23 identifies the predetermined orientation of the moving object M from the shape of the moving object M, or identifies the direction of movement of the moving object M as the predetermined orientation of the moving object M. The identified predetermined orientation of the moving object M is output to the instruction unit 24.

[0100] <Step S37> This step changes the next step depending on whether the predetermined orientation of the moving body M was identified in step S36. In this step, if the control device 20 cannot identify the predetermined orientation of the moving body M in step S36, it returns to step S31, and if the control device 20 can identify the predetermined orientation of the moving body M in step S36, it proceeds to step S38. Note that in this operation of the present embodiment, after step S35, the moving body M is photographed from a specific direction. If this photographing is continuing, the photographing continues.

[0101] <Step S38> This step is generally similar to step S18 in the first embodiment, and is an instruction step in which an instruction is output to the image capture device 30 to move to a position in a predetermined direction from the position of the moving object M, and an instruction is output to turn the image capture direction toward the position of the moving object M. However, this step differs from step S18 in that the predetermined direction is identified from data related to the video signal.

[0102] After this step, the photographing device 30 performs photographing in accordance with instructions from the control device 20 or instructions from the photographing control unit 36. In this way, through this operation, the photographing device 30 can photograph the moving object M in a predetermined orientation or from a specific direction. If the predetermined orientation is the direction of movement of the moving object M, the photographing device 30 photographs the moving object M from the direction of movement. The video signal photographed by the photographing unit 34 is processed by the image processing unit 35 and output from the antenna 39. The control device 20 then receives the video signal from the antenna 29 and outputs it to the monitor 60. Upon receiving the video signal, the monitor 60 displays the video.

[0103] Next, a second operation of this embodiment will be described.

[0104] 7 is a flowchart showing a second operation of the control device 20 in this embodiment. The second operation in this embodiment is an operation in which the extraction unit 21 extracts a moving object M from point cloud data and extracts a predetermined portion of the moving object M from data related to the image capture signal. As shown in FIG. 7, the operation of the control device 20 in this embodiment includes steps S41 to S49.

[0105] <Steps S41 to S44> Steps S41 to S44 are the same as steps S11 to S14 in the first operation.

[0106] <Step S45> This step is similar to step S35 of the present embodiment and step S17 of the first embodiment. However, in this step, data related to the video signal input from the image capturing device 30 is input to the extraction unit 21.

[0107] <Step S46> This step is an extraction step in which a predetermined part of the moving object M is extracted from data related to the video signal from the imaging device 30. This step differs from step S24 in the second operation of the first embodiment in that the extraction unit 21 extracts the predetermined part of the moving object M from data related to the video signal. Therefore, the extraction unit 21 extracts the predetermined part, for example, from the shape of the moving object M extracted from the data related to the video signal. Also, for example, the extraction unit 21 extracts the part of the moving object M that moves the most from the data related to the video signal as the predetermined part. The data related to the extracted predetermined part is input to the position identification unit 22.

[0108] <Step S47> This step is similar to step S25 in the second operation of the first embodiment, and is a step in which the next step is changed depending on whether or not a predetermined part of the moving object M has been extracted from the data related to the video signal. If the control device 20 is unable to extract the predetermined part from the data related to the video signal, it returns to step S41, and if the extraction has been successful, it proceeds to step S48. Note that in this operation of the present embodiment, after step S45, the moving object M is photographed from a specific direction. If this photographing is ongoing, the photographing continues.

[0109] <Step S48> This step is a step in which the position identification unit 22 identifies the position of a predetermined part of the moving object M. In this step, similarly to step S27 in the second operation of the first embodiment, for example, the position of the predetermined part of the moving object M is indicated, for example, in a coordinate system. Data relating to the identified position of the predetermined part of the moving object M is output to the instruction unit 24.

[0110] <Step S49> Similar to step S28 in the second operation of the first embodiment, this step is an instruction step of outputting an instruction to the image capturing device 30 to move the position of the identified predetermined part of the moving object M to a position where it can be photographed. Therefore, similar to step S28, in this step, the instructing unit 24 identifies a position where the position of the predetermined part of the moving object M can be photographed, specifies the direction from that position toward the predetermined part, and outputs an instruction to the image capturing device 30 to move to a position where it can be photographed and to orient the image capturing direction toward the predetermined part. Data related to the output instruction is transmitted to the image capturing device 30 via the antenna 29.

[0111] The data relating to the specified position and direction transmitted from the antenna 29 is input to the drive control unit 33 via the antenna 39 of the imaging device 30, and the imaging device 30 moves to a position where it can image the treatment site of the moving object M, and the imaging direction of the imaging device 30 is set to face the predetermined site of the moving object M. Thereafter, the imaging device 30 performs imaging in accordance with instructions from the control device 20 or the imaging control unit 36, similar to the first operation of the first embodiment. Thus, according to this operation, the imaging device 30 can image the predetermined site of the moving object M. The video signal captured by the imaging unit 34 is processed by the image processing unit 35 and output from the antenna 39. The control device 20 then receives the video signal from the antenna 29 and outputs it to the monitor 60. Upon receiving the video signal, the monitor 60 displays the image.

[0112] Third Embodiment Next, a third embodiment of the present invention will be described in detail with reference to Fig. 8. Note that components that are the same as or equivalent to those in the above-described embodiment will be assigned the same reference numerals and redundant description will be omitted unless otherwise specified.

[0113] 8 is a conceptual diagram showing a monitoring system according to a third embodiment of the present invention. As shown in FIG. 8, the monitoring system according to this embodiment differs from the monitoring system 1 according to the above embodiment in that it includes multiple image capturing devices 30 that capture images of a single moving object M.

[0114] In this embodiment, the control device 20 controls each of the imaging devices 30 that capture images of the moving object M to perform either the first operation or the second operation of the first embodiment, or the first operation or the second operation of the second embodiment. Therefore, the control device 20 has a configuration similar to at least one of the first and second embodiments.

[0115] In addition, in this embodiment, the instruction unit 24 of the control device 20 instructs each of the image capturing devices 30 to move the position of the moving object M to a position where the moving object M can be captured from different directions, and to orient the image capturing direction toward the moving object M. Therefore, according to this embodiment, multi-angle images can be obtained.

[0116] As described above, the control device 20 of the present invention is a control device 20 that receives point cloud data of the monitoring area AR output by the LiDAR device 10 and controls an imaging device 30 that can move within the monitoring area AR, and is equipped with an extraction unit 21 that extracts a moving body M from the point cloud data, a position identification unit 22 that identifies the position of the extracted moving body M, and an instruction unit 24 that outputs an instruction to the imaging device 30 to move the position of the moving body M to a position where it can be photographed.

[0117] In addition, the program of the present invention is a program executed by a control device 20 that controls an imaging device 30 that can move within a monitoring area AR based on point cloud data of the monitoring area AR output by the LiDAR device 10, and includes an extraction step that extracts a moving body M from the point cloud data, a position identification step that identifies the position of the extracted moving body M, and an instruction step that outputs an instruction to the imaging device 30 to move the position of the moving body M to a position where it can be photographed.

[0118] In addition, the monitoring system 1 of the present invention comprises a LiDAR device 10, an imaging device 30 that can move within the monitoring area AR of the LiDAR device 10, and a control device 20 that controls the imaging device 30 based on point cloud data of the monitoring area AR output by the LiDAR device 10, and the control device 20 has an extraction unit 21 that extracts a moving body M from the point cloud data, a position identification unit 22 that identifies the position of the extracted moving body M, and an instruction unit 24 that outputs an instruction to the imaging device 30 to move the position of the moving body M to a position where it can be photographed.

[0119] As described above, in the present invention, the control device 20 moves the image capturing device 30 to a position where it can capture an image of the position of the moving object M based on the point cloud data from the LiDAR device 10. Therefore, even if a fixed object F other than the moving object M, which is the object to be captured, is present in the monitoring area AR, the image capturing device 30 can capture an image of the moving object M while avoiding the fixed object F.

[0120] Although the present invention has been described above using the above-mentioned embodiments as examples, the present invention is not limited to these.

[0121] For example, in the above embodiment, an example has been described in which there is one moving body M, but there may be multiple moving bodies M, and an image capturing device 30 may be assigned individually to each moving body M. In this case, the control device 20 controls either the first operation or the second operation of the first embodiment or the second embodiment for the image capturing device 30 that captures each moving body M.

[0122] Furthermore, in the first operation of the first embodiment, it is not essential to identify the predetermined orientation of the moving body M. In this case, the control device 20 does not need to include the orientation identification unit 23, and the process may proceed to step S17 after step S14 in FIG. 4, and steps S15, S16, and S18 may not be performed.

[0123] Furthermore, in the second operation of the first embodiment, it is not essential to identify a predetermined part of the moving object M. In this case, the extraction unit 21 of the control device 20 does not need to extract a predetermined part, and the process proceeds to step S26 after step S23 in Fig. 5, and steps S24, S25, S27, and S28 do not need to be performed. However, if the orientation of the moving object M is not identified in step S26, step S26 may include steps S14 and S17.

[0124] In addition, in the above embodiment, an example is shown in which only one LiDAR device 10 is connected to the control device 20, but multiple LiDAR devices 10 may be connected to the control device 20.

[0125] In the above embodiment, the image capturing device 30 moves along the rails 50, but the image capturing device 30 may be configured differently from the above embodiment as long as it is movable within the monitoring area AR. For example, the image capturing device 30 may be configured to move by itself on the floor of the monitoring area AR.

[0126] In the above embodiment, the imaging device 30 is configured so that the imaging direction can be changed by the imaging direction change unit 32. However, the imaging direction of the imaging device 30 may be fixed. In this case, however, in the first operation of the first embodiment, step S14 is followed by step S17, and in the first operation of the second embodiment, steps S36 to S38 are not performed, and the moving object M is imaged from a fixed direction. Also, in the second operations of the first and second embodiments, a specific part is always imaged from a fixed direction.

[0127] In the above embodiment, the control device 20 and the image capturing device 30 are connected wirelessly. However, the control device 20 and the image capturing device 30 may be connected by a wire such as a signal line.

[0128] As described above, according to the present invention, a control device, a program, and a monitoring system are provided that can realize a monitoring system that can photograph a moving object even if there are objects other than the moving object being photographed in the monitoring area, and can be used in fields such as monitoring.

Claims

1. A control device that receives point cloud data of a monitoring area output by a LiDAR device and controls an imaging device that can move within the monitoring area, comprising: an extraction unit that extracts a moving object from the point cloud data; a position identification unit that identifies the position of the extracted moving object; and an instruction unit that outputs an instruction to the imaging device to move the position of the moving object to a position where it can be photographed.

2. The control device according to claim 1, further comprising an orientation identification unit that identifies a specific orientation of the extracted moving body, wherein the instruction unit outputs an instruction to the imaging device, which is movable within the monitoring area and capable of changing the imaging direction, to move from the position of the moving body to a position moved toward the specific orientation, and outputs an instruction to turn the imaging direction toward the position of the moving body.

3. The control device according to claim 2, characterized in that the orientation identification unit identifies the predetermined orientation from the point cloud data.

4. The control device according to claim 3, characterized in that the direction identification unit determines the direction of movement of the moving object to be the predetermined direction.

5. The control device according to claim 2, characterized in that data relating to a video signal from the imaging device is input to the orientation determination unit, and the orientation determination unit determines the specified orientation from the data relating to the video signal.

6. The control device according to claim 1, characterized in that the extraction unit extracts a specified part of the moving object, the position identification unit identifies the position of the specified part, and the instruction unit outputs an instruction to the imaging device to move the position of the specified part to a position where it can be photographed.

7. The control device according to claim 6, characterized in that the extraction unit extracts the specified portion from the point cloud data.

8. The control device according to claim 7, characterized in that the extraction unit determines the part of the moving object that moves the most as the predetermined part.

9. The control device according to claim 6, characterized in that the extraction unit receives data related to a video signal from the imaging device, and the extraction unit extracts the specified portion from the data related to the video signal.

10. The control device according to claim 1, characterized in that the instruction unit outputs instructions to the multiple imaging devices to move the position of the identified moving object to positions where it can be photographed from different directions.

11. A program executed by a control device that controls an imaging device capable of moving within a monitoring area based on point cloud data of the monitoring area output by a LiDAR device, the program comprising: an extraction step of extracting a moving object from the point cloud data; a position identification step of identifying the position of the extracted moving object; and an instruction step of outputting an instruction to the imaging device to move the position of the moving object to a position where it can be photographed.

12. A surveillance system comprising: a LiDAR device; an imaging device capable of moving within the surveillance area of ​​the LiDAR device; and a control device that controls the imaging device based on point cloud data of the surveillance area output by the LiDAR device, wherein the control device has an extraction unit that extracts a moving object from the point cloud data, a position identification unit that identifies the position of the extracted moving object, and an instruction unit that outputs an instruction to the imaging device to move the position of the moving object to a position where it can be photographed.

Images