Mobile body system, mobile body, and person region detection method

Abstract

To efficiently set a region which may be falsely detected as a person, as a mask region, thereby reducing false detection of persons.SOLUTION: A mobile body system transmits, to a mobile body control apparatus 20, image data captured by a mobile body 30a and person candidate region data in which a person has been detected. The mobile body control apparatus 20 acquires a false detection region designated by a monitoring person, as mask region data. The mobile body control apparatus 20 transmits the mask region data to the mobile body 30a. The mobile body 30a calculates latitude-longitude real-space coordinates and a mask time which is the time in which the mask region data is effective, from the received mask region data, and detects a person in regions other than the mask region, thereby efficiently setting a region which may be detected as a person, as a mask region, resulting in reduction of false detection of persons.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a mobile system, a mobile body, and a human region detection method that can efficiently set a region that may be erroneously detected as a person as a mask region, thereby reducing erroneous detection of a person. [Background technology]

[0002] Conventionally, there is known a technology for evacuating people remaining in disaster areas such as rivers by using a mobile object such as a drone (for example, see Patent Document 1). In this case, it is necessary to automatically analyze images captured by an imaging device mounted on the mobile object to detect the remaining people, but there is a risk that trees, rocks, signs, etc. may be erroneously detected as people, which may hinder the evacuation guidance of people.

[0003] In order to eliminate objects that are considered to be the cause of erroneous detection, a technique is known in which an area including these objects is registered in advance as a mask area, and the area is excluded from targets for human detection (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2022-191772 A [Patent Document 2] Patent No. 7042911 Summary of the Invention [Problem to be solved by the invention]

[0005] However, when an area including an object that is assumed to be a cause of erroneous detection is registered as a mask area based on an image acquired in advance, an image of the flight area needs to be acquired in advance. Also, for example, if an area where trees grow closely together is set as a mask area, a person among the trees cannot be detected, which is a major problem for a system that aims to guide people in disaster areas to evacuate. Furthermore, since a large environmental change occurs when a disaster such as a typhoon, flooding of a river, etc. occurs, there is a problem that a mask area registered in advance cannot adequately respond to erroneous detection.

[0006] The present invention has been made to solve the problems (issues) associated with the above-mentioned conventional technology, and aims to provide a mobile body system, a mobile body, and a human area detection method that can efficiently set areas that may be erroneously detected as people as mask areas, thereby reducing erroneous detection of people. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the object, the present invention provides a mobile body system having a mobile body equipped with an imaging device that detects a person from a captured image and transmits the person as a candidate person area, and a mobile body control device that controls the movement of the mobile body, wherein the mobile body control device sets a candidate person area that does not show a person from among the candidate person areas transmitted from the mobile body as a mask area, and when the mask area is set by the mobile body control device, the mobile body detects people in the captured image excluding the mask area.

[0008] In the present invention, in the above-mentioned invention, the mobile body control device has an input unit, and sets the person candidate area transmitted from the mobile body based on an input from the input unit as the mask area.

[0009] Further, in the present invention, in the above-mentioned invention, the mobile body control device sets in advance an effective time for which a mask area is effective, and when a mask area is set by the mobile body control device, the mobile body detects people in the captured image excluding the mask area during the effective time.

[0010] In addition, in the present invention, in the above invention, the mobile body control device sets an effective time for which a mask area is effective based on input from the input unit, and when a mask area is set by the mobile body control device, the mobile body detects people in the captured image excluding the mask area during the effective time.

[0011] In the present invention, when the moving object detects that the mask area has deviated from the angle of view of the imaging device, the moving object cancels the mask area.

[0012] In the present invention, when the moving object detects that a person area in which a person is detected has moved into the mask area, the moving object cancels the mask area.

[0013] The present invention also relates to a moving body equipped with an imaging device that detects people from an image captured by the imaging device, and a person candidate area in which a person is detected from the image captured by the imaging device and in which no person is captured is set as a mask area, and after the mask area is set, people are detected in the image excluding the mask area.

[0014] The present invention also provides a person area detection method in a mobile body system having a mobile body equipped with an imaging device and a mobile body control device that controls the movement of the mobile body, the method including the steps of: detecting a person from an image captured by the imaging device as a person candidate area; setting a person candidate area that does not show a person among the person candidate areas as a mask area; and, once the mask area has been set, detecting people in the captured image excluding the mask area. Effect of the Invention

[0015] According to the present invention, an area that may be erroneously detected as a person can be efficiently set as a mask area, thereby reducing erroneous detection of a person. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram showing an overview of a mobile system according to a first embodiment. [Diagram 2] FIG. 2 is a diagram showing a system configuration of the mobile system according to the first embodiment. [Diagram 3] FIG. 3 is a functional block diagram showing the configuration of the mobile object control device shown in FIG. [Figure 4] FIG. 4 is a functional block diagram showing a configuration of the moving object shown in FIG. [Diagram 5] FIG. 5 is a sequence diagram showing a processing procedure of the mobile system shown in FIG. [Figure 6] FIG. 6 is a diagram showing an overview of the first modification. [Figure 7] FIG. 7 is a diagram showing an overview of the second modification. [Figure 8] FIG. 8 is a diagram illustrating an overview of a mobile body system according to the second embodiment. [Figure 9] FIG. 9 is a diagram showing an example of a captured image captured by a moving object. [Figure 10] FIG. 10 is a functional block diagram showing the configuration of the mobile object control device shown in FIG. [Figure 11] FIG. 11 is a functional block diagram showing the configuration of the moving body shown in FIG. [Figure 12] FIG. 12 is a functional block diagram showing the configuration of a moving object control device according to the third embodiment. [Figure 13] FIG. 13 is a functional block diagram showing the configuration of a moving object according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of a mobile body system, a mobile body, and a human region detection method according to the present invention will be described in detail with reference to the accompanying drawings.

[0018] [Embodiment 1] <Overview of mobile systems> An overview of the mobile body system according to the first embodiment will be described. FIG. 1 is a diagram showing an overview of the mobile body system according to the first embodiment. The mobile body system according to the first embodiment includes a mobile body 30a and a mobile body control device 20. First, the mobile body 30a transmits to the mobile body control device 20 captured image data including the captured image and position information of the mobile body 30a, and person candidate area data indicating an area detected as a person. The mobile body control device 20 displays the captured image and person candidate areas G1 and G2 indicated by the person candidate area data included in the received captured image data. When a person candidate area G1 in which no person is captured is specified by an observer who has confirmed the display contents, the mobile body control device 20 sets the person candidate area G1 specified by the observer as a mask area M, and transmits mask area data including a valid time for validating the mask area M to the mobile body 30a. The mobile body 30a calculates the real space coordinates of the moving longitude and the mask time, which is the valid time of the mask area data, based on the received mask area data, and identifies a person area in which a person exists in an area other than the mask area M during the valid time. This allows areas that may be erroneously detected as people to be efficiently set as mask areas, thereby reducing erroneous detection of people. Here, the explanation will be centered on the case where an unmanned flying object called a drone is used as the moving object.

[0019] The moving body 30a (drone) of the present embodiment 1 is an unmanned flying body that can fly autonomously under the control of the moving body control device 20. In other words, the movement of the moving body 30a is not controlled by the operation of an operator, but is controlled autonomously in response to an instruction from the moving body control device 20. In addition, although described in detail later, the moving body 30a of the present embodiment 1 includes a means for acquiring a captured image of its surroundings and a means for detecting a person based on the captured image, and transmits the acquired captured image and person candidate area data to the moving body control device 20.

[0020] As shown in Fig. 1, a mobile object 30a of a mobile object system according to this embodiment captures an image of a disaster area (S1). The mobile object 30a detects people in the captured image (S2). At this time, the mobile object 30a may detect an object other than a person, such as a rock, as a person.

[0021] When the moving body 30a detects a person, it transmits captured image data and person candidate area data to the moving body control device 20 (S3). When the moving body control device 20 receives the captured image data and person candidate area data from the moving body 30a, it displays the person candidate areas G1 and G2 indicated by the captured image and person candidate area data included in the captured image data on a predetermined display unit. Then, when a monitor who has checked the displayed contents specifies a person candidate area G1 in which no person is photographed, the moving body control device 20 sets the person candidate area G1 specified by the monitor as a mask area M (S4). In addition, the effective time for which the mask area M is effective, which is input by the monitor via the input unit 22, is acquired, and set as the effective time for the mask area M in the image data.

[0022] The mobile body control device 20 transmits mask area data including the mask area M and the effective time to the mobile body 30a (S5). When the mobile body 30a receives the mask area data, it sets the real space coordinates including the latitude and longitude of the mask area M and the mask time for setting the mask area based on the mask area data (S6). Based on the real space coordinates of the mask area M and the effective time, the mobile body 30a specifies the person candidate area detected in the area excluding the mask area M in the captured image as a person area where a person exists during the effective time (S7), and performs evacuation guidance, etc.

[0023] <Configuration of Mobile System 10> Next, a system configuration of the mobile body system 10 will be described. Fig. 2 is a diagram showing the system configuration of the mobile body system 10 according to embodiment 1. As shown in Fig. 2, in the mobile body system 10, a mobile body control device 20 and a plurality of mobile bodies 30a, 30b, and 30c (hereinafter, may be collectively referred to as "mobile bodies 30") are connected to a network N.

[0024] The mobile body control device 20 performs the following processes: (1) receiving captured image data and person candidate area data from the mobile body 30 and displaying them on a predetermined display unit, (2) setting the person candidate area designated by the monitor who confirmed the display content as a mask area M and setting the effective time for which the mask area M is effective, and (3) transmitting the mask area data including the mask area M and the effective time to the mobile body 30. The captured image data and mask area data include data on the latitude, longitude, and altitude at the time the image was captured.

[0025] While flying over a disaster area or the like according to a set flight route, the mobile body 30 performs the following processes: (1) capturing images of the surroundings using an imaging unit and acquiring the captured images; (2) detecting a person based on the captured images; (3) acquiring latitude and longitude data from a GNSS (Global Navigation Satellite System) unit; (4) transmitting the captured image, captured image data including the latitude, longitude, altitude, and imaging direction of the mobile body 30, and person candidate area data indicating an area detected as a person (person candidate area) to the mobile body control device 20; (5) receiving mask area data from the mobile body control device 20; (6) calculating the real space coordinates of the mask area M; and (7) identifying the person area in which a person exists in the imaging area excluding the mask area M during the valid time.

[0026] <Configuration of the mobile object control device 20> Next, a description will be given of the configuration of the mobile object control device 20 shown in Fig. 2. Fig. 3 is a functional block diagram showing the configuration of the mobile object control device 20 shown in Fig. 2. As shown in Fig. 3, the mobile object control device 20 has a display unit 21, an input unit 22, a wireless communication unit 23, a storage unit 24, and a control unit 25.

[0027] The display unit 21 is a display device such as a liquid crystal panel or a display device, and the input unit 22 is an input device such as a keyboard, a mouse, etc. The wireless communication unit 23 is an interface unit for wireless communication with the mobile object 30.

[0028] The storage unit 24 is a storage device such as a hard disk drive or a non-volatile memory, and stores mask area data 24a. The mask area data 24a is data including a mask area that is a person candidate area designated by a monitor as a person not captured among the person candidate areas received from the moving object 30, and a valid time for validating the mask area. The mask area data 24a includes data on the latitude and longitude, altitude, and imaging direction of the moving object 30 at the time of image capture.

[0029] The control unit 25 is a control unit that controls the entire mobile object control device 20, and has a captured image display unit 25a, a mask area setting unit 25b, and a mask area data transmission unit 25c. In practice, by loading these programs into a CPU and executing them, the captured image display unit 25a, the mask area setting unit 25b, and the mask area data transmission unit 25c execute processes corresponding to them, respectively.

[0030] The captured image display unit 25a performs a process of displaying the captured image data and the person candidate area data received from the moving object 30 on a predetermined display unit 21. Specifically, the captured image display unit 25a performs a process of displaying and controlling the person candidate area in which the moving object 30 has performed person detection on the captured image received from the moving object 30, for example, by using a rectangular frame.

[0031] When the observer specifies an area (mask area M) in which no person is captured among the captured image and candidate person areas displayed on the display unit 21 via the input unit 22, the mask area setting unit 25b acquires data on the mask area M and stores it in the memory unit 24 as mask area data 24a together with the latitude, longitude, altitude, and imaging direction data contained in the captured image data.

[0032] The mask region data transmission unit 25 c reads out the mask region data 24 a stored in the storage unit 24 and transmits it to the mobile object 30 .

[0033] <Configuration of moving body 30a> Next, the configuration of the moving body 30a shown in Fig. 2 will be described. Fig. 4 is a functional block diagram showing the configuration of the moving body 30a shown in Fig. 2. The moving bodies 30b and 30c also have a similar configuration. As shown in Fig. 4, the moving body 30a has an imaging unit 31, an altitude sensor 32, a GNSS unit 33, a wireless communication unit 34, a rotor 35, a storage unit 36, and a control unit 37.

[0034] The imaging unit 31 is an imaging device that captures images of the situation in the flight area using a camera. The camera mounted on the imaging unit 31 uses a device such as a CCD (Charge Coupled Device) that converts an image into an electrical signal and captures the image.

[0035] The altitude sensor 32 is a sensor that measures the altitude of the moving body 30a. For example, a barometric pressure sensor can be used as the altitude sensor 32, and a barometric pressure sensor and an ultrasonic sensor can also be combined. This allows the moving body 30a to stably maintain its altitude in a location close to the ground. The GNSS unit 33 is a unit that receives signals from multiple GNSS satellites and identifies the position coordinates of the own device.

[0036] The wireless communication unit 34 is an interface unit for wireless communication with the moving body control device 20. The multiple rotors 35 rotate individually under the control of a movement control unit (not shown), enabling attitude control and movement of the moving body 30a.

[0037] The storage unit 36 ​​is a storage device such as a hard disk drive or a non-volatile memory, and stores a trained model for human detection 36a, captured image data 36b, human candidate area data 36c, real space mask area data 36d, and mask time data 36e. The trained model for human detection 36a is a trained model obtained by deep learning. When using such deep learning, a trained model is generated in advance by performing supervised learning using an image containing a person and the result as teacher data in a server device (not shown), and downloaded to the mobile body 30a. Machine learning other than deep learning can also be used. In addition, to detect a person, a pattern matching process using a template of a person may be performed on the captured image.

[0038] The captured image data 36b is image data captured by the moving body 30a using the imaging unit 31. The captured image data 36b includes latitude, longitude, and altitude acquired by the latitude and longitude processing unit 37c described later, and data on the imaging direction when the captured image was captured. The person candidate area data 36c is data indicating an area detected as a person by detecting a person included in the captured image of the captured image data 36b using the person detection trained model 36a. The real space mask area data 36d is data calculated by calculating the real space coordinates of the actual latitude and longitude from the relative coordinates in the captured image of the mask area data 24a received from the moving body control device 20 and the data of the latitude, longitude, altitude, and imaging direction included in the mask area data 24a. The mask time data 36e is data included in the mask area data 24a and indicates a time for processing the detection data of the area set as the mask area not to be detected using the real space mask area data 36d. Note that the effective time will be described taking as an example a case where the monitor inputs and sets the effective time when setting the mask area, but is not limited to this, and may be set in advance.

[0039] The control unit 37 is a control unit that controls the entire moving body 30a, and includes an imaging control unit 37a, a human detection unit 37b, a latitude and longitude processing unit 37c, an image data transmission unit 37d, a mask area data receiving unit 37e, a real space coordinate calculation unit 37f, and a human area identification unit 37g. In practice, these programs are loaded into a CPU and executed, causing the imaging control unit 37a, the human detection unit 37b, the latitude and longitude processing unit 37c, the image data transmission unit 37d, the mask area data receiving unit 37e, the real space coordinate calculation unit 37f, and the human area identification unit 37g to execute processes corresponding thereto.

[0040] The imaging control unit 37a causes the imaging unit 31 to capture an image of the disaster area, and stores the obtained captured image as captured image data 36b in the storage unit 36. The human detection unit 37b detects a person included in the captured image of the captured image data 36b using the trained model for human detection 36a, and performs processing to store the area of ​​the detected person in the storage unit 36 ​​as human candidate area data 36c.

[0041] The latitude and longitude processing unit 37c acquires altitude data of the moving body 30a from the altitude sensor 32, and acquires latitude and longitude data about the location at which the moving body 30a is flying from the GNSS unit 33, and performs processing to store the acquired data in the memory unit 36 ​​as attribute data of the captured image data 36b.

[0042] The image data transmission unit 37d reads out the captured image data 36b and the person candidate area data 36c, and performs processing to transmit them to the mobile object control device 20. The mask area data reception unit 37e performs processing to receive the mask area data 24a transmitted from the mobile object control device 20.

[0043] The real space coordinate calculation unit 37f calculates the latitude and longitude coordinates of the real space of the mask area from the relative coordinates in the captured image of the mask area M of the mask area data 24a received from the mobile body control device 20 and the data of latitude, longitude, altitude, and imaging direction included in the mask area data 24a, and performs processing to store the calculated coordinates as real space mask area data 36d in the storage unit 36, and also performs processing to set an effective time for which the mask area M is effective and store the calculated coordinates as mask time data 36e in the storage unit 36. The latitude and longitude coordinates of the real space are calculated by using trigonometric functions to calculate the latitude and longitude of a specific part of the image from the data of latitude, longitude, and altitude at the time of image capture included in the mask area data 24a and the installation angle data of the imaging unit arranged on the mobile body 30, and then calculates the real space coordinates from the relative coordinates in the captured image.

[0044] The person region identifying unit 37g compares the person candidate region data 36c detected by the person detection unit 37b based on the captured image captured by the moving body 30a with the real space mask region data 36d, and performs a process of identifying an area in which a person is detected as a person region, in an area excluding the real space mask area in the captured image. Note that the person region identifying unit 37g performs the process of identifying the person region for the area excluding the real space mask area, within the time set by the mask time data 36e.

[0045] <Processing Procedure of the Mobile System 10> Next, the processing procedure of the mobile body system 10 will be described. Fig. 5 is a sequence diagram showing the processing procedure of the mobile body system 10 shown in Fig. 2. As shown in Fig. 5, the mobile body 30a captures an image of the disaster area while flying over the disaster area (step S101), and stores the captured image data together with altitude data, latitude and longitude data, and imaging direction information acquired by the altitude sensor 32 and the GNSS unit 33 (step S102). Then, the mobile body 30a detects a person based on the captured image (step S103), and when a person is detected, transmits captured image data 36b including the altitude data and latitude and longitude data and person candidate area data 36c to the mobile body control device 20 (step S104).

[0046] When the mobile object control device 20 receives the captured image data 36b and the person candidate area data 36c from the mobile object 30a, the mobile object control device 20 stores the captured image data 36b and the person candidate area data 36c in the storage unit 36, and displays the received captured image and person candidate area on the display unit 21 (step S105). Then, when the monitor designates a person candidate area as an area in which no person is shown among the person candidate areas displayed on the display unit 21, the mobile object control device 20 sets the person candidate area as a mask area and stores it together with the effective time set by the monitor (step S106). After that, the mobile object control device 20 transmits the mask area data 24a including the altitude data, latitude and longitude data, and effective time to the mobile object 30a (step S107).

[0047] When the moving body 30a receives the mask region data 24a from the moving body control device 20 (step S107), the moving body 30a calculates the real space coordinates of the mask region data 24a based on the altitude data and latitude and longitude data included in the mask region data 24a (step S109), and sets the effective time of the mask region data 24a (step S110). Then, during the set effective time, if the human detection unit 37b detects a human in an area excluding the mask region in real space, the moving body 30a specifies the area as a human area (step S111).

[0048] As described above, in this embodiment 1, the mobile body system 10 transmits the captured image data 36b captured by the mobile body 30a and the candidate human area data 36c in which human detection has been performed to the mobile body control device 20, and the mobile body control device 20 sets the erroneous detection area specified by the monitor as mask area data 24a, and the mobile body control device 20 transmits the mask area data 24a to the mobile body 30a, and the mobile body 30a calculates the real space coordinates of the moving longitude from the received mask area data 24a and performs human detection in the area excluding the mask area M during the set effective time, so that the area that may be erroneously detected as a person can be efficiently set as the mask area M, thereby reducing erroneous detection of people.

[0049] <Variation 1> In the above embodiment 1, the case where the effective time of the mask area M of the moving body 30a is set by an observer is described. However, in variant example 1, a case where the setting of the mask area M is cancelled when the imaging area of ​​the moving body 30a leaves the range of the already set mask area data 24a is described.

[0050] Fig. 6 is a diagram showing an overview of Modification 1. As shown in Fig. 6(a), when a mask area M is set in an imaging area that is an area captured by the imaging unit 31, the moving body 30a invalidates the detection of a person even if the moving body 30a detects a person in the mask area M. Moreover, in the moving body system 10 according to the first embodiment, the moving body 30a sets a valid time for the mask area M, and releases the mask area M when the valid time has elapsed.

[0051] On the other hand, in the mobile body system 10 according to the first modification, as shown in FIG. 6(b), when the mobile body 30a moves along the flight path and the mask area M is no longer included in the imaging area, the mobile body 30a automatically cancels the set mask area M. In this way, it becomes unnecessary for the observer to set the effective time, and the burden on the observer can be reduced. In addition, the mask area is not set for an unnecessarily long time, and even if a person moves into an area that was set as a mask area, the person can be found reliably.

[0052] <Variation 2> Next, a description will be given of Modification 2. Fig. 7 is a diagram showing an overview of Modification 2. As shown in Fig. 7(a), a mask area M is set for a moving body 30a based on mask area data 24a transmitted from the moving body control device 20. Here, as shown in Fig. 7(b), if an already detected person enters the range of the mask area M, the detection of the person becomes invalid.

[0053] In the mobile body system 10 according to the second modification, when an already detected person enters the range of the mask area M, the mask area M is released. Then, as shown in Fig. 7(c), even if the already detected person leaves the released mask area M, the release of the mask area M is continued. Note that, when the detected person moves away from the released mask area M, the mask area M may be set again.

[0054] <Variation 3> Next, a description will be given of Modification 3. In the mobile body system 10 according to Modification 3, the mask area M of the mobile body 30a is set based on the mask area data 24a transmitted from the mobile body control device 20, but the human detection unit 37b of the mobile body 30a may automatically cancel the mask area M when the detection probability of human detection is equal to or higher than a predetermined threshold (e.g., 0.8 or higher) in a plurality of consecutive frames (e.g., 10 frames) within the range of the mask area M.

[0055] In the above embodiment 1, the case where the monitor sets the mask area has been described, but the present invention is not limited to this. The moving body 30 may determine whether a person is detected or not based on the likelihood of human detection when detecting a person. For example, when the human detection unit 37b of the moving body 30 detects a person included in a captured image using the trained model 36a for human detection, if the probability of human detection is 0.8 or more, it may be determined that a person has been detected, and if the probability of human detection is 0.3 or less, it may be determined that a person has been detected. In addition, it may be determined that the object is a block, tree, signboard, or the like that may cause a false detection, not a person, and if the probability of this determination is a certain level or more, it may be determined that the object is a possible cause of a false detection, and set as a mask area.

[0056] [Embodiment 2] In the above embodiment 1, we have described a case in which the captured image data 36b and candidate person area data 36c are transmitted from the moving body 30a to the moving body control device 20, and the area specified by the monitor is set as the mask area M in the moving body control device 20. However, in the mobile body system of embodiment 2, we will describe a case in which object detection area data is received from multiple moving bodies, an object detection area is identified from the multiple object detection area data, and the mask area M is set.

[0057] FIG. 8 is a diagram showing an overview of a mobile body system according to the second embodiment. The mobile body system according to the second embodiment includes a mobile body control device 40 and mobile bodies 50a, 50b, and 50c (hereinafter, may be collectively referred to as "mobile bodies 50"). As shown in FIG. 8, the mobile body 50a transmits object detection area data (see FIG. 9(a)) including real space coordinates to the mobile body control device 40 (S11). The mobile body 50b also transmits object detection area data (see FIG. 9(b)) including real space coordinates to the mobile body control device 40 (S12). Furthermore, the mobile body 50c also transmits object detection area data (see FIG. 9(c)) including real space coordinates to the mobile body control device 40 (S13).

[0058] The mobile object control device 40 sets a mask area M based on the object detection area data received from the multiple mobile objects 50a, 50b, and 50c (S14). Here, as shown in FIG. 9, the object detection area data transmitted from the mobile objects 50a and 50b includes the detected object detection area B, but the object detection area data transmitted from the mobile object 50c does not include the object detection area B. If the object detection area B is at a close latitude and longitude and the number of mobile objects that detect an object is greater than the number of mobile objects that do not detect an object, the mobile object control device 40 sets the object detection area B transmitted from the mobile objects 50a and 50b as the mask area M. Then, the mobile object control device 40 transmits the mask area data to the mobile objects 50a, 50b, and 50c, respectively (S15, S16, and S17).

[0059] <Configuration of the mobile object control device 40> Next, a configuration of a mobile body control device 40 of a mobile body system according to embodiment 2 will be described. Note that the same functional units as those of the mobile body control device 20 shown in Fig. 3 are given the same reference numerals, and detailed description thereof will be omitted. As shown in Fig. 10, the mobile body control device 40 has a display unit 21, an input unit 22, a wireless communication unit 23, a storage unit 44, and a control unit 45.

[0060] The storage unit 44 is a storage device such as a hard disk drive or a non-volatile memory, and stores object detection area data 44a and mask area data 44b. The object detection area data 44a is data of the object detection area B including real space coordinates received from multiple moving objects 50. The mask area data 44b is data in which the object detection area B detected in the multiple moving objects 50 is set as a mask area M.

[0061] The control unit 45 is a control unit that controls the entire mobile object control device 40, and has a mask region data transmission unit 25c, an object detection region data processing unit 45a, and an erroneous detection determination unit 45b. In practice, these programs are loaded into a CPU and executed, causing the mask region data transmission unit 25c, the object detection region data processing unit 45a, and the erroneous detection determination unit 45b to execute processes corresponding thereto.

[0062] Object detection area data processing unit 45a performs a process of acquiring object detection area data detected by each of the multiple moving bodies 50, and storing the data as object detection area data 44a in storage unit 44. False detection determination unit 45b reads out object detection area data 44a stored in storage unit 44, and, when an object detection area B exists due to the multiple moving bodies 50 in a predetermined area (an area close in real space coordinates), determines that object detection area B is a mask area M, and stores the object detection area data 44b in storage unit 44.

[0063] <Configuration of moving body 50a> Next, the configuration of a moving body 50a of the moving body system according to the second embodiment will be described. The moving bodies 50b and 50c have the same configuration. The same reference numerals are given to the same functional parts as the moving body 30a shown in FIG. 4, and detailed description thereof will be omitted. As shown in FIG. 11, the moving body 50a has an imaging unit 31, an altitude sensor 32, a GNSS unit 33, a wireless communication unit 34, a rotor 35, a storage unit 56, and a control unit 57.

[0064] The storage unit 56 is a storage device such as a hard disk device or a non-volatile memory, and stores a trained model for human detection 36a, captured image data 36b, human candidate area data 36c, real space mask area data 36d, mask time data 36e, a trained model for object detection 56a, and object detection area data 56b. The trained model for object detection 56a is a trained model obtained by deep learning. When using such deep learning, a trained model is generated in advance by performing supervised learning using images including objects that may be mistakenly detected as humans, such as blocks, trees, and signs, and the results as teacher data in a server device (not shown), and the trained model is downloaded to the mobile body 50a. Machine learning other than deep learning can also be used. Note that, for the detection of a person, a pattern matching process using a template of an object may be performed on the captured image.

[0065] The object detection area data 56b is data of a detection area including real space coordinates in which an object is detected by an object detection unit 57a (to be described later) based on the captured image data 36b.

[0066] The control unit 57 is a control unit that controls the entire moving body 50a, and includes an imaging control unit 37a, a person detection unit 37b, a latitude and longitude processing unit 37c, an image data transmission unit 37d, a mask region data receiving unit 37e, a real space coordinate calculation unit 37f, a person region identification unit 37g, an object detection unit 57a, and an object detection region data transmission unit 57b. In practice, these programs are loaded into a CPU and executed, causing the imaging control unit 37a, the person detection unit 37b, the latitude and longitude processing unit 37c, the image data transmission unit 37d, the mask region data receiving unit 37e, the real space coordinate calculation unit 37f, the person region identification unit 37g, the object detection unit 57a, and the object detection region data transmission unit 57b to execute processes corresponding thereto.

[0067] The object detection unit 57a is a processing unit that detects objects included in an image captured by the moving body 50a using the trained model for object detection 56a, and identifies an object detection area including the real space coordinates of the detected object, and performs a process of storing the characterized area in the memory unit 56 as object detection area data 56b.

[0068] The object detection area data transmission unit 57b reads out the object detection area data 56b stored in the storage unit 56 and transmits it to the mobile body control device 40.

[0069] As described above, in the second embodiment, the mobile body control device 40 of the mobile body system receives, from the multiple mobile bodies 50, object detection region data 56b detected in each of the multiple mobile bodies 50, and specifies the mask region M based on the object detection region data 56b. Then, the mobile body control device 40 transmits the mask region data 44b to the multiple mobile bodies 50. Since the mobile body 50 performs human detection in a region other than the mask region M based on the mask region data 44b, a region that may be erroneously detected as a human can be efficiently set as a mask region, thereby reducing erroneous detection of a human.

[0070] [Embodiment 3] Incidentally, in the above-mentioned embodiment 1, the case where the moving body 30 calculates the real space coordinates based on the mask area data 24a received from the moving body control device 20 has been described, but the calculations can also be performed in the moving body control device 60. For this reason, in a moving body system according to embodiment 3, a case will be described where the captured image data 36b captured by the moving body 30a and the human candidate area data 36c obtained by human detection are transmitted to the moving body control device 60, the mask area designated by the monitor is set as the mask area data 24a in the moving body control device 60, and further the real space coordinates of latitude and longitude and the mask time which is the effective time of the mask area data 24a are calculated from the mask area data 24a.

[0071] <Configuration of the mobile object control device 60> Fig. 12 is a functional block diagram showing the configuration of a mobile object control device 60 according to the third embodiment. The same components as those in the first embodiment are given the same reference numerals, and detailed description thereof will be omitted. As shown in Fig. 12, the mobile object control device 60 according to the third embodiment has a display unit 21, an input unit 22, a wireless communication unit 23, a storage unit 64, and a control unit 65.

[0072] The storage unit 64 is a storage device such as a hard disk drive or a non-volatile memory, and stores the mask area data 24a, the real space mask area data 64a, and the mask time data 64b. The real space mask area data 64a is data calculated based on the latitude and longitude received from the moving body 70 and the altitude data of the moving body 70, and is the coordinates of the real space of the mask area set by the mask area setting unit 25b. The mask time data 64b is data of the time for processing the detection data of the erroneously detected area using the real space mask area data 64a so as not to detect it. The mask time data 64b is set by the observer.

[0073] The control unit 65 is a control unit that controls the entire mobile object control device 60, and has a captured image display unit 25a, a mask area setting unit 25b, a real space position calculation unit 65a, and a real space mask area data transmission unit 65b. In practice, by loading these programs into a CPU and executing them, the captured image display unit 25a, the mask area setting unit 25b, the real space position calculation unit 65a, and the real space mask area data transmission unit 65b are caused to execute processes corresponding to them, respectively.

[0074] The real space position calculation unit 65a calculates the coordinates of the moving longitude, which is the real space, from the relative coordinates in the captured image of the mask area M of the mask area data 24a set by the mask area setting unit 25b, performs processing to store it in the storage unit 64 as real space mask area data 64a, and performs processing to store it in the storage unit 64 as mask time data 64b. The latitude and longitude coordinates, which are the real space, are calculated by using trigonometric functions to calculate the latitude and longitude of a specific part of the image from the latitude, longitude and altitude data at the time of image capture included in the mask area data 24a and the imaging direction data of the imaging unit 31 received from the moving object 70, and then calculates the real space coordinates from the relative coordinates in the captured image.

[0075] The real space mask area data transmission unit 65b reads out the real space mask area data 64a and the mask time data 64b stored in the storage unit 64, and transmits them to the moving body .

[0076] <Configuration of moving body 70> Next, the configuration of a moving body 70 according to the third embodiment will be described. Fig. 13 is a functional block diagram showing the configuration of a moving body according to the third embodiment. Note that the same reference numerals are used for the same parts as those in the first embodiment, and detailed description thereof will be omitted. As shown in Fig. 13, the moving body 70 has an imaging unit 31, an altitude sensor 32, a GNSS unit 33, a wireless communication unit 34, a rotor 35, a storage unit 76, and a control unit 77.

[0077] The storage unit 76 is a storage device such as a hard disk drive or a non-volatile memory, and stores the trained model for human detection 36a, the captured image data 36b, the human candidate region data 36c, the real space mask region data 76a, and the mask time data 76b. The real space mask region data 76a is data obtained by calculating the real space coordinates of the actual latitude and longitude from the relative coordinates in the captured image of the mask region data 24a received from the mobile body control device 60. The mask time data 76b is data indicating the time when human detection is performed in the region of the captured image excluding the mask region M, using the real space mask region data 76a.

[0078] The control unit 77 is a control unit that controls the entire moving body 70, and has an imaging control unit 37a, a person detection unit 37b, a latitude and longitude processing unit 37c, a person area identification unit 37g, an image data transmission unit 77a, and a real space mask area data receiving unit 77b. In practice, these programs are loaded into a CPU and executed, causing the imaging control unit 37a, the person detection unit 37b, the latitude and longitude processing unit 37c, the person area identification unit 37g, the image data transmission unit 77a, and the real space mask area data receiving unit 77b to execute processes corresponding thereto.

[0079] The image data transmission unit 77a reads out the captured image data 36b and the person candidate area data 36c, and performs processing to transmit the captured image data 36b and the person candidate area data 36c to the mobile body control device 60. The real space mask area data receiving unit 77b performs processing to receive the real space mask area data 64a transmitted from the mobile body control device 60.

[0080] As described above, in the third embodiment, the mobile body system transmits the captured image data 36b captured by the mobile body 70 and the human candidate area data 36c in which human detection has been performed to the mobile body control device 60, and the mobile body control device 60 sets the erroneous detection area designated by the monitor as the mask area data 24a. The mobile body control device 60 then calculates the real space coordinates of latitude and longitude from the mask area data 24a. The mobile body 70 receives the real space mask area data 64a and mask time data 64b from the mobile body control device 60, and performs human detection in an area excluding the mask area M, so that the area that may be erroneously detected as a human can be efficiently set as the mask area M, thereby reducing erroneous detection of a human.

[0081] <Modification 4> In the above embodiment 3, the case where the effective time of the mask area M of the moving body 70 is set by an observer is described. However, in variant example 4, a case where the setting of the mask area M is cancelled when the imaging area of ​​the moving body 70 leaves the range of the already set real space mask area data 76a is described.

[0082] When a mask area M is set within an imaging area that is an area captured by the imaging unit 31, the moving body 70 invalidates the detection of a person even if the moving body 70 detects a person within this mask area M. Furthermore, in the moving body system according to the third embodiment, the moving body 70 sets a valid time for this mask area M, and releases the mask area M when the valid time has elapsed.

[0083] On the other hand, in the moving body system according to the fourth modification, when the moving body 70 moves along the flight path and the mask area M is no longer included in the imaging area, the moving body 70 automatically cancels the set mask area M. In this way, it becomes unnecessary for the observer to set the effective time, and the burden on the observer can be reduced. In addition, the mask area is not set for an unnecessarily long time, and even if a person moves into an area that was set as a mask area, the person can be found reliably.

[0084] <Variation 5> Next, a description will be given of Modification 5. A mask area M is set for the moving body 70 based on the real space mask area data 76a transmitted from the moving body control device 60. Therefore, when a person who has already been detected enters the range of the mask area M, the detection of the person becomes invalid.

[0085] In the mobile system according to the fifth modification, when an already detected person enters the range of the mask area M, the mask area M is released. Then, even if the already detected person leaves the released mask area M, the release of the mask area M is continued. Note that, when the detected person moves away from the released mask area M, the mask area M may be set again.

[0086] <Variation 6> Next, a description will be given of Modification 6. In the mobile body system according to Modification 6, the mask area M of the mobile body 70 is set based on the real space mask area data 76a transmitted from the mobile body control device 60, but the human detection unit 37b of the mobile body 70 may automatically cancel the mask area M when the detection probability of human detection is equal to or higher than a predetermined threshold (e.g., 0.8 or higher) in a plurality of consecutive frames (e.g., 10 frames) within the range of the mask area M.

[0087] The configurations illustrated in the above embodiments are merely functional schematics, and are not necessarily physically configured as illustrated. In other words, the form of distribution and integration of each device is not limited to that illustrated, and all or part of the devices can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc. [Industrial Applicability]

[0088] The moving body system, moving body, and human region detection method according to the present invention are suitable for efficiently setting a region that may be erroneously detected as a person as a mask region, thereby reducing erroneous detection of a person. [Explanation of symbols]

[0089] 10 Mobile Systems 20 Mobile control device 21 Display section 22 Input section 23 Wireless Communication Department 24 Memory section 24a Mask area data 25 Control Unit 25a Captured image display unit 25b Mask area setting section 25c Mask area data transmission unit 30, 30a, 30b, 30c mobile 31 Imaging unit 32 Altitude Sensor 33 GNSS unit 34 Wireless Communication Section 35 Rotor 36 Memory section 36a Pre-trained model for person detection 36b Captured image data 36c Person candidate area data 36d Real space mask area data 36e Mask time data 37 Control Unit 37a Imaging control unit 37b Person detection section 37c Latitude and longitude processing section 37d Image data transmission unit 37e Mask area data receiving unit 37f Real space coordinate calculation section 37g Human area identification part 40 Mobile control device 44 Memory section 44a Object detection area data 44b Mask area data 45 Control section 45a Object detection area data processing unit 45b False positive judgment unit 50, 50a, 50b, 50c mobile 56 Memory section 56a Pre-trained model for object detection 56b Object detection area data 57 Control Unit 57a Object detection unit 57b Object detection area data transmission unit 60 Mobile control device 64 Storage section 64a Real space mask area data 64b Mask time data 65 Control Unit 65a Real space position calculation unit 65b Real space mask area data transmission unit 70 Mobile 76 Memory section 76a Real space mask area data 76b Mask time data 77 Control Unit 77a Image data transmission unit 77b Real space mask area data receiving unit

Claims

1. A mobile body system including a mobile body mounted with an imaging device and configured to transmit a person as a person candidate area when a person is detected from a captured image, and a mobile body control device configured to control the movement of the mobile body, The mobile object control device includes: A person candidate area in which no person is shown is set as a mask area among the person candidate areas transmitted from the moving body; The moving body is When the mask area is set by the mobile object control device, detection of a person is performed in the captured image excluding the mask area. A mobile system comprising:

2. The mobile object control device includes: An input unit is provided, The person candidate area transmitted from the moving body based on the input from the input unit is set as the mask area.

2. The mobile system according to claim 1 .

3. The mobile object control device includes: The effective time for which the mask area is effective is set in advance, The moving body When a mask area is set by the mobile object control device, detection of a person is performed in the captured image excluding the mask area during a valid time.

2. The mobile system according to claim 1 .

4. The mobile object control device includes: setting an effective time for which the mask region is effective based on an input from the input unit; The moving body is When a mask area is set by the mobile object control device, detection of a person is performed in the captured image excluding the mask area during a valid time.

3. The mobile system according to claim 2.

5. The moving body is When it is detected that the mask area has deviated from the angle of view of the imaging device, the mask area is released.

2. The mobile system according to claim 1 .

6. The moving body is 2. The mobile system according to claim 1, wherein when it is detected that a person area in which a person is detected has moved into the mask area, the mask area is released.

7. A moving object equipped with an imaging device and capable of detecting a person from a captured image, A person candidate region in which a person is not shown is set as a mask region among the person candidate regions in which a person is detected from the captured image captured by the imaging device; After the mask area is set, detection of a person is performed in the captured image excluding the mask area. A moving object characterized by:

8. A method for detecting a person area in a mobile body system having a mobile body equipped with an imaging device and a mobile body control device that controls the movement of the mobile body, comprising: detecting a person from an image captured by the imaging device as a person candidate region; setting a person candidate region in which no person is shown as a mask region among the person candidate regions; When the mask area is set, detecting a person in the captured image excluding the mask area. A person region detection method comprising:

Images