Control device and control program

Abstract

To provide: a control device which issues an appropriate instruction for securing a distance between multiple moving objects which move relatively;to each other; and a control program.SOLUTION: A control device 20 comprises: a detection unit 21 for detecting multiple moving objects which move relatively to each other, on the basis of a video image of an object region; a calculation unit 22 for calculating a distance between the multiple moving objects in a case where the multiple moving objects is detected; and a processing unit 23 for setting, when the distance is shorter than a threshold, an instruction object which is the moving object to be instructed to change of a moving direction and a moving direction after the change, of the instruction object so that the distance becomes equal to or longer than the threshold.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a control device and a control program. [Background technology]

[0002] In recent years, there has been a demand to ensure social distancing in facilities used by an unspecified number of people, such as offices, sports gyms, commercial facilities, etc. Patent Document 1 discloses a device that calculates a predicted inter-vehicle distance based on the position and speed of a merging vehicle, and projects an indicator on a main road based on the calculation result. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-175004 A Summary of the Invention [Problem to be solved by the invention]

[0004] Patent Document 1 describes a technology that projects an indicator to provide instructions to vehicles traveling on a main line, and is not intended to issue instructions to an unspecified number of people to maintain social distance. There is a demand for technology that can issue appropriate instructions to maintain distance between multiple moving objects that are moving relatively, not limited to maintaining social distance.

[0005] The present invention has been made in consideration of the above, and aims to provide a control device and a control program that are capable of issuing appropriate instructions to ensure a distance between multiple moving objects that move relative to one another. [Means for solving the problem]

[0006] The control device of the present invention comprises a detection unit that detects multiple moving targets moving relatively based on an image of a target area, a calculation unit that calculates the distance between the multiple moving targets when the multiple moving targets are detected, and a processing unit that sets a pointing target, which is the moving target that instructs the user to change the direction of movement so that the distance becomes greater than or equal to the threshold value, and the moving direction after the pointing target is changed.

[0007] The control program of this embodiment causes a computer to execute the following processes: detect multiple moving targets moving relatively based on an image of a target area; if multiple moving targets are detected, calculate the distance between the multiple moving targets; and, if the distance is less than a threshold, set a pointing target, which is the moving target that instructs the change in movement direction, and the movement direction after the pointing target is changed, so that the distance becomes greater than or equal to the threshold. Effect of the Invention

[0008] According to the present invention, it is possible to issue appropriate instructions to ensure a distance between a plurality of moving objects that move relatively to one another. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of an instruction system according to the present embodiment. [Diagram 2] FIG. 2 is a diagram showing a schematic diagram of a state in which a plurality of people move within a target area. [Diagram 3] FIG. 3 is a diagram showing an example of an image of a target area captured by an imaging device. [Figure 4] FIG. 4 is a diagram showing an example of an image of a target area captured by an imaging device. [Diagram 5] FIG. 5 is a diagram showing an example in which a sub-object exists in one moving object. [Figure 6] FIG. 6 is a diagram showing an example of an image of a target area captured by an imaging device. [Figure 7]FIG. 7 shows a mode in which the main subject pushes a person sitting in a wheelchair. [Figure 8] FIG. 8 shows a state in which the main subject moves while pulling luggage (carry bag). [Figure 9] FIG. 9 is a diagram showing a state in which the main subject moves while holding a child. [Figure 10] FIG. 10 is a diagram showing an example in which one moving object has a sub-object. [Figure 11] FIG. 11 is a diagram showing an example of an image of a target area captured by an imaging device. [Figure 12] FIG. 12 is a diagram showing a schematic diagram of a state in which a person and a moving object move in a target area. [Figure 13] FIG. 13 is a diagram showing an example of an image of a target area captured by an imaging device. [Figure 14] FIG. 14 is a diagram that illustrates a state in which a plurality of moving objects move within a target area. [Figure 15] FIG. 15 is a diagram showing an example of an image of a target area captured by an imaging device. [Figure 16] FIG. 16 is a flowchart showing an example of the flow of operations of the instruction system according to this embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of a control device and a control program according to the present invention will be described with reference to the drawings. Note that the present invention is not limited to the embodiment. In addition, the components in the following embodiments include those that are replaceable and easy for a person skilled in the art, or those that are substantially the same.

[0011] 1 is a diagram showing an example of a command system 100 according to the present embodiment. In the present embodiment, the command system 100 can be used, for example, to ensure social distancing for an unspecified number of people. As shown in FIG. 1, the command system 100 includes an imaging device 10, a control device 20, and a projection device 30.

[0012] The imaging device 10 captures an image of a target area. For example, at least one of a visible light camera and an infrared camera is used for the imaging device 10. The target area may be, for example, the inside of a facility used by an unspecified number of people, such as an office, a sports gym, or a commercial facility, or an indoor space in the case of an outdoor facility. The target area has a size that allows multiple moving objects to move relatively. In this embodiment, the moving objects include people and objects.

[0013] The control device 20 includes a processing device such as a CPU (Central Processing Unit) and a storage device such as a RAM (Random Access Memory) and a ROM (Read Only Memory). The control device 20 includes a detection unit 21, a calculation unit 22, a processing unit 23, and a storage unit 24.

[0014] The detection unit 21 detects multiple moving objects that move relatively based on the video captured by the imaging device 10. The detection unit 21 performs image processing such as feature extraction, and is capable of detecting multiple moving objects included in the video using a predetermined recognition algorithm or the like.

[0015] The detection unit 21 can detect, for example, a person as a moving target. The detection unit 21 can detect, as a secondary target, a person or object that exists within a predetermined distance from the detected person and moves integrally or following the person. In this case, the detection unit 21 detects the person to which the secondary target belongs as a main target. Examples of secondary targets include, for example, children such as infants and small children, and people sitting in wheelchairs, and objects include luggage. Examples of luggage include luggage that is carried by directly wearing a bag, and luggage that is carried by pushing or pulling, such as a dolly, cart, and a trunk with casters. Note that, although a person or object that moves integrally or following the person can be detected as a secondary target, whether or not the secondary target moves integrally or following the person may be determined based on whether or not the secondary target is in contact with the person determined as the main target for a predetermined period of time. Furthermore, the determination of whether the detected person and the person that moves integrally or following the person may be performed by comparing the heights and surface areas of the people, and a person with a higher height, etc., may be determined as a main target and a person with a lower height, etc., may be determined as a secondary target. In addition, a detected person and an object moving together with or following the person can be determined by image recognition of a predetermined object (background, etc.), with the person being the main target and the object being the secondary target. These are merely examples, and various known techniques can be used.

[0016] The detection unit 21 can detect, for example, a moving body, which is an object that moves independently of a person, as a moving target. Examples of the moving body include mobile robots such as surveillance robots and cleaning robots. In addition, such a robot has, for example, a moving device that can freely move on the floor, walls, etc. of the target area, and a detection device such as a camera or a sensor that can detect an object projected in the target area.

[0017] The detection unit 21 can perform highly accurate detection by using images captured by a visible light camera and images captured by an infrared camera. The infrared camera outputs, for example, temperature information captured by far-infrared rays as brightness. As the infrared camera, an infrared camera having characteristics specialized for the wavelength emitted by the human body among far-infrared rays can be used. For example, in the image captured by the visible light camera, the face of a person may be hidden, or the area on the image may be unclear due to luggage or clothing worn by the person. In such cases, the accuracy of distinguishing between a person and other moving objects and the accuracy of detecting the contour of a person on the image can be improved by determining whether or not there is a part with a surface temperature equivalent to the body temperature of the person in the image captured by the infrared camera, and if so, whether or not the part corresponds to the person's face.

[0018] When multiple moving objects are detected, the calculation unit 22 calculates the distance between the multiple moving objects. When multiple moving objects are detected, the calculation unit 22 calculates the distance between the moving objects, for example, every predetermined period, until the multiple moving objects are no longer detected. When the moving objects are people, the calculation unit 22 can calculate the distance between the heads or between the mouths as the distance between the moving objects, for example. When calculating the distance between the heads, for example, the calculation unit 22 may calculate the distance of a line segment connecting the center coordinates of the heads in a three-dimensional space (three-dimensional coordinates), or may calculate the distance of a line segment connecting the center coordinates of the heads in a planar view (two-dimensional coordinates). The calculation unit 22 can also calculate the position, movement direction, and movement speed of the multiple moving objects. The calculation unit 22 may calculate the position, movement direction, and movement speed in a three-dimensional space, or may calculate the position, movement direction, and movement speed in a two-dimensional space in a planar view.

[0019] When one of multiple moving objects is the main object and a person such as a child or a person sitting in a wheelchair is detected as a secondary object, the calculation unit 22 calculates the distance between the other moving object and either the main object or the secondary object, whichever is closer.

[0020] When the distance is less than the threshold, the processing unit 23 sets the instruction target, which is a moving target for which a change in the moving direction is instructed, and the changed moving direction of the instruction target so that the distance is equal to or greater than the threshold. For example, in order not to disturb the line of movement of the moving target, the processing unit 23 sets the changed moving direction so that the changed moving direction is within ±45° of the moving direction at the time of distance calculation. More specifically, the processing unit 23 calculates the distance between multiple moving targets, and increases the changed moving direction as the distance becomes shorter. For example, when the threshold is 3 m and the distance between the moving targets is 2 m, the processing unit 23 sets the changed moving direction to 23°, and when the distance between the moving targets is 0.5 m, the processing unit 23 sets the changed moving direction to 45°. In this way, the processing unit 23 may set the changed moving direction according to the distance between multiple moving targets. Note that the processing unit 23 may set a direction greater than ±45° if the changed moving direction is a direction that includes a component of the moving direction at the time of distance calculation. Moreover, the processing unit 23 sets the changed movement direction so that the moving object can move a sufficient distance.

[0021] When the detection unit 21 detects a main object and a secondary object, the processing unit 23 selects a target object based on the size of the secondary object relative to the main object, the manner in which the secondary object belongs to the main object, and the amount of change in the moving direction required for the distance between the moving objects to be equal to or greater than a threshold. The size of the secondary object relative to the main object can be obtained by image processing. As an example, it can be shown by the ratio of the area on the image of the person or object determined to be the secondary object to the area on the image of the main object.

[0022] For example, it is considered that the larger the secondary object, the greater the burden on the primary object. On the other hand, even if the secondary object is the same size, the degree of burden is considered to differ depending on the physique or height of the primary object. Therefore, the degree of burden can be adjusted by indicating the ratio between the secondary object and the primary object.

[0023] Regarding the manner in which a secondary object belongs to a primary object, following the above examples, if the secondary object is a child, it can be a manner in which the child moves while holding hands with the primary object, or a manner in which the child moves while being held in the primary object. If the secondary object is a person in a wheelchair, it can be a manner in which the person moves while being pushed in the wheelchair by the primary object. If the secondary object is luggage such as a bag, it can be a manner in which the primary object carries the luggage by wearing it directly on its body. If the secondary object is a dolly, cart, trunk with casters, or the like, it can be a manner in which the primary object carries the luggage by pushing or pulling it.

[0024] For example, when a secondary object (luggage) is transported by pushing or pulling it, such as a dolly, cart, or trunk with casters, the degree of load of the movement is considered to be smaller than the mode of moving while holding a child. On the other hand, when a large luggage is pushed or pulled, the degree of load when changing the direction of movement is considered to be larger than the mode of moving while holding a small child. Therefore, for example, when the amount of change in the direction of movement calculated according to the distance between the moving objects is smaller than a predetermined value, the processing unit 23 can set the one with the smaller degree of load of the movement, that is, the main object transporting the luggage by pushing or pulling it, to be the instruction target. On the other hand, when the amount of change in the direction of movement calculated according to the distance between the moving objects is larger than a predetermined value, the processing unit 23 sets the one with the smaller degree of load for changing the direction of movement, that is, the main object moving while holding a small child, to be the instruction target.

[0025] When a person and a moving body are detected as multiple moving objects, the processing unit 23 can make the moving body a priority target of instruction. The moving body often has a smaller load when changing its moving direction than the person. Also, unlike a person moving by walking, the moving body may be able to move within a range of ±90° with respect to the moving direction at the time of detection with a small load. Therefore, for example, when a person and a moving body are detected as multiple moving objects, the load on the person can be reduced by setting the moving body as a priority target of instruction, such as not setting the person as a moving object or making the change amount of the person's moving direction smaller than the change amount of the moving direction of the moving body. Also, since the moving body can continue to move without stopping, the movement efficiency of the moving body can be improved.

[0026] The processing unit 23 sets the target of instruction and the changed movement direction, and when the distance between the multiple moving targets becomes equal to or greater than a threshold value after an object described below is projected, the processing unit 23 cancels the setting of the target of instruction and the changed movement direction.

[0027] The storage unit 24 has a storage such as a hard disk drive or a solid state drive. Note that an external storage medium such as a removable disk may be used as the storage unit 24. The storage unit 24 stores various programs, data, and the like for controlling the imaging device 10 and the projection device 30.

[0028] The memory unit 24 stores a control program that causes the computer to execute the following processes: detecting multiple moving targets that move relatively based on an image of the target area; calculating the distance between the multiple moving targets when multiple moving targets are detected; and, if the distance is less than a threshold, setting a pointing target, which is a moving target that instructs the user to change the movement direction, and the movement direction after the pointing target is changed, so that the distance becomes greater than or equal to the threshold.

[0029] The projection device 30 projects an object indicating the changed movement direction onto the target area in correspondence with the position of the target. For example, one or more projectors are used as the projection device 30. The projection device 30 projects the object onto a wall, floor, or the like of the target area. The projection device 30 projects the object onto the wall, floor, or the like of the target area, so that the changed movement direction can be instructed to the target. When the target moves, the projection device 30 moves the object so as to follow the change in the position of the target. For example, the projection device 30 can project the object so as to move in accordance with the movement speed of the target. Note that the projection device 30 ends the projection of the object when the processing unit 23 cancels the settings of the target and the changed movement direction.

[0030] Next, an example of a usage mode of the instruction system 100 will be described. Fig. 2 is a diagram showing a state in which multiple people move in a target area. Fig. 3 is a diagram showing an example of an image of the target area captured by the imaging device 10. As shown in Figs. 2 and 3, based on the image captured by the imaging device 10, the detection unit 21 detects multiple people P1 and P2 as moving targets.

[0031] The calculation unit 22 calculates the distance between the person P1 and the person P2. As shown in Fig. 2 and Fig. 3, the calculation unit 22 calculates the distance d1 between the head of the person P1 and the head of the person P2. The calculation unit 22 also calculates the positions, movement directions, and movement speeds of the people P1 and P2.

[0032] When the distance d1 is less than the threshold, the processing unit 23 sets the target of instruction and the moving direction after the target of instruction is changed so that the distance d1 is equal to or greater than the threshold. In the example shown in Fig. 2 and Fig. 3, the persons P1 and P2 have no secondary targets and are similar in build and height, so it is estimated that the degree of load when moving and the degree of load when changing the moving direction are equivalent. In addition, no obstacles that impede the movement are detected around the persons P1 and P2. Therefore, the processing unit 23 sets both persons P1 and P2 as target of instruction and sets the moving direction after the change so that the amount of change is approximately the same with respect to the moving direction at the time of detection.

[0033] 4 is a diagram showing an example of an image of a target area captured by the imaging device 10. As shown in FIG. 4, the projection device 30 projects objects O1 and O2 of the movement direction to correspond to the positions of the people P1 and P2 on, for example, the floor F of the target area in order to instruct the people P1 and P2, who are the target of instruction set by the processing unit 23, of the changed movement direction. The directions indicated by the objects O1 and O2 are changed by approximately the same amount in the direction away from the movement directions (indicated by dashed arrows in the figure) at the time of detection of the people P1 and P2. The people P1 and P2 can easily recognize the direction in which they should move by looking at the projected objects O1 and O2.

[0034] Even after the objects O1 and O2 are projected, the distance between the multiple moving targets is calculated by the calculation unit 22. If the calculated distance is equal to or greater than a threshold, the processing unit 23 cancels the settings of the designated target and the changed moving direction. The projection device 30 ends the projection of the objects O1 and O2 based on the cancellation of the settings.

[0035] Fig. 5 is a diagram showing an example of a case where a secondary object exists in one moving object. Fig. 6 is a diagram showing an example of an image of a target area captured by the imaging device 10. The examples shown in Figs. 5 and 6 show a mode in which a main object moves while holding hands with a child. In this case, the detection unit 21 detects persons P3 and P4 as moving objects. Also, one of the persons, P3, is a main object, and the detection unit 21 detects a child P5, which is a secondary object belonging to the main object. The mode is one in which the child P5 moves while holding hands with the person P3.

[0036] The calculation unit 22 calculates the distances between the person P3 and the child P5 and the person P4. As shown in Fig. 5, the calculation unit 22 calculates the distance d2 between the head of the person P3 and the head of the person P4, and the distance d3 between the head of the child P5 and the head of the person P4. The calculation unit 22 calculates the shorter of the distances d2 and d3 as the distance between the moving objects. In this embodiment, the distance d3 is calculated as the distance between the moving objects. The calculation unit 22 also calculates the positions, moving directions, and moving speeds of the person P3, the child P5, and the person P4.

[0037] When the distance d3 is less than the threshold, the processing unit 23 sets the instruction target and the changed moving direction of the instruction target so that the distance d3 is equal to or greater than the threshold. In the example shown in FIG. 5 and FIG. 6, the person P3 has a child P5 as a secondary target, but the person P4 does not have a secondary target. In this case, it is estimated that the degree of load when moving and the degree of load when changing the moving direction are greater for the person P3. Also, no obstacles or the like that would impede the movement are detected around the persons P1 and P2. Therefore, the processing unit 23 does not set the person P3 as the instruction target, but sets the person P4 as the instruction target, and sets the changed moving direction so that the amount of change is a predetermined amount relative to the moving direction at the time of detection.

[0038] As shown in FIG. 6, the projection device 30 projects an object O3 of a moving direction on the target area, for example, on the floor F, so as to correspond to the position of the person P4 in order to instruct the person P4, who is the target of instruction set by the processing unit 23, of the changed moving direction. The direction indicated by the object O3 is a direction away from the person P3 with respect to the moving direction at the time of detection of the person P4 (indicated by a dashed arrow in the figure). The person P4 can easily recognize the direction in which he or she should move by looking at the projected object O3. Note that, after the object O3 is projected, if the distance between the multiple moving bodies becomes equal to or greater than a threshold value, the processing unit 23 cancels the setting of the designated target and the changed moving direction, and the projection device 30 ends the projection of the object O3.

[0039] 7 is a diagram showing a state in which a main subject pushes a person in a wheelchair. In this case, the detection unit 21 detects persons P6 and P7 as moving subjects. Also, one of the persons, P6, is the main subject, and a person P8 sitting in a wheelchair is detected as a secondary subject belonging to the main subject.

[0040] The calculation unit 22 calculates the distances between the persons P6, P8, and P7. As shown in Fig. 7, the calculation unit 22 calculates a distance d4 between the head of the person P6 and the head of the person P7, and a distance d5 between the head of the person P8 and the head of the person P7. The calculation unit 22 calculates the shorter of the distances d4 and d5 as the distance between the moving objects. In this embodiment, the distance d5 is calculated as the distance between the moving objects.

[0041] When the distance d5 is less than the threshold, the processing unit 23 sets the target of instruction and the moving direction after the change of the target of instruction so that the distance d5 is equal to or greater than the threshold. In the example shown in FIG. 7, as in the examples shown in FIG. 5 and FIG. 6, the person P6 has a secondary target, the person P8, but the person P7 has no secondary target. In this case, it is estimated that the load level when moving and the load level when changing the moving direction are greater for the person P6. In addition, no obstacles or the like that impede the movement are detected around the persons P6 and P7. Therefore, the processing unit 23 does not set the person P6 as the target of instruction, but sets the person P7 as the target of instruction, and sets the moving direction after the change so that the moving direction at the time of detection is changed by a predetermined amount.

[0042] The projection device 30 projects an object of the movement direction to correspond to the position of the person P7 on the floor F of the target area, for example, in order to instruct the person P7, who is the target of instruction set by the processing unit 23, of the changed movement direction. The person P7 can easily recognize the direction in which he or she should move by looking at the projected object. After the object is projected, if the distance between the multiple moving bodies becomes equal to or greater than a threshold value, the processing unit 23 cancels the settings of the designated target and the changed movement direction, and the projection device 30 ends the projection of the object.

[0043] Fig. 8 is a diagram showing a manner in which a main subject moves while pulling a baggage (carry bag). In this case, the detection unit 21 detects persons P9 and P10 as moving subjects. Also, one of the persons, P9, is the main subject, and the detection unit 21 detects baggage M1, which is a secondary subject belonging to the main subject. The calculation unit 22 calculates the distance between the persons P9 and P10. As shown in Fig. 8, the calculation unit 22 calculates the distance d6 between the head of the person P9 and the head of the person P10.

[0044] When the distance d6 is less than the threshold, the processing unit 23 sets the instruction target and the moving direction after the instruction target is changed so that the distance d6 is equal to or greater than the threshold. In the example shown in FIG. 8, as in the examples shown in FIG. 5 and FIG. 6, the person P9 has a baggage M1 as a secondary target, but the person P10 does not have a secondary target. In this case, it is estimated that the degree of load when moving and the degree of load when changing the moving direction are greater for the person P10. In addition, no obstacles or the like that impede the movement are detected around the persons P9 and P10. Therefore, the processing unit 23 does not set the person P9 as the instruction target, for example, but sets the person P10 as the instruction target, and sets the moving direction after the change so that the moving direction at the time of detection is changed by a predetermined amount.

[0045] The projection device 30 projects an object of the movement direction to correspond to the position of the person P10 on the floor F of the target area, for example, in order to instruct the person P10, who is the target of instruction set by the processing unit 23, of the changed movement direction. The person P10 can easily recognize the direction in which he or she should move by looking at the projected object. Note that after the object is projected, if the distance between the multiple moving bodies becomes equal to or greater than a threshold value, the processing unit 23 cancels the settings of the designated target and the changed movement direction, and the projection device 30 ends the projection of the object.

[0046] 9 is a diagram showing a state in which a main subject moves while holding a child. In this case, the detection unit 21 detects persons P11 and P12 as moving subjects. In addition, one of the persons P11 is the main subject, and the detection unit 21 detects a child P13 as a secondary subject belonging to the main subject. The child P13 moves while being held by the person P11.

[0047] The calculation unit 22 calculates the distance between the persons P11, P13, and the person P12. As shown in Fig. 9, the calculation unit 22 calculates a distance d7 between the head of the person P11 and the head of the person P12, and a distance d8 between the head of the child P13 and the head of the person P12. The calculation unit 22 calculates the shorter of the distances d7 and d8 as the distance between the moving objects. In this embodiment, the distance d8 is calculated as the distance between the moving objects.

[0048] When the distance d8 is less than the threshold, the processing unit 23 sets the instruction target and the moving direction after the instruction target is changed so that the distance d8 is equal to or greater than the threshold. In the example shown in FIG. 9, as in the examples shown in FIG. 5 and FIG. 6, the person P11 has a child P13 as a secondary target, but the person P12 does not have a secondary target. In this case, it is estimated that the degree of load when moving and the degree of load when changing the moving direction are greater for the person P11. In addition, no obstacles or the like that impede the movement are detected around the persons P11 and P12. Therefore, the processing unit 23 does not set the person P11 as the instruction target, but sets the person P12 as the instruction target, and sets the moving direction after the change so that the amount of change is a predetermined amount relative to the moving direction at the time of detection.

[0049] The projection device 30 projects an object of the movement direction to correspond to the position of the person P12 on the floor F of the target area, for example, in order to instruct the person P12, who is the target of instruction set by the processing unit 23, of the changed movement direction. The person P12 can easily recognize the direction in which he or she should move by looking at the projected object. Note that after the object is projected, if the distance between the multiple moving bodies becomes equal to or greater than a threshold value, the processing unit 23 cancels the settings of the designated target and the changed movement direction, and the projection device 30 ends the projection of the object.

[0050] FIG. 10 is a diagram showing an example of a case where a secondary object exists in one moving object. FIG. 11 is a diagram showing an example of an image of a target area captured by the imaging device 10. The examples shown in FIG. 10 and FIG. 11 are a mode in which one main object moves while holding a child, and the other main object moves while holding hands with the child. In this case, the detection unit 21 detects persons P14 and P15 as moving objects. Also, the person P14 is the main object, and the detection unit 21 detects the child P16, which is a secondary object belonging to the main object. The child P16 moves while being held by the person P14. Also, the person P15 is the main object, and the detection unit 21 detects the child P17, which is a secondary object belonging to the main object. The detection unit 21 detects the child P17, which is a secondary object belonging to the main object, while holding hands with the person P15.

[0051] The calculation unit 22 calculates the distance between the person P14 and the child P16, and the person P15 and the child P17. As shown in Fig. 10, the calculation unit 22 calculates the distance d9 between the head of the person P14 and the head of the person P15, the distance between the head of the person P14 and the head of the child P17, the distance between the head of the child P16 and the head of the person P15, and the distance between the head of the child P16 and the head of the child P17. The calculation unit 22 calculates the distance d9, which is the shortest distance among the calculated distances, as the distance between the moving objects. The calculation unit 22 also calculates the positions, moving directions, and moving speeds of the people P14, P15, and the children P16, P17.

[0052] When the distance d9 is less than the threshold, the processing unit 23 sets the instruction target and the moving direction after the instruction target is changed so that the distance d9 is equal to or greater than the threshold. In the example shown in FIG. 10 and FIG. 11, the person P14 has a child P16 as a secondary target, and the person P15 has a child P17 as a secondary target. Meanwhile, the child P16 is being held by the person P14, while the child P17 is holding hands with the person P15. The people P14 and P15 have the same physique, height, etc., and the children P16 and P17 have the same physique, height, etc. Therefore, it is estimated that the degree of load when moving and the degree of load when changing the moving direction are greater for the person P14 holding the child P16. Also, no obstacles or the like that impede the movement are detected around the people P14 and P15. Therefore, the processing unit 23 does not set the person P14 as the pointing target, but sets the person P15 as the pointing target, and sets the changed moving direction so that the changed moving direction is a predetermined amount relative to the moving direction at the time of detection.

[0053] As shown in FIG. 11, the projection device 30 projects an object O4 of a moving direction on the target area, for example, on the floor F, so as to correspond to the position of the person P15 in order to instruct the person P15, who is the target of instruction set by the processing unit 23, of the changed moving direction. The direction indicated by the object O4 is a direction away from the person P14 with respect to the moving direction at the time of detection of the person P15 (indicated by a dashed arrow in the figure). The person P15 can easily recognize the direction in which he or she should move by looking at the projected object O4. Note that, after the object O4 is projected, if the distance between the multiple moving bodies becomes equal to or greater than a threshold value, the processing unit 23 cancels the setting of the designated target and the changed moving direction, and the projection device 30 ends the projection of the object O4.

[0054] FIG. 12 is a diagram showing a state in which a person and a moving object move in a target area. FIG. 13 is a diagram showing an example of an image of a target area captured by the imaging device 10. As shown in FIG. 12 and FIG. 13, based on the image captured by the imaging device 10, the detection unit 21 detects a person P18 and a surveillance robot R1 as moving targets. The surveillance robot R1 can move autonomously by a mobile device R1a and captures an image of the surroundings by a camera R1b. The surveillance robot R1 can detect surrounding obstacles and the like based on the image captured by the camera R1b, and can move by the mobile device R1a so as to avoid contact or collision with the surrounding obstacles. The camera R1b can detect an object projected by the projection device 30.

[0055] The calculation unit 22 calculates the distance between the person P18 and the surveillance robot R1. As shown in Fig. 12 and Fig. 13, the calculation unit 22 calculates the shortest distance d10 between the surface of the person P18 and the surface of the surveillance robot R1. The calculation unit 22 also calculates the positions, moving directions, and moving speeds of the person P18 and the surveillance robot R1.

[0056] When the distance d10 is less than the threshold, the processing unit 23 sets the instruction target and the moving direction after the instruction target is changed so that the distance d10 is equal to or greater than the threshold. In the example shown in FIG. 12 and FIG. 13, one of the moving targets is a person P18, and the person P18 does not have a sub-target. The other moving target is the surveillance robot R1. Therefore, although the degree of load when moving and the degree of load when changing the moving direction are lower for the surveillance robot R1 than for the person P18, it is estimated that the degree of load for the person P18 is not particularly high. In addition, no obstacles or the like that impede the movement are detected around the person P18 and the surveillance robot R1. Therefore, the processing unit 23 sets both the person P18 and the surveillance robot R1 as instruction targets, and sets the moving direction so that the change amount for the surveillance robot R1 is larger than that for the person P18.

[0057] As shown in FIG. 13, the projection device 30 projects objects O5 and O6 of the moving direction to correspond to the positions of the person P18 and the surveillance robot R1 on the floor F of the target area, for example, in order to instruct the person P18 and the surveillance robot R1, which are the target of instruction set by the processing unit 23, of the changed moving direction. The direction indicated by the objects O5 and O6 is changed by θ1 and θ2 (where θ1<θ2) in the direction away from the moving direction (indicated by the solid arrow in the figure) at the time of detection of the person P18 and the surveillance robot R1. The person P18 and the surveillance robot R1 can easily recognize the direction in which they should move by looking at the projected objects O5 and O6 or by photographing and detecting them with the camera R1b. Note that, after the objects O5 and O6 are projected, if the distance between the multiple moving bodies becomes equal to or greater than a threshold value, the processing unit 23 cancels the setting of the designated target and the changed moving direction, and the projection device 30 ends the projection of the objects O5 and O6.

[0058] FIG. 14 is a diagram showing a state where a plurality of moving bodies move in a target area. FIG. 15 is a diagram showing an example of an image of a target area captured by the imaging device 10. As shown in FIG. 14 and FIG. 15, based on the image captured by the imaging device 10, the detection unit 21 detects a monitoring robot R2 and a cleaning robot R3, which are moving bodies, as moving targets. The monitoring robot R2 has a configuration similar to that of the monitoring robot R1 described above, and has a moving device R2a and a camera R2b. The cleaning robot R3 can move autonomously by the moving device R3a and captures an image of the surroundings by the camera R3b. The cleaning robot R3 can detect surrounding obstacles and the like based on the image captured by the camera R3b, and can suck up foreign objects from the suction unit R3c while moving by the moving device R3a so as to avoid contact or collision with the surrounding obstacles. The camera R3b can detect an object projected by the projection device 30.

[0059] The calculation unit 22 calculates the distance between the surveillance robot R2 and the cleaning robot R3. As shown in Fig. 14 and Fig. 15, the calculation unit 22 calculates the shortest distance d11 between the surface of the surveillance robot R2 and the surface of the cleaning robot R3. The calculation unit 22 also calculates the positions, moving directions, and moving speeds of the surveillance robot R2 and the cleaning robot R3.

[0060] When the distance d11 is less than the threshold, the processing unit 23 sets the instruction target and the moving direction after the instruction target is changed so that the distance d11 is equal to or greater than the threshold. In the example shown in FIG. 14 and FIG. 15, one of the moving targets is the surveillance robot R2, and the other is the cleaning robot R3. The surveillance robot R2 is larger than the cleaning robot R3. Therefore, although the load level when moving and the load level when changing the moving direction are lower for the cleaning robot R3 than for the surveillance robot R2, it is estimated that the load level of the surveillance robot R2 is not particularly high. In addition, no obstacles or the like that hinder the movement are detected around the surveillance robot R2 and the cleaning robot R3. Therefore, the processing unit 23 sets both the surveillance robot R2 and the cleaning robot R3 as instruction targets, and sets the moving direction so that the change amount of the cleaning robot R3 is larger than that of the surveillance robot R2.

[0061] As shown in FIG. 15, the projection device 30 projects objects O7 and O8 of the moving direction to correspond to the positions of the surveillance robot R2 and the cleaning robot R3 on the target area, for example, the floor F, in order to instruct the surveillance robot R2 and the cleaning robot R3, which are the target of instruction set by the processing unit 23, of the changed moving direction. The directions indicated by the objects O7 and O8 are changed by θ3 and θ4 (where θ3<θ4) in the direction away from the moving direction (indicated by the solid arrow in the figure) at the time of detection of the surveillance robot R2 and the cleaning robot R3. The surveillance robot R2 and the cleaning robot R3 can easily recognize the direction in which they should move by photographing and detecting the projected objects O7 and O8 with a camera. Note that, if the distance between the multiple moving bodies becomes equal to or greater than a threshold value after the objects O7 and O8 are projected, the processing unit 23 cancels the setting of the designated target and the changed moving direction, and the projection device 30 ends the projection of the objects O7 and O8.

[0062] 16 is a flowchart showing an example of the operation flow of the instruction system 100 according to this embodiment. First, the imaging device 10 acquires an image of the target area (step S10). The detection unit 21 detects multiple moving targets based on the acquired image (step S20). If multiple moving targets are not detected (No in step S20), the process of step S20 is repeated. If multiple moving targets are detected (Yes in step S20), the calculation unit 22 calculates the distance between the detected multiple moving targets (step S30).

[0063] The processing unit 23 determines whether the calculated distance is less than a threshold value (step S40). If the calculated distance is not less than the threshold value (No in step S40), the processing ends. If the calculated distance is less than the threshold value (Yes in step S40), the processing unit 23 sets the target of instruction and the post-change movement direction of the target of instruction so that the distance is equal to or greater than the threshold value (step S50). The projection device 30 projects an object indicating the post-change movement direction that has been set, in correspondence with the position of the target of instruction, onto the corresponding area (step S60).

[0064] The processing unit 23 determines whether the distance calculated by the calculation unit 22 is equal to or greater than a threshold (step S70). If the calculated distance is less than the threshold (No in step S70), the processing of step S70 is repeated. If the calculated distance is equal to or greater than the threshold (Yes in step S70), the processing unit 23 cancels the settings of the designated target and the changed moving direction (step S80). The projection device 30 ends the projection of the object based on the cancellation of the settings (step S90).

[0065] As described above, the control device 20 of this embodiment includes a detection unit 21 that detects multiple moving targets moving relatively based on an image of the target area, a calculation unit 22 that calculates the distance between the multiple moving targets when multiple moving targets are detected, and a processing unit 23 that, when the distance is less than a threshold, sets a pointing target, which is a moving target that instructs the user to change the movement direction so that the distance becomes equal to or greater than the threshold, and sets the movement direction of the pointing target after the change.

[0066] In addition, the control program of this embodiment causes a computer to execute the following processes: detect multiple moving targets that move relatively based on an image of the target area; if multiple moving targets are detected, calculate the distance between the multiple moving targets; and, if the distance is less than a threshold, set a pointing target, which is a moving target that instructs the user to change the movement direction, and the movement direction after the pointing target is changed, so that the distance becomes greater than or equal to the threshold.

[0067] According to this configuration, multiple moving objects moving relatively to each other are detected based on the video, and when the distance between the moving objects becomes less than a threshold, the target to be pointed to and the new moving direction of the target to be pointed to can be automatically set. Therefore, by notifying the set target to its new moving direction, the target to be pointed to can easily recognize its new moving direction. This makes it possible to issue appropriate instructions to ensure the distance between multiple moving objects moving relatively to each other.

[0068] The instruction system 100 according to the present embodiment includes an imaging device 10 that captures an image of a target area, the above-mentioned control device 30, and a projection device 30 that projects an object indicating a changed moving direction onto the target area in correspondence with the position of the target. With this configuration, it is possible to project an object indicating a changed moving direction in correspondence with the position of the target. Therefore, the moving target that has become the target of instruction can easily recognize the changed moving direction by looking at or detecting the object. This makes it possible to issue appropriate instructions to ensure the distance between multiple moving targets that move relatively.

[0069] In the control device 20 according to this embodiment, the detection unit 21 can detect a person as a moving object, detects the person as a main object, and detects a person or object that exists within a predetermined distance from the person and moves integrally with or following the person as a secondary object, and when the main object is detected as one of multiple moving objects, the processing unit 23 selects an instruction target based on the size of the secondary object relative to the main object, the manner in which the secondary object belongs to the main object, and the magnitude of change in the moving direction of the multiple moving objects required for the distance to be equal to or greater than a threshold. With this configuration, the load level when the main object moves with the secondary object can be appropriately adjusted.

[0070] In the control device 20 according to the present embodiment, when the secondary object is a person, the calculation unit 22 calculates the shorter distance between the main object and the secondary object as the distance to the other moving object. With this configuration, the distances between multiple moving objects can be calculated more appropriately.

[0071] In the control device 20 according to this embodiment, the detection unit 21 detects a moving body, which is an object that moves independently of a person, as a moving target, and when a person and a moving body are detected as multiple moving targets, the processing unit 23 gives priority to the moving body as a target to be pointed out. With this configuration, it is possible to appropriately adjust the load level during the movement of the person and the moving body.

[0072] The technical scope of the present invention is not limited to the above embodiment, and can be modified as appropriate without departing from the spirit of the present invention. For example, in the above embodiment, the operation for ensuring social distance in a target area such as an office, a sports gym, or a commercial facility is described as an example, but the present invention is not limited to this. For example, the present invention can be applied to other situations where the distance between multiple moving objects may be less than a threshold. As such other situations, the present invention can be applied to, for example, when adjusting the distance between members when dancing in a group. [Explanation of symbols]

[0073] 10: imaging device, 20: control device, 21: detection unit, 22: calculation unit, 23: processing unit, 30: projection device, 100: indication system

Claims

1. A detection unit that detects a plurality of moving objects that move relatively based on an image of a target area; a calculation unit that calculates distances between the plurality of moving objects when the plurality of moving objects are detected; a processing unit that sets a designated object, which is a moving object that instructs a change in a moving direction, and the moving direction after the designated object is changed, so that the distance is equal to or greater than the threshold value when the distance is less than the threshold value; Equipped with the detection unit is capable of detecting a person as the moving target, detects the person as a main target, and detects a person or object that exists within a predetermined distance from the person and moves integrally with or following the person as a secondary target; The processing unit is a control device that, when the main object is detected as one of the multiple moving objects, selects the object to be pointed to based on the size of the secondary object relative to the main object, the manner in which the secondary object belongs to the main object, and the magnitude of change in the direction of movement required for the distance to be greater than or equal to a threshold.

2. When the secondary object is a person, the calculation unit calculates a shorter distance between the main object and the secondary object as a distance to the other moving object. The control device according to claim 1 .

3. A computer comprising: A process of detecting a plurality of moving objects that move relatively based on an image of a target area; When a plurality of moving objects are detected, a process of calculating distances between the plurality of moving objects; if the distance is less than a threshold, executing a process of setting a designated object, which is a moving object that instructs a change in a moving direction, and the moving direction after the change of the designated object, so that the distance becomes equal to or greater than the threshold; The detection process is capable of detecting a person as the moving object, detecting the person as a main object, and detecting a person or object that exists within a predetermined distance from the person and moves integrally with or following the person as a secondary object, The setting process is a control program that, when the main object is detected as one of the multiple moving objects, selects the object to be pointed to based on the size of the secondary object relative to the main object, the manner in which the secondary object belongs to the main object, and the magnitude of change in the movement direction required for the distance to be greater than or equal to a threshold.

Images