Real space image projection system

Abstract

To provide a real space image projection system capable of simply and reliably notifying a wide variety of people moving in a real space such as a road of situations and dangers in the real space.SOLUTION: A real space image projection system includes one or more sensor devices 1 and projection devices 3 installed in a real space related to a static object Y and a dynamic object X, and a management device 2 connected in a communicable state to the sensor devices 1 and the projection devices 3. The sensor devices 1 acquire image sensor data consisting of point clouds in the real space, and the management device 2 aggregates the image sensor data consisting of point clouds in the real space transmitted from the sensor devices 1 to form three-dimensional shapes of the static object Y and the dynamic object X in the real space, and performs a mapping process of a predetermined image G based on the three-dimensional shapes. Then, the projection devices 3 project the predetermined image G onto predetermined locations of the static object Y or the dynamic object X in the real space.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a real space image projection system that projects a predetermined image onto a static or dynamic object in real space. [Background technology]

[0002] In recent years, smart cities have been attracting attention as a form of urban development that uses ICT (Information and Communication Technology) to improve the quality of life, promote economic circulation through the creation of new value, and resolve social issues. This smart city is not simply digitalization, but a transformation similar to the revolution from mobile phones to smartphones. As a cyber-physical system that supports smart cities, real machines such as robots, drones, and self-driving cars use virtual information such as 3D images to carry out tasks.

[0003] To realize such smart cities, self-driving cars are already equipped with cameras and millimeter-wave radars to obtain virtual information such as the above-mentioned 3D images. For example, cameras can detect most of the information necessary for driving, such as white lines, signs, other vehicles, and the surrounding environment, but it is difficult to detect the distance to an object. In addition, millimeter-wave radar is a device that emits radio waves with frequencies of 30 GHz to 300 GHz called millimeter waves and measures their reflection. It can easily detect the relative speed with other vehicles driving around, but its azimuth resolution is not very high, making it difficult to detect objects with low radio wave reflectivity, such as trees.

[0004] Therefore, in recent years, a sensor called LiDAR (light detection and ranging) has been attracting attention, along with cameras and millimeter wave radars. This LiDAR is a type of sensor that uses laser light, and has a higher radiation flux density than radio waves. It accurately detects the distance to the object, as well as the position and shape of the object, by scanning and irradiating the object with a short-wavelength laser light. Therefore, when LiDAR is used in a real space where static objects such as walls and dynamic objects such as automobiles exist, three-dimensional information of the real space can be obtained as image sensor data consisting of a point cloud of laser light. Moreover, when LiDAR installed in the automobile and LiDAR placed in the surrounding environment such as traffic lights and streetlights on the road are used, these can transmit image sensor data to the automobile, thereby informing the automobile of the situation of blind spots and dangers on the road (for example, see Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Shackleton, J., VanVoorst, B., & Hesch, J. (2010, August). Tracking people with a 360-degree lidar. In 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance (pp. 420-426). IEEE. Summary of the Invention [Problem to be solved by the invention]

[0006] However, conventional systems only transmit image sensor data to the vehicle, and so while they can inform the driver of the situation and dangers in blind spots on the road, they cannot inform pedestrians on the road other than the driver, or people on bicycles or in wheelchairs, of the situation and dangers in blind spots on the road. The present invention has been made in consideration of the above-mentioned technical background, and has an objective of providing a real-space image projection system that can easily and reliably inform a wide variety of people moving in real spaces such as roads, such as drivers driving cars, pedestrians walking on the road, and people on bicycles or in wheelchairs, of the situation and dangers in the real space. [Means for solving the problem]

[0007] In order to achieve the above object, the present invention provides a method and apparatus for controlling a real space sensor system, comprising: one or more sensor devices installed in a real space related to static objects and dynamic objects; a management device connected to the sensor devices in a communicable state; and one or more projection devices connected to the management device in a communicable state and installed in the real space, the sensor devices comprising a sensor unit that acquires image sensor data consisting of a point cloud in the real space; and a sensor-side transmission unit that transmits the image sensor data consisting of the point cloud in the real space acquired by the sensor unit to the management device, the management device comprising a management-side receiving unit that receives the image sensor data consisting of the point cloud in the real space transmitted from the sensor devices; an aggregation unit that aggregates the image sensor data consisting of the point cloud in the real space received by the management-side receiving unit; and a projection device that aggregates the image sensor data consisting of the point cloud in the real space aggregated by the aggregation unit. the projection device further comprises a three-dimensional shape construction unit which constructs a three-dimensional shape of a static object and / or a dynamic object in real space based on image sensor data; a mapping processing unit which performs a mapping process of a predetermined image onto a static object and / or a dynamic object in real space based on the three-dimensional shape of the static object and / or dynamic object in real space constructed by the three-dimensional shape construction unit; and a management side transmitting unit which transmits to the projection device information regarding the mapping of the predetermined image which has been mapped by the mapping processing unit, wherein the projection device further comprises a projection side receiving unit which receives information regarding the mapping of the predetermined image transmitted from the management device; and a projection unit which projects the predetermined image onto the static object and / or dynamic object in real space based on the information regarding the mapping of the predetermined image received by the management side receiving unit.

[0008] The mapping processing unit may also determine a projection position of a predetermined image onto a static object and / or a dynamic object in real space.

[0009] The mapping processing unit may determine a projection position of a predetermined image onto a static object and / or a dynamic object in the real space, depending on a state of the dynamic object in the real space.

[0010] The mapping processing unit may determine a projection position of a predetermined image at a predetermined location of a static object that avoids dynamic objects in real space.

[0011] The mapping processing unit may successively determine a projection position of a predetermined image so as to follow a predetermined location of a dynamic object moving in real space.

[0012] Furthermore, when a predetermined dynamic object is detected in the real space, the mapping processing unit may determine a projection position of the predetermined image onto a static object and / or a dynamic object in the real space.

[0013] Furthermore, when two or more dynamic objects are detected in the real space, the mapping processing unit may determine a projection position of a predetermined image at a predetermined location of the dynamic object moving ahead.

[0014] The mapping processing unit may generate a predetermined image according to a state of a dynamic object in real space.

[0015] Furthermore, when a predetermined dynamic object is detected in real space, the mapping processing unit may generate a predetermined image that warns of the approach of the dynamic object.

[0016] Furthermore, when a predetermined dynamic object is detected in real space, the mapping processing unit may determine a predetermined location of a static object corresponding to the position of the dynamic object as the projection position of the predetermined image.

[0017] In addition, when the mapping processing unit detects a specific dynamic object in real space, it may determine a projection position of a specific image so that the specific image moves on a specific location of a static object in accordance with the movement of the dynamic object.

[0018] A display device may be installed in a real space related to a static object and a dynamic object, and the mapping processing unit may generate a predetermined image related to information displayed on the display device. Effect of the Invention

[0019] According to the present invention, it is possible to easily and reliably inform a wide variety of people moving through real spaces such as roads, including drivers, pedestrians walking on the road, and people on bicycles or in wheelchairs, of the situation and dangers in the real space, thereby making it possible to provide a safe and secure transportation system. [Brief description of the drawings]

[0020] [Figure 1] 1 is a schematic diagram showing an overview of a real space image projection system according to the present invention. [Diagram 2] FIG. 2 is a block diagram showing a configuration of the real space image projection system of FIG. [Diagram 3] FIG. 3 is a block diagram showing the functions of the sensor device and the projection device of FIG. 2. [Figure 4] FIG. 3 is a block diagram showing the functions of the management device of FIG. 2. [Diagram 5] FIG. 1 is a diagram showing a three-dimensional real space relating to a facility. [Figure 6] FIG. 1 is a diagram showing an example of image sensor data obtained by Lidar. [Figure 7] 1A to 1C are diagrams illustrating a process of forming a three-dimensional shape. [Figure 8] 2 is a flowchart showing the operation of the real space image projection system of FIG. 1. [Figure 9] 1 is a schematic diagram illustrating an overview of a real space image projection system according to a first embodiment. [Figure 10] FIG. 11 is a schematic diagram illustrating an overview of a real space image projection system according to a second embodiment. [Figure 11] FIG. 11 is a schematic diagram illustrating an overview of a real space image projection system according to a third embodiment. [Figure 12] FIG. 13 is a schematic diagram illustrating an overview of a real space image projection system according to a fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Next, an embodiment of a real space image projection system (hereinafter, referred to as the present system) according to the present invention will be described with reference to FIGS.

[0022] [Overall configuration] As shown in Figures 1 and 2, this system comprises a plurality of sensor devices 1 installed in real space, a management device 2 connected to the sensor devices 1 in a state capable of communicating with each other, and a plurality of projection devices 3 installed in real space and connected to the management device 2 in a state capable of communicating with each other.

[0023] In this embodiment, the real space is assumed to be an outdoor three-dimensional real space related mainly to dynamic objects X (pedestrians, vehicles, etc.) and static objects Y (roads, walls, etc.), but it may also be a real three-dimensional space related to social life or the environment, such as a facility, indoors, a river, or a mountain.

[0024] [Configuration of sensor device 1] The sensor device 1 is installed on a road via a support pole 4 as shown in FIG. 1, and includes a sensor unit 11 that acquires image sensor data consisting of a point cloud in real space at predetermined time intervals as shown in FIG. 3, and a sensor side transmission unit 12 that sequentially transmits the image sensor data consisting of the point cloud in real space acquired by the sensor unit 11 to the management device 2.

[0025] The sensor unit 11 is a so-called LiDAR (light detection and ranging) sensor. The LiDAR is a type of sensor that uses laser light, and has a higher radiant flux density than radio waves. It irradiates a target object with a short-wavelength laser light while scanning it, thereby acquiring image sensor data consisting of a point cloud in real space, and accurately detects not only the distance to the target object, but also the position and shape of the target object.

[0026] Examples of LiDAR include a method of acquiring 360-degree omnidirectional image sensor data by rotating a light emitter that emits multiple laser beams, and a method of acquiring image sensor data by directly irradiating laser beams within a range of a predetermined light irradiation angle. In addition, the more light emitters that emit laser beams, the higher the accuracy of the image sensor data, but the higher the cost, so an inexpensive LiDAR with a small number of light emitters may be used.

[0027] Here, an example of acquiring image sensor data consisting of a cloud of points in real space by LiDAR will be described. As shown in FIG. 5, a certain facility surrounded by a ceiling, floor, and walls has certain desks, chairs, and shelves arranged at various locations, and two dynamic objects X (pedestrians) are present near the center. The facility is treated as a three-dimensional real space. A LiDAR is installed on the front side of FIG. 5, and a laser beam is scanned and irradiated from the LiDAR into the facility at a certain irradiation angle. In this case, as shown in FIG. 6, the LiDAR acquires image sensor data consisting of a cloud of points in real space, recognizes the dynamic object X (pedestrian) in the center, and also recognizes static objects Y (walls, floors, ceilings, desks, chairs, shelves). Such image sensor data consisting of a cloud of points in real space by LiDAR is also acquired for outdoor static objects Y (roads, walls, etc.) and dynamic objects X (vehicles, pedestrians, etc.).

[0028] In addition, when image sensor data is acquired by one LiDAR, a shadow of dynamic object X is cast, and the spatial region where the shadow is cast becomes blank, which may make it difficult to distinguish and determine the spatial region related to static object Y from the spatial region related to dynamic object X. Therefore, by combining image sensor data made up of point clouds acquired by multiple sensor devices 1, the influence of the shadow of dynamic object X can be reduced, so it is preferable to provide multiple LiDARs so as to acquire image sensor data made up of point clouds from different directions in the same real space.

[0029] [Configuration of management device 2] The management device 2 is arranged in a state capable of wired or wireless communication with the sensor device 1, and as shown in Figure 4, is equipped with a management side receiving unit 21 that sequentially receives image sensor data consisting of a point cloud in real space sequentially transmitted from the sensor device 1, an aggregation unit 22 that aggregates the image sensor data consisting of the point cloud in real space, a three-dimensional shape construction unit 23 that constructs a three-dimensional shape in real space based on the image sensor data consisting of the point cloud in real space, a mapping processing unit 24 that performs mapping processing of a specified image G onto a static object Y and / or a dynamic object X in real space, and a management side transmitting unit 25 that transmits information regarding the mapping of the specified image G to the projection device 3.

[0030] The aggregation unit 22 aggregates image sensor data consisting of point clouds in real space received by the management side receiving unit 21, and in this embodiment, aggregates image sensor data consisting of point clouds in real space transmitted sequentially from each sensor device 1 by synthesizing them in a chronological order.

[0031] In addition, when aggregating image sensor data consisting of a cloud of points in real space received by the management side receiving unit 21, the aggregation unit 22 may perform a smoothing process on the point cloud of the image sensor data, or may separately store the image sensor data consisting of the point cloud in real space before and after aggregation in a storage device not shown.

[0032] The three-dimensional shape construction unit 23 constructs three-dimensional shapes of the static object Y and the dynamic object X in real space based on the image sensor data consisting of a point cloud in real space aggregated by the aggregation unit 22. For example, when the three-dimensional shape construction unit 23 acquires a point cloud of image sensor data consisting of a point cloud in real space as shown in Fig. 7(a), it meshes the image sensor data by connecting adjacent point clouds as shown in Fig. 7(b), and then constructs three-dimensional shapes (3D) of the static object Y and the dynamic object X in real space as shown in Fig. 7(c).

[0033] The method for constructing the three-dimensional shapes of the static object Y and the dynamic object X in real space by the three-dimensional shape construction unit 23 is not particularly limited, and the three-dimensional shapes may remain as the point cloud of FIG. 7(a) or may be in a meshed state as shown in FIG. 7(b).

[0034] The mapping processing unit 24 performs mapping processing of a predetermined image G onto a static object Y and / or a dynamic object X in real space based on the three-dimensional shape of the static object Y and / or the dynamic object X in real space constructed by the three-dimensional shape construction unit 23.

[0035] Furthermore, the mapping processing unit 24 determines a projection position of a predetermined image G on a static object Y and / or a dynamic object X in real space. For example, the mapping processing unit 24 may determine, as the projection position of the predetermined image G, a predetermined location of a static object Y in real space (such as the surface of a road or a wall), a predetermined location of a dynamic object X in real space (such as the back of a pedestrian), or a predetermined location of a static object Y avoiding dynamic object X in real space (such as the surface of a road or a wall).

[0036] The mapping processing unit 24 may also determine the projection position of a predetermined image G onto a static object Y and / or a dynamic object X in the real space according to the state of the dynamic object X in the real space. For example, when the mapping processing unit 24 detects a predetermined dynamic object (such as a vehicle) in the real space, the mapping processing unit 24 may determine the projection position of a predetermined image G onto a static object Y and / or a dynamic object X in the real space. When the mapping processing unit 24 detects two or more dynamic objects X (such as pedestrians) in the real space, the mapping processing unit 24 may determine the projection position of a predetermined image G at a predetermined location (such as the back of a pedestrian) of the dynamic object X moving ahead. The mapping processing unit 24 may also sequentially determine the projection position of the predetermined image G so as to follow a predetermined location of the dynamic object X (such as a pedestrian) moving in the real space.

[0037] Furthermore, the mapping processing unit 24 generates a predetermined image G (mapping image) to be projected at the above-mentioned projection position of the static object Y and / or the dynamic object X in the real space. For example, the mapping processing unit 24 generates an image of characters or figures that calls the attention of pedestrians at a predetermined location (such as the surface of a road or wall) of the static object Y, generates an image of characters or figures that calls the attention of a vehicle driver or other pedestrians at a predetermined location (such as the back of a pedestrian) of the dynamic object X, or generates an image related to advertisement or publicity to inform pedestrians, etc. Note that these predetermined images G may be either still images or videos (including those in which still images are switched sequentially).

[0038] The mapping processing unit 24 may generate a plurality of different images assuming that a predetermined image G is to be projected while being synthesized by a plurality of projection devices 3, or may perform projective transformation on a predetermined image G assuming that the image G is to be projected on an inclined surface such as the back of a pedestrian. In this case, the mapping processing unit 24 may use a technique called projection mapping to give the image or other material a design that matches the unevenness of the object, or three-dimensional information and surface information regarding the object's shape, and adjust the shape of the object to overlap when projected.

[0039] The mapping processing unit 24 may generate a predetermined image G that is set in advance, or may generate a predetermined image G according to the state of the dynamic object X in the real space.

[0040] [Configuration of Projection Device 3] The projection device 3 is installed on the street via a pole 4 in a state in which it can communicate with the management device 2 via a wired or wireless connection, as shown in Fig. 1, and includes a projection-side receiving unit 31 that receives information regarding the mapping of a predetermined image G transmitted from the management device 2, and a projection unit 32 that projects the predetermined image G onto the projection position of a static object Y and / or a dynamic object X in real space, as shown in Fig. 3. In this embodiment, the sensor device 1 and the projection device 3 are provided integrally.

[0041] The projection unit 32 is a so-called projector, and projects the predetermined image G onto the projection position of the static object Y and / or the dynamic object X in the real space based on the information on the mapping of the predetermined image G received by the projection side receiving unit 31. For example, when the mapping processing unit 24 of the management device 2 determines a predetermined location of the static object Y (such as the surface of a road or a wall) as the projection position of the predetermined image G and generates a predetermined image G of characters or figures that call the attention of pedestrians, the projection unit 32 projects the predetermined image G of characters or figures that call the attention of pedestrians onto the predetermined location of the static object Y (such as the surface of a road or a wall). In addition, when the mapping processing unit 24 of the management device 2 sequentially determines a predetermined location of the dynamic object X (such as the back of a pedestrian) as the projection position of the predetermined image G and generates a predetermined image G of characters or figures that call the attention of pedestrians, the projection unit 32 sequentially projects the predetermined image G of characters or figures that call the attention of pedestrians so as to follow the predetermined location of the dynamic object X (such as the back of a pedestrian). In this embodiment, a projector is used as the projection unit 32, but other devices may be used as long as they are capable of projecting characters, figures, and the like.

[0042] [System operation] Next, the operation of this system from acquisition of image sensor data consisting of a cloud of points in real space to projection of a predetermined image G will be described with reference to the flowchart in Fig. 8. In the following description, "step" will be abbreviated as "S".

[0043] First, in the sensor device 1, the sensor unit 11 acquires image sensor data consisting of a group of points in a three-dimensional real space relating to a static object Y (such as a road or a wall) and a dynamic object X (such as a pedestrian or a vehicle) (S1).

[0044] Then, the sensor side transmission unit 12 transmits image sensor data consisting of a group of points in real space acquired by the sensor unit 11 to the management device 2 (S2).

[0045] Next, in the management device 2, the management-side receiving unit 21 receives the image sensor data made up of a group of points in real space transmitted from the sensor device 1 (S3).

[0046] Then, the aggregation unit 22 aggregates the image sensor data consisting of a group of points in real space received by the management side receiving unit 21 (S4).

[0047] Then, the three-dimensional shape constructing section 23 constructs the three-dimensional shapes of the dynamic object X and the static object Y in the real space based on the image sensor data consisting of the point cloud in the real space aggregated by the aggregating section 22 (S5).

[0048] Then, the mapping processing unit 24 performs a mapping process of a predetermined image G onto a static object Y and / or a dynamic object X in the real space (S6). Specifically, the mapping processing unit 24 determines a projection position of the predetermined image G onto the dynamic object X and / or the static object Y in the real space based on the three-dimensional shapes of the static object Y and the dynamic object X in the real space constructed by the three-dimensional shape construction unit 23, and generates a predetermined image G to be projected onto the projection position. At this time, the mapping processing unit 24 may perform the mapping process when a dynamic object X (such as a vehicle or a pedestrian) is detected in the real space.

[0049] Then, the management side transmitting unit 25 transmits information about the mapping of the predetermined image G that has been subjected to the mapping process by the mapping processing unit 24 to the projection device 3 (S7).

[0050] Next, in the projection device 3, the projection side receiving unit 31 receives the information on the mapping of the predetermined image G transmitted from the management device 2 (S8).

[0051] Then, the projection unit 32 projects the specified image G onto the projection position of the static object Y and / or the dynamic object X in the real space based on the information regarding the mapping of the specified image G received by the projection side receiving unit 31 (S9).

[0052] In this embodiment, a specified image G is generated after determining the projection position in real space, but the projection position in real space may be determined after generating the specified image G, or these may be performed simultaneously.

[0053] Furthermore, although a LiDAR is used as the sensor device 1, other sensors may be used as long as they are capable of acquiring image sensor data consisting of a point cloud.

[0054] Furthermore, the three-dimensional shape constructing section 23 constructs the three-dimensional shapes of the static object Y and the dynamic object X in real space. However, it may construct only the three-dimensional shape of either the static object Y or the dynamic object X. EXAMPLES

[0055] Next, embodiments 1 to 4 of the present invention will be described with reference to FIGS.

[0056] <Example 1> In the present embodiment 1, as shown in FIG. 9, an integrated sensor device 1 and projection device 3 are provided via supports 4 at two locations in a real space related to a predetermined road.

[0057] Each sensor device 1 acquires image sensor data consisting of a cloud of points in real space relating to a static object Y (road) and a dynamic object X (vehicles and pedestrians), and transmits the image sensor data consisting of the cloud of points in real space to a management device 2.

[0058] The management device 2 receives image sensor data consisting of a cloud of points in real space transmitted from each sensor device 1, consolidates each image sensor data, and when a predetermined dynamic object X (vehicle) is detected, it determines a predetermined location (surface of the road) of a static object Y avoiding the dynamic object X (pedestrian or vehicle) in real space as the projection position of a predetermined image G, generates a predetermined image G that alerts pedestrians, and transmits them to the projection device 3 as information on mapping of the predetermined image. Note that in order to make the shape and size of the predetermined image G projected from each projection device 3 the same, a projective transformation is performed on the predetermined image G.

[0059] Each projection device 3 receives information about the mapping of a predetermined image G transmitted from the management device 2, and projects the predetermined image G in a superimposed manner onto a predetermined location (the surface of the road) of a static object Y while avoiding dynamic objects X (pedestrians and vehicles). An example of projection of the predetermined image G onto a predetermined location (the surface of the road) of a static object Y is shown in FIG. 9(b).

[0060] In this way, by projecting predetermined images G in an overlapping manner onto a predetermined location of a static object Y in real space (the surface of the road) using a plurality of projection devices 3, the predetermined image G can be projected brightly.

[0061] <Example 2> In the second embodiment, as shown in FIG. 10, an integrated sensor device 1 and projection device 3 are provided via a support 4 at one location in real space related to a predetermined road.

[0062] The sensor device 1 acquires image sensor data consisting of a cloud of points in real space relating to static objects Y (roads and walls) and dynamic objects X (vehicles and pedestrians), and transmits the image sensor data consisting of the cloud of points in real space to the management device 2.

[0063] The management device 2 receives the image sensor data in the real space transmitted from the sensor device 1, aggregates each image sensor data, and when a predetermined dynamic object X (vehicle) is detected, it determines a predetermined location of the dynamic object X (the back of pedestrian B) as the projection position of a predetermined image G, generates a predetermined image G that calls the attention of pedestrian A behind, and transmits them to the projection device 3 as information on the mapping of the predetermined image G. In particular, in this embodiment 2, the management device 2 sequentially determines the projection position of the predetermined image G so as to follow the predetermined location of the dynamic object X (the back of pedestrian B) based on the image sensor data consisting of a point cloud in the real space sequentially transmitted from the sensor device 1 by the mapping processing unit 24, and sequentially transmits information on the mapping of the predetermined image G.

[0064] Each projection device 3 sequentially receives information related to mapping of the predetermined image G transmitted from the management device 2, and sequentially projects the predetermined image G so as to follow a predetermined location (the rear of pedestrian B) of the dynamic object X. An example of projection of the predetermined image G onto the predetermined location (the rear of pedestrian B) of the dynamic object X is shown in FIG. 10(b).

[0065] In this way, by using the projection device 3 to sequentially project a predetermined image G so as to follow a predetermined location of the dynamic object X (the back of the pedestrian B), it is possible to issue a warning to other dynamic objects X (such as pedestrian A moving backward).

[0066] <Example 3> In this embodiment 3, as shown in Fig. 11, an integrated sensor device 1 and projection device 3 are installed via supports 4 at two locations in a real space related to a specific road. A large display device 5 is also installed in the real space, and various information such as advertisement information and facility information is displayed.

[0067] Each sensor device 1 acquires image sensor data consisting of a cloud of points in real space relating to a static object Y (road) and a dynamic object X (vehicles and pedestrians), and transmits the image sensor data consisting of the cloud of points in real space to a management device 2.

[0068] The management device 2 receives image sensor data consisting of a cloud of points in real space transmitted from each sensor device 1, aggregates each image sensor data, and then determines the projection position of a predetermined image G relative to the static object Y (road), generates a predetermined image G related to the information displayed on the display device 5, and transmits them to the projection device 3 as information regarding the mapping of the predetermined image G. This predetermined image G can be, for example, notification information displayed on the display device 5 to draw the attention of pedestrians ("There is a notice on the wall screen!"), but it may also be various types of information such as advertising information or facility information.

[0069] Each projection device 3 receives information about the mapping of a predetermined image G transmitted from the management device 2, and projects the predetermined image G onto a predetermined location of the static object Y (the surface of the road) in a superimposed manner.

[0070] In this way, by using the projection device 3 to project a predetermined image G related to the display device 5 onto a predetermined location of the static object Y (the surface of the road), it is possible to draw the attention of pedestrians and the like to the display device 5. <Example 4> In the fourth embodiment, as shown in FIG. 12, an integrated sensor device 1 and projection device 3 are provided via a support 4 at one location in real space related to a predetermined road.

[0071] The sensor device 1 acquires image sensor data consisting of a cloud of points in real space relating to static objects Y (roads and walls) and dynamic objects X (vehicles and pedestrians), and transmits the image sensor data consisting of the cloud of points in real space to the management device 2.

[0072] The management device 2 receives image sensor data consisting of a cloud of points in real space transmitted from each sensor device 1, and aggregates each image sensor data. When a dynamic object X (vehicle) is detected at a predetermined location in real space (for example, a predetermined road that is a blind spot behind a wall when viewed from the other dynamic object X (pedestrian)), a predetermined location of a static object Y corresponding to the position of the dynamic object X (vehicle) in real space (the surface of a wall or road on the pedestrian side) is determined as a projection location of a predetermined image G, and a predetermined image G (for example, an image imitating a vehicle) that warns pedestrians of the approach of the dynamic object X (vehicle) is generated and transmitted to the projection device 3 as information on the mapping of the predetermined image G. The predetermined location of the static object Y corresponding to the position of the dynamic object X (vehicle) in this real space may be a location where a straight line connecting one dynamic object X (vehicle) and the other dynamic object X (pedestrian) intersects with the static object Y (wall or road) or a location nearby.

[0073] In addition, the mapping processing unit 24 may determine the projection position of the specified image at any time so that the specified image G moves on a specified location of the static object Y (a wall on the pedestrian side or the surface of the road) in accordance with the movement of the dynamic object X (vehicle).

[0074] Each projection device 3 receives information related to the mapping of the predetermined image G transmitted from the management device 2, and projects the predetermined image G onto a predetermined location (a wall on the pedestrian side or the surface of the road) of the static object Y. Note that, when the projection position of the predetermined image G is determined at any time by the mapping processing unit 24 as described above, the projection device 3 projects the predetermined image G so that it moves on the predetermined location of the static object Y (a wall on the pedestrian side or the surface of the road) in accordance with the movement of the dynamic object X (vehicle).

[0075] In this way, by projecting a predetermined image G, which warns of the approach of a dynamic object X (vehicle) traveling on a road with a blind spot, by the projection device 3 onto a predetermined location (a wall on the pedestrian side or the surface of the road) of the static object Y corresponding to the position of the dynamic object X (vehicle), it is possible to warn other dynamic objects X (pedestrians or other vehicles) of the approach of the dynamic object X (vehicle).

[0076] Although the embodiment of the present invention has been described above with reference to the drawings, the present invention is not limited to the illustrated embodiment. Various modifications and variations can be made to the illustrated embodiment within the same scope as the present invention or within an equivalent scope. [Explanation of symbols]

[0077] 1...Sensor device 11…Sensor section 12...Sensor side transmitter 2…Management device 21...Administrative receiver 22…Collecting section 23…3D shape component 24...Mapping processing section 25…Administrative transmission section 3...Projection device 31...Projection side receiving unit 32…Projection section 4…Strut 5...Display device G: Predefined image X: Dynamic object Y…Static object

Claims

1. The present invention includes one or more sensor devices installed in a real space related to static objects and dynamic objects, a management device connected to the sensor devices in a communicable state, and one or more projection devices connected to the management device in a communicable state and installed in the real space, The sensor device includes: A sensor unit for acquiring image sensor data consisting of a point cloud in a real space; a sensor side transmission unit that transmits image sensor data consisting of a point cloud in real space acquired by the sensor unit to the management device, The management device includes: a management side receiving unit that receives image sensor data consisting of a point cloud in real space transmitted from the sensor device; an aggregation unit that aggregates image sensor data consisting of a cloud of points in real space received by the management side receiving unit; a three-dimensional shape constructing unit that constructs a three-dimensional shape of a static object and / or a dynamic object in real space based on image sensor data consisting of a cloud of points in real space aggregated by the aggregation unit; a mapping processing unit that performs a mapping process of a predetermined image onto a static object and / or a dynamic object in real space based on the three-dimensional shape of the static object and / or the dynamic object in real space constructed by the three-dimensional shape construction unit; a management-side transmitting unit configured to transmit information regarding mapping of a predetermined image that has been subjected to mapping processing by the mapping processing unit to the projection device, The projection device is a projection-side receiving unit that receives information regarding mapping of a predetermined image transmitted from the management device; A real space image projection system comprising: a projection unit that projects a specified image onto a static object and / or a dynamic object in real space based on information regarding mapping of the specified image received by the management side receiving unit.

2. The real space image projection system according to claim 1 , wherein the mapping processing unit determines a projection position of a predetermined image with respect to a static object and / or a dynamic object in the real space.

3. The real space image projection system according to claim 2 , wherein the mapping processing unit determines a projection position of a predetermined image onto a static object and / or a dynamic object in the real space according to a state of the dynamic object in the real space.

4. 4. The real space image projection system according to claim 3, wherein the mapping processing unit determines a projection position of a predetermined image at a predetermined location of a static object that avoids dynamic objects in the real space.

5. 4. The real space image projection system according to claim 3, wherein the mapping processing unit successively determines the projection position of a predetermined image so as to track a predetermined location of a dynamic object moving in real space.

6. The real space image projection system according to claim 3 , wherein the mapping processing unit, when detecting a predetermined dynamic object in the real space, determines a projection position of the predetermined image onto a static object and / or a dynamic object in the real space.

7. 4. The real space image projection system according to claim 3, wherein, when two or more dynamic objects are detected in the real space, the mapping processing unit determines a projection position of a predetermined image at a predetermined location of the dynamic object moving ahead.

8. The real space image projection system according to claim 1 , wherein the mapping processing unit generates a predetermined image according to a state of a dynamic object in the real space.

9. 9. The real space image projection system according to claim 8, wherein the mapping processing unit generates a predetermined image for warning of the approach of a predetermined dynamic object when the mapping processing unit detects the dynamic object in the real space.

10. The real space image projection system according to claim 9 , wherein when the mapping processing unit detects a predetermined dynamic object in the real space, the mapping processing unit determines a predetermined location of a static object corresponding to the position of the dynamic object as the projection position of the predetermined image.

11. The real space image projection system according to claim 10, wherein the mapping processing unit, when detecting a predetermined dynamic object in real space, determines a projection position of a predetermined image so that the predetermined image moves on a predetermined location of a static object in accordance with the movement of the dynamic object.

12. A display device is installed in a real space related to the static object and the dynamic object, The real space image projection system according to claim 1 , wherein the mapping processing unit generates a predetermined image related to information displayed on the display device.

Images