Real space image projection system

Abstract

The present invention addresses the problem of providing a real space image projection system that can easily and reliably inform diverse people moving in a real space, such as a road, about the conditions and level of danger in the real space. This real space image projection system comprises: one or a plurality of each of a sensor device 1 and a projection device 3 which are installed in a real space occupied by a static body Y and a dynamic body X; and a management device 2 connected to the sensor device 1 and the projection device 3 in a communicable state. The sensor device 1 acquires image sensor data comprising a point cloud in the real space. The management device 2 consolidates the image sensor data, said data being transmitted from the sensor device 1 and comprising the point cloud in the real space, configures the three-dimensional shapes of the static body Y and the dynamic body X in the real space, and subjects a prescribed image G to a mapping process on the basis of the three-dimensional shapes, after which the projection device projects the prescribed image G onto a prescribed portion of the static body Y and dynamic body X in the real space.

Description

Real-space image projection system

[0001] The present invention relates to a real space image projection system that projects a predetermined image onto a static or dynamic object in real space.

[0002] In recent years, smart cities have been attracting attention as a form of urban development aimed at resolving social issues by utilizing information and communication technology (ICT) to improve quality of life and promote economic circulation through the creation of new value. Smart cities are not simply digitalization; they represent a transformation similar to the evolution from mobile phones to smartphones. Furthermore, the cyber-physical systems that support smart cities involve real machines such as robots, drones, and self-driving cars that use virtual information such as 3D images to perform tasks.

[0003] To realize such smart cities, self-driving cars are already equipped with cameras and millimeter-wave radar to acquire virtual information such as the 3D images mentioned above. For example, cameras can detect most of the information necessary for driving, such as white lines, signs, other vehicles, and the surrounding environment, but they have difficulty detecting the distance to an object. Furthermore, millimeter-wave radar is a device that emits millimeter-wave radio waves with frequencies between 30 GHz and 300 GHz and measures their reflection. This makes it easy to detect the relative speed of other vehicles in the vicinity, but its directional resolution is not very high, making it difficult to detect objects with low radio wave reflectivity, such as trees.

[0004] Recently, a sensor called LiDAR (light detection and ranging) has been attracting attention alongside cameras and millimeter-wave radar. LiDAR is a type of sensor that uses laser light. It has a higher radiant flux density than radio waves, and by scanning and irradiating a target with short-wavelength laser light, it can accurately detect not only the distance to the target but also the target's position and shape. Therefore, when LiDAR is used in a real space where static objects such as walls and dynamic objects such as automobiles are present, three-dimensional information of the real space can be obtained as image sensor data consisting of a point cloud of laser light. Furthermore, when LiDAR is used in the surrounding environment, such as roadside traffic lights and streetlights, in addition to LiDAR installed in automobiles, these can transmit image sensor data to the automobile to notify the situation and dangers of blind spots on the road (see, for example, Non-Patent Document 1).

[0005] Shackleton, J., VanVoorst, B., & Hesch, J. (2010, August). Tracking people with a 360-degree lidar. In 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance (pp. 420-426). IEEE.

[0006] However, because conventional systems merely transmit image sensor data to automobiles, they can only inform automobile drivers of the conditions and dangers of blind spots on the road, but cannot inform pedestrians walking on the road other than the automobile driver, or persons on bicycles or wheelchairs of the conditions and dangers of blind spots on the road. The present invention has been made in consideration of the above-mentioned technical background, and an object of the present invention is to provide a real-space image projection system that can easily and reliably inform a wide variety of people moving in real spaces such as roads, including automobile drivers, pedestrians walking on the road, and persons on bicycles or wheelchairs, of the conditions and dangers of real spaces.

[0007] In order to achieve the above object, the present invention provides a system comprising one or more sensor devices installed in a real space relating to static objects and dynamic objects, a management device connected to the sensor devices in a state capable of communicating with the sensor devices, and one or more projection devices connected to the management device in a state capable of communicating with the management device and installed in the real space, wherein the sensor devices comprise a sensor unit that acquires image sensor data consisting of a point cloud in the real space, and a sensor-side transmission unit that transmits the image sensor data consisting of the point cloud in the real space acquired by the sensor unit to the management device, and the management device comprises a management-side receiving unit that receives the image sensor data consisting of the point cloud in the real space transmitted from the sensor devices, an aggregation unit that aggregates the image sensor data consisting of the point cloud in the real space received by the management-side receiving unit, and an image sensor unit that aggregates the image sensor data consisting of the point cloud in the real space aggregated by the aggregation unit. the projection device comprises a three-dimensional shape construction unit that constructs the three-dimensional shape of a static object and / or a dynamic object in real space based on image sensor data; a mapping processing unit that performs mapping processing of a predetermined image onto a static object and / or a dynamic object in real space based on the three-dimensional shape of the static object and / or dynamic object in real space constructed by the three-dimensional shape construction unit; and a management side transmitting unit that transmits information regarding the mapping of the predetermined image that has been mapped by the mapping processing unit to the projection device, wherein the projection device comprises a projection side receiving unit that receives information regarding the mapping of the predetermined image transmitted from the management device, and a projection unit that projects the predetermined image onto a static object and / or a dynamic object in real space based on the information regarding the mapping of the predetermined image received by the management side receiving unit.

[0008] The mapping processing unit may also determine a projection position of a predetermined image onto a static object and / or a dynamic object in real space.

[0009] The mapping processing unit may also determine a projection position of a predetermined image onto a static object and / or a dynamic object in real space, depending on the state of the dynamic object in real space.

[0010] The mapping processing unit may determine a projection position of a predetermined image at a predetermined location of a static object that avoids dynamic objects in real space.

[0011] The mapping processing unit may successively determine a projection position of a predetermined image so as to follow a predetermined location of a dynamic object moving in real space.

[0012] Furthermore, when a predetermined dynamic object is detected in the real space, the mapping processing unit may determine a projection position of the predetermined image onto a static object and / or a dynamic object in the real space.

[0013] Furthermore, when two or more dynamic objects are detected in real space, the mapping processing unit may determine a projection position of a predetermined image at a predetermined location of the dynamic object moving ahead.

[0014] The mapping processing unit may generate a predetermined image according to the state of a dynamic object in real space.

[0015] Furthermore, when a predetermined dynamic object is detected in real space, the mapping processing unit may generate a predetermined image that warns of the approach of the dynamic object.

[0016] Furthermore, when a predetermined dynamic object is detected in real space, the mapping processing unit may determine a predetermined location of a static object corresponding to the position of the dynamic object as the projection position of the predetermined image.

[0017] In addition, when the mapping processing unit detects a predetermined dynamic object in real space, it may determine the projection position of the predetermined image so that the predetermined image moves on a predetermined location of a static object in accordance with the movement of the dynamic object.

[0018] A display device may be installed in a real space related to the static object and the dynamic object, and the mapping processing unit may generate a predetermined image related to information displayed on the display device.

[0019] According to the present invention, it is possible to easily and reliably notify a wide variety of people moving through real spaces such as roads, including drivers of cars, pedestrians walking on roads, and people on bicycles or in wheelchairs, of the situation and dangers in the real space, thereby making it possible to provide a safe and secure transportation system.

[0020] FIG. 1 is a schematic diagram showing an overview of a real-space image projection system of the present invention. FIG. 2 is a block diagram showing the configuration of the real-space image projection system of FIG. 1. FIG. 3 is a block diagram showing the functions of the sensor device and projection device of FIG. 2. FIG. 4 is a block diagram showing the functions of the management device of FIG. 2. FIG. 5 is a diagram showing a three-dimensional real space related to a facility. FIG. 6 is a diagram showing an example of image sensor data by Lidar. FIG. 7 is a diagram showing a process of constructing a three-dimensional shape. FIG. 8 is a flowchart showing the operation of the real-space image projection system of FIG. 1. FIG. 9 is a schematic diagram showing an overview of a real-space image projection system according to Example 1. FIG. 10 is a schematic diagram showing an overview of a real-space image projection system according to Example 2. FIG. 11 is a schematic diagram showing an overview of a real-space image projection system according to Example 3. FIG. 12 is a schematic diagram showing an overview of a real-space image projection system according to Example 4.

[0021] Next, an embodiment of a real space image projection system (hereinafter referred to as the present system) according to the present invention will be described with reference to FIGS.

[0022] [Overall Configuration] As shown in Figures 1 and 2, this system comprises a plurality of sensor devices 1 installed in real space, a management device 2 connected to the sensor devices 1 in a state where they can communicate with each other, and a plurality of projection devices 3 connected to the management device 2 in a state where they can communicate with each other and installed in real space.

[0023] In this embodiment, the real space is mainly assumed to be an outdoor three-dimensional real space related to dynamic objects X (pedestrians, vehicles, etc.) and static objects Y (roads, walls, etc.), but it may also be other real three-dimensional spaces related to social life or the environment, such as facilities, indoors, rivers, mountains, etc.

[0024] [Configuration of sensor device 1] As shown in FIG. 1, the sensor device 1 is installed on a road via a support pole 4, and as shown in FIG. 3, it includes a sensor unit 11 that acquires image sensor data consisting of a point cloud in real space at predetermined time intervals, and a sensor-side transmission unit 12 that sequentially transmits the image sensor data consisting of the point cloud in real space acquired by the sensor unit 11 to the management device 2.

[0025] The sensor unit 11 is a so-called LiDAR (light detection and ranging) sensor. This LiDAR is a type of sensor that uses laser light, and by scanning and irradiating an object with laser light having a higher radiant flux density than radio waves and a short wavelength, it acquires image sensor data consisting of a point cloud in real space and accurately detects not only the distance to the object but also the position and shape of the object.

[0026] Examples of this LiDAR include a method of acquiring image sensor data in all directions of 360 degrees by rotating light emitters that emit multiple laser beams, and a method of acquiring image sensor data by directly irradiating laser beams within a predetermined light irradiation angle range. Furthermore, the more light emitters that emit laser beams, the higher the accuracy of the image sensor data, but the higher the cost, so an inexpensive LiDAR with a small number of light emitters may be used.

[0027] Here, an example of acquiring image sensor data consisting of a point cloud in real space using LiDAR will be described. As shown in Figure 5, a certain facility is surrounded by a ceiling, floor, and walls. Predetermined desks, chairs, and shelves are arranged in various locations, and two dynamic objects X (pedestrians) are present near the center. The facility is considered a three-dimensional real space. A LiDAR is installed in the foreground of Figure 5, and laser light is scanned and emitted from the LiDAR into the facility at a predetermined irradiation angle. In this case, the LiDAR acquires image sensor data consisting of a point cloud in real space, recognizes the dynamic object X (pedestrians) in the center, and also recognizes static objects Y (walls, floors, ceilings, desks, chairs, shelves), as shown in Figure 6. Such image sensor data consisting of a point cloud in real space acquired by LiDAR is also acquired for outdoor static objects Y (roads, walls, etc.) and dynamic objects X (vehicles, pedestrians, etc.).

[0028] When image sensor data is acquired using one LiDAR, the shadow of dynamic object X is cast, and the spatial region where the shadow is cast is left blank, making it difficult to distinguish between the spatial region related to static object Y and the spatial region related to dynamic object X. Therefore, combining image sensor data made up of point clouds acquired by multiple sensor devices 1 can reduce the influence of the shadow of dynamic object X, so it is preferable to provide multiple LiDARs so as to acquire image sensor data made up of point clouds from different directions in the same real space.

[0029] [Configuration of management device 2] The management device 2 is arranged in a state where it can communicate with the sensor device 1 via wired or wireless communication, and as shown in FIG. 4 , it includes a management-side receiving unit 21 that sequentially receives image sensor data consisting of point clouds in real space sequentially transmitted from the sensor device 1, an aggregation unit 22 that aggregates the image sensor data consisting of point clouds in real space, a three-dimensional shape construction unit 23 that constructs a three-dimensional shape in real space based on the image sensor data consisting of point clouds in real space, a mapping processing unit 24 that performs mapping processing of a predetermined image G onto a static object Y and / or a dynamic object X in real space, and a management-side transmitting unit 25 that transmits information regarding the mapping of the predetermined image G to the projection device 3.

[0030] The aggregation unit 22 aggregates image sensor data consisting of point clouds in real space received by the management side receiving unit 21, and in this embodiment, aggregates image sensor data consisting of point clouds in real space that are sequentially transmitted from each sensor device 1 by synthesizing them in a chronological order.

[0031] In addition, when aggregating image sensor data consisting of point clouds in real space received by the management side receiving unit 21, the aggregation unit 22 may perform a smoothing process on the point cloud of the image sensor data, or may separately store the image sensor data consisting of point clouds in real space before and after aggregation in a storage device not shown.

[0032] The three-dimensional shape construction unit 23 constructs three-dimensional shapes of the static object Y and the dynamic object X in real space based on the image sensor data consisting of a point cloud in real space aggregated by the aggregation unit 22. For example, when the three-dimensional shape construction unit 23 acquires a point cloud of image sensor data consisting of a point cloud in real space as shown in Fig. 7(a), the three-dimensional shape construction unit 23 performs meshing by connecting adjacent point clouds as shown in Fig. 7(b), and then constructs three-dimensional shapes (3D) of the static object Y and the dynamic object X in real space as shown in Fig. 7(c).

[0033] The method for constructing the three-dimensional shapes of the static object Y and the dynamic object X in real space by the three-dimensional shape construction unit 23 is not particularly limited, and the three-dimensional shapes may remain as the point cloud shown in FIG. 7( a) or may be in a meshed state as shown in FIG. 7( b).

[0034] The mapping processing unit 24 performs mapping processing of a predetermined image G onto a static object Y and / or a dynamic object X in real space based on the three-dimensional shape of the static object Y and / or the dynamic object X in real space constructed by the three-dimensional shape construction unit 23.

[0035] Furthermore, the mapping processing unit 24 determines a projection position of a predetermined image G on a static object Y and / or a dynamic object X in real space. For example, the mapping processing unit 24 may determine, as the projection position of the predetermined image G, a predetermined location of a static object Y in real space (such as the surface of a road or a wall), a predetermined location of a dynamic object X in real space (such as the back of a pedestrian), or a predetermined location of a static object Y in real space that avoids the dynamic object X (such as the surface of a road or a wall).

[0036] Furthermore, the mapping processing unit 24 may determine the projection position of a predetermined image G onto a static object Y and / or a dynamic object X in real space according to the state of the dynamic object X in real space. For example, when the mapping processing unit 24 detects a predetermined dynamic object (e.g., a vehicle) in real space, it may determine the projection position of a predetermined image G onto the static object Y and / or the dynamic object X in real space. Furthermore, when the mapping processing unit 24 detects two or more dynamic objects X (e.g., all pedestrians) in real space, it may determine the projection position of a predetermined image G at a predetermined location of the dynamic object X moving ahead (e.g., the back of a pedestrian). Furthermore, the mapping processing unit 24 may sequentially determine the projection position of the predetermined image G so as to follow a predetermined location of the dynamic object X (e.g., a pedestrian) moving in real space.

[0037] Furthermore, the mapping processing unit 24 generates a predetermined image G (mapping image) to be projected onto the above-mentioned projection position of the static object Y and / or the dynamic object X in real space. For example, the mapping processing unit 24 may generate an image of text or a graphic to alert pedestrians at a predetermined location of the static object Y (such as the surface of a road or a wall), generate an image of text or a graphic to alert a vehicle driver or other pedestrians at a predetermined location of the dynamic object X (such as the back of a pedestrian), or generate an image related to an advertisement or publicity to inform pedestrians, etc. Note that these predetermined images G may be either still images or videos (including images in which still images are switched sequentially).

[0038] The mapping processing unit 24 may also generate a plurality of different images assuming that a predetermined image G will be synthesized and projected by a plurality of projection devices 3, or may perform projective transformation on a predetermined image G assuming that the image will be projected onto an inclined surface such as the back of a pedestrian. In this case, the mapping processing unit 24 may use a technique called projection mapping to give materials such as images a design that matches the unevenness of the object, or three-dimensional information and surface information regarding the shape of the object, and adjust the shapes of the object to overlap when projected.

[0039] The mapping processing unit 24 may generate a predetermined image G that is set in advance, or may generate a predetermined image G according to the state of the dynamic object X in the real space.

[0040] [Configuration of Projection Device 3] As shown in Fig. 1, the projection device 3 is installed on the street via a pole 4 in a state where it can communicate with the management device 2 via wired or wireless communication, and as shown in Fig. 3, it includes a projection-side receiving unit 31 that receives information about the mapping of a predetermined image G transmitted from the management device 2, and a projection unit 32 that projects the predetermined image G at a projection position of a static object Y and / or a dynamic object X in real space. In this embodiment, the sensor device 1 and the projection device 3 are provided integrally.

[0041] The projection unit 32 is a so-called projector, and projects the predetermined image G onto a projection position of a static object Y and / or a dynamic object X in real space based on information regarding the mapping of the predetermined image G received by the projection-side receiving unit 31. For example, when the mapping processing unit 24 of the management device 2 determines a predetermined location of the static object Y (such as the surface of a road or a wall) as the projection position of the predetermined image G and generates a predetermined image G of characters or figures that alert pedestrians, the projection unit 32 projects the predetermined image G of characters or figures that alert pedestrians onto the predetermined location of the static object Y (such as the surface of a road or a wall). Furthermore, when the mapping processing unit 24 of the management device 2 sequentially determines a predetermined location of the dynamic object X (such as the back of a pedestrian) as the projection position of the predetermined image G and generates a predetermined image G of characters or figures that alert pedestrians, the projection unit 32 sequentially projects the predetermined image G of characters or figures that alert pedestrians so as to follow the predetermined location of the dynamic object X (such as the back of the pedestrian). In this embodiment, a projector is used as the projection unit 32, but other devices may be used as long as they are capable of projecting characters, figures, and the like.

[0042] [Operation of the System] Next, the operation of the system from acquiring image sensor data consisting of a point cloud in real space to projecting a predetermined image G will be described with reference to the flowchart in Fig. 8. In the following description, "step" will be abbreviated as "S".

[0043] First, in the sensor device 1, the sensor unit 11 acquires image sensor data consisting of a point cloud in three-dimensional real space relating to a static object Y (such as a road or a wall) and a dynamic object X (such as a pedestrian or a vehicle) (S1).

[0044] Then, the sensor-side transmitting unit 12 transmits image sensor data consisting of a point cloud in real space acquired by the sensor unit 11 to the management device 2 (S2).

[0045] Next, in the management device 2, the management-side receiving unit 21 receives the image sensor data consisting of a point cloud in real space transmitted from the sensor device 1 (S3).

[0046] Then, the aggregation unit 22 aggregates the image sensor data consisting of the point cloud in the real space received by the management side receiving unit 21 (S4).

[0047] Then, the three-dimensional shape constructing unit 23 constructs three-dimensional shapes of the dynamic object X and the static object Y in real space based on the image sensor data consisting of the point cloud in real space aggregated by the aggregating unit 22 (S5).

[0048] Then, the mapping processing unit 24 performs a mapping process of a predetermined image G onto a static object Y and / or a dynamic object X in real space (S6). Specifically, the mapping processing unit 24 determines a projection position of the predetermined image G onto the dynamic object X and / or the static object Y in real space based on the three-dimensional shapes of the static object Y and the dynamic object X in real space constructed by the three-dimensional shape construction unit 23, and generates the predetermined image G to be projected onto the projection position. At this time, the mapping processing unit 24 may perform the mapping process when a dynamic object X (such as a vehicle or a pedestrian) is detected in real space.

[0049] Then, the management-side transmitting unit 25 transmits information about the mapping of the predetermined image G that has been subjected to the mapping process by the mapping processing unit 24 to the projection device 3 (S7).

[0050] Next, in the projection device 3, the projection-side receiving unit 31 receives the information about the mapping of the predetermined image G transmitted from the management device 2 (S8).

[0051] Then, the projection unit 32 projects the specified image G onto the projection position of the static object Y and / or the dynamic object X in real space based on the information regarding the mapping of the specified image G received by the projection side receiving unit 31 (S9).

[0052] In this embodiment, the predetermined image G is generated after the projection position in real space is determined, but the projection position in real space may be determined after the predetermined image G is generated, or these may be performed simultaneously.

[0053] Furthermore, although LiDAR is used as the sensor device 1, other sensors may be used as long as they can acquire image sensor data consisting of a point cloud.

[0054] Furthermore, although the three-dimensional shape constructing section 23 constructs the three-dimensional shapes of the static object Y and the dynamic object X in real space, it may construct only the three-dimensional shape of either the static object Y or the dynamic object X.

[0055] Next, first to fourth embodiments of the present invention will be described with reference to FIGS.

[0056] Example 1 In Example 1, as shown in FIG. 9, an integrated sensor device 1 and projection device 3 are provided via supports 4 at two locations in real space related to a predetermined road.

[0057] Each sensor device 1 acquires image sensor data consisting of a cloud of points in real space relating to a static object Y (road) and a dynamic object X (vehicle or pedestrian), and transmits the image sensor data consisting of the cloud of points in real space to the management device 2.

[0058] The management device 2 receives image sensor data consisting of a cloud of points in real space transmitted from each sensor device 1, aggregates the image sensor data, and when a predetermined dynamic object X (vehicle) is detected, determines a predetermined location (the surface of the road) of a static object Y avoiding the dynamic object X (pedestrian or vehicle) in real space as the projection position of a predetermined image G, generates a predetermined image G that alerts pedestrians, and transmits this as information related to the mapping of the predetermined image to the projection device 3. Note that projective transformation is performed on the predetermined image G to make the shape and size of the predetermined image G projected from each projection device 3 the same.

[0059] Each projection device 3 receives information about the mapping of the predetermined image G transmitted from the management device 2, and projects the predetermined image G superimposed on a predetermined location (the surface of the road) of the static object Y while avoiding dynamic objects X (pedestrians and vehicles). An example of projection of the predetermined image G onto the predetermined location (the surface of the road) of the static object Y is shown in Fig. 9(b).

[0060] In this way, by projecting a predetermined image G onto a predetermined location of a static object Y in real space (the surface of the road) in superimposed fashion using a plurality of projection devices 3, the predetermined image G can be projected brightly.

[0061] Second Embodiment In a second embodiment, as shown in FIG. 10, an integrated sensor device 1 and projection device 3 are provided via a support 4 at one location in real space on a predetermined road.

[0062] The sensor device 1 acquires image sensor data consisting of a cloud of points in real space relating to static objects Y (roads and walls) and dynamic objects X (vehicles and pedestrians), and transmits the image sensor data consisting of the cloud of points in real space to the management device 2.

[0063] The management device 2 receives the image sensor data in real space transmitted from the sensor device 1, aggregates the image sensor data, and when a predetermined dynamic object X (vehicle) is detected, determines a predetermined location of the dynamic object X (the back of pedestrian B) as the projection position of a predetermined image G, generates a predetermined image G that alerts pedestrian A behind, and transmits this as information related to the mapping of the predetermined image G to the projection device 3. In particular, in this second embodiment, the management device 2 sequentially determines the projection position of the predetermined image G so that it follows the predetermined location of the dynamic object X (the back of pedestrian B) based on the image sensor data consisting of a point cloud in real space sequentially transmitted from the sensor device 1 by the mapping processing unit 24, and sequentially transmits information related to the mapping of the predetermined image G.

[0064] Each projection device 3 sequentially receives information related to the mapping of the predetermined image G transmitted from the management device 2, and sequentially projects the predetermined image G so as to follow the predetermined location (the back of the pedestrian B) of the dynamic object X. An example of the projection of the predetermined image G onto the predetermined location (the back of the pedestrian B) of the dynamic object X is shown in FIG. 10(b).

[0065] In this way, by using the projection device 3 to sequentially project a predetermined image G so as to follow a predetermined location of the dynamic object X (the back of the pedestrian B), it is possible to issue a warning to other dynamic objects X (such as the pedestrian A moving behind).

[0066] 11 , in this example, an integrated sensor device 1 and projection device 3 are installed via supports 4 at two locations in a real space related to a predetermined road. A large display device 5 is also installed in the real space, and various information such as advertising information and facility information is displayed on the display.

[0067] Each sensor device 1 acquires image sensor data consisting of a cloud of points in real space relating to a static object Y (road) and a dynamic object X (vehicle or pedestrian), and transmits the image sensor data consisting of the cloud of points in real space to the management device 2.

[0068] The management device 2 receives image sensor data consisting of a point cloud in real space transmitted from each sensor device 1, aggregates the image sensor data, and then determines the projection position of a predetermined image G relative to the static object Y (road), generates a predetermined image G related to the information to be displayed on the display device 5, and transmits this to the projection device 3 as information related to the mapping of the predetermined image G. This predetermined image G may be, for example, notification information displayed on the display device 5 to draw pedestrians' attention ("There is a notice on the wall screen!"), but may also be various other information such as advertising information or facility information.

[0069] Each projection device 3 receives information regarding the mapping of a predetermined image G transmitted from the management device 2, and projects the predetermined image G superimposed on a predetermined location of the static object Y (the surface of the road).

[0070] In this way, by projecting a predetermined image G related to the display device 5 onto a predetermined location (surface of the road) of the static object Y by the projection device 3, it is possible to draw the attention of pedestrians and the like to the display device 5. <Example 4> In this Example 4, as shown in Fig. 12 , an integrated sensor device 1 and projection device 3 are provided via a support 4 at one location in real space related to a predetermined road.

[0071] The sensor device 1 acquires image sensor data consisting of a cloud of points in real space relating to static objects Y (roads and walls) and dynamic objects X (vehicles and pedestrians), and transmits the image sensor data consisting of the cloud of points in real space to the management device 2.

[0072] The management device 2 receives image sensor data consisting of a cloud of points in real space transmitted from each sensor device 1 and aggregates the image sensor data. When a dynamic object X (vehicle) is detected at a predetermined location in real space (e.g., a predetermined road behind a wall that is in a blind spot from the other dynamic object X (pedestrian)), the management device 2 determines a predetermined location of a static object Y corresponding to the position of the dynamic object X (vehicle) in real space (the surface of a wall or road on the pedestrian's side) as the projection location of a predetermined image G. The management device 2 also generates a predetermined image G (e.g., an image simulating a vehicle) that warns the pedestrian of the approach of the dynamic object X (vehicle), and transmits this image G to the projection device 3 as information related to the mapping of the predetermined image G. The predetermined location of the static object Y corresponding to the position of the dynamic object X (vehicle) in real space may be a location where a line connecting one dynamic object X (vehicle) and the other dynamic object X (pedestrian) intersects with the static object Y (wall or road), or a location nearby.

[0073] In addition, the mapping processing unit 24 may determine the projection position of a specified image at any time so that a specified image G moves on a specified location of a static object Y (a wall on the pedestrian side or the surface of the road) in accordance with the movement of a dynamic object X (a vehicle).

[0074] Each projection device 3 receives information related to the mapping of the predetermined image G transmitted from the management device 2, and projects the predetermined image G onto a predetermined location (the wall on the pedestrian side or the surface of the road) of the static object Y. Note that, when the projection position of the predetermined image G is determined as needed by the mapping processing unit 24 as described above, the projection device 3 projects the predetermined image G so that it moves on the predetermined location (the wall on the pedestrian side or the surface of the road) of the static object Y in accordance with the movement of the dynamic object X (vehicle).

[0075] In this way, by projecting a predetermined image G, which warns of the approach of a dynamic object X (vehicle) traveling on a road that has a blind spot, by the projection device 3 onto a predetermined location (the wall on the pedestrian side or the surface of the road) of a static object Y that corresponds to the position of the dynamic object X (vehicle), it is possible to warn other dynamic objects X (pedestrians or other vehicles) of the approach of the dynamic object X (vehicle).

[0076] Although the embodiments of the present invention have been described above with reference to the drawings, the present invention is not limited to the illustrated embodiments. Various modifications and variations can be made to the illustrated embodiments within the same scope as the present invention or within an equivalent scope.

[0077] DESCRIPTION OF SYMBOLS 1...Sensor device 11...Sensor section 12...Sensor side transmission section 2...Management device 21...Management side reception section 22...Collecting section 23...Three-dimensional shape construction section 24...Mapping processing section 25...Management side transmission section 3...Projection device 31...Projection side reception section 32...Projection section 4...Support 5...Display device G...Predetermined image X...Dynamic object Y...Static object

Claims

1. A system comprising: one or more sensor devices installed in a real space relating to static objects and dynamic objects; a management device connected to the sensor devices in a communicable state; and one or more projection devices connected to the management device in a communicable state and installed in the real space, wherein the sensor devices comprise: a sensor unit that acquires image sensor data consisting of a point cloud in the real space; and a sensor side transmission unit that transmits the image sensor data consisting of the point cloud in the real space acquired by the sensor unit to the management device, wherein the management device comprises: a management side receiving unit that receives the image sensor data consisting of the point cloud in the real space transmitted from the sensor devices; an aggregation unit that aggregates the image sensor data consisting of the point cloud in the real space received by the management side receiving unit; a three-dimensional shape construction unit that constructs a three-dimensional shape of the static object and / or dynamic object in the real space based on the image sensor data consisting of the point cloud in the real space aggregated by the aggregation unit; and a mapping processing unit that performs mapping processing of a predetermined image for the static object and / or dynamic object in the real space based on the three-dimensional shape of the static object and / or dynamic object in the real space constructed by the three-dimensional shape construction unit. a management-side transmitting unit that transmits to the projection device information regarding the mapping of a specified image for which mapping processing has been performed by the mapping processing unit, wherein the projection device comprises: a projection-side receiving unit that receives the information regarding the mapping of the specified image transmitted from the management device; and a projection unit that projects the specified image onto a static object and / or a dynamic object in real space based on the information regarding the mapping of the specified image received by the management-side receiving unit.

2. The real space image projection system according to claim 1, wherein the mapping processing unit determines a projection position of a predetermined image onto a static object and / or a dynamic object in the real space.

3. A real space image projection system according to claim 2, wherein the mapping processing unit determines the projection position of a predetermined image onto a static object and / or a dynamic object in the real space according to the state of the dynamic object in the real space.

4. A real space image projection system according to claim 3, wherein the mapping processing unit determines the projection position of a predetermined image at a predetermined location of a static object in real space, avoiding dynamic objects.

5. A real space image projection system according to claim 3, wherein said mapping processing unit successively determines the projection position of a predetermined image so as to track a predetermined location of a dynamic object moving in real space.

6. A real space image projection system as described in claim 3, wherein the mapping processing unit, when detecting a predetermined dynamic object in the real space, determines a projection position of a predetermined image onto a static object and / or a dynamic object in the real space.

7. A real space image projection system as described in claim 3, wherein said mapping processing unit, when detecting two or more dynamic objects in real space, determines a projection position of a predetermined image at a predetermined location of a dynamic object moving in front.

8. The real space image projection system according to claim 1, wherein the mapping processing unit generates a predetermined image according to the state of a dynamic object in the real space.

9. The real space image projection system according to claim 8, wherein said mapping processing unit generates a predetermined image for warning of the approach of a predetermined dynamic object when the mapping processing unit detects the dynamic object in the real space.

10. A real space image projection system as described in claim 9, wherein when the mapping processing unit detects a predetermined dynamic object in real space, it determines a predetermined location of a static object corresponding to the position of the dynamic object as the projection position of a predetermined image.

11. A real space image projection system as described in claim 10, wherein the mapping processing unit, when detecting a predetermined dynamic object in real space, determines a projection position of a predetermined image so that the predetermined image moves on a predetermined location of a static object in accordance with the movement of the dynamic object.

12. A real space image projection system as described in claim 1, wherein a display device is installed in a real space related to a static object and a dynamic object, and the mapping processing unit generates a predetermined image related to information displayed on the display device.

Images