Radio wave propagation simulation system, radio wave propagation simulation device, radio wave propagation simulation method, and radio wave propagation simulation program

Abstract

This radio wave propagation simulation system includes: a camera that captures an image; a 3D scanner that outputs 3D data on the basis of image data captured by the camera; and a radio wave propagation simulation device that performs a radio wave propagation simulation. The radio wave propagation simulation device comprises: a 3D model creation unit that creates a 3D model of a target environment on the basis of information acquired from the camera or the 3D scanner; a transmission / reception setting unit that sets the position of a transmitter and the position of a receiver in the target environment; a parameter setting unit that sets parameters for a simulation in the target environment; a material estimation unit that estimates, on the basis of the image captured by the camera and a learning model created by machine learning, the material of an object in the image; and a material mirroring unit that causes information regarding the material estimated by the material estimation unit to be mirrored in the 3D model created by the 3D model creation unit.

Description

Radio wave propagation simulation system, radio wave propagation simulation device, radio wave propagation simulation method, and radio wave propagation simulation program

[0001] The present invention relates to a radio wave propagation simulation system, a radio wave propagation simulation device, a radio wave propagation simulation method, and a radio wave propagation simulation program.

[0002] As a method for simulating radio wave propagation, a ray tracing method, an electromagnetic field analysis, or the like is generally performed using a computer.

[0003] To obtain accurate simulation results, it is necessary to create a precise three-dimensional (3D) model of the target area. In recent years, the use of camera-based 3D scanners and LiDAR (Light Detection and Ranging) has made it relatively easy to grasp and reproduce the shape.

[0004] To obtain more accurate simulation results, it is necessary to accurately set not only the shape of the 3D model but also the material. In practice, however, it is common to use a method in which the material of all objects is unified to metal (see, for example, Non-Patent Document 1).

[0005] G. Durgin et. al., "An advanced 3D ray launching method for wireless propagation prediction," 1997 IEEE 47th Vehicular Technology Conference. Technology in Motion.

[0006] FIG. 15 is a diagram showing an example of a target area 110 for which radio wave propagation simulation is performed by the conventional radio wave propagation simulation device 100.

[0007] The radio wave propagation simulator 100 performs, for example, a radio wave propagation simulation in the target area 110. The radio wave propagation simulator 100 performs, for example, a radio wave propagation simulation using a ray tracing method, electromagnetic field analysis, or the like, using a computer (not shown).

[0008] As shown in FIG. 15, a target area 110 has, for example, a plurality of objects 120, a transmitter 130, and a receiver 140 arranged therein.

[0009] The plurality of objects 120 are, for example, buildings existing within the target area 110, and their positions, shapes, etc. are reproduced using a 3D scanner (not shown) using a camera (not shown), a LiDAR (not shown), etc. On the other hand, the material of each of the plurality of objects 120 is not reproduced.

[0010] The transmitter 130 performs wireless communication with the receiver 140 by transmitting radio waves toward the receiver 140. In the example shown in Fig. 14, the radio waves transmitted from the transmitter 130 arrive at the receiver 140 after being reflected by one of the objects 120.

[0011] The receiver 140 performs wireless communication with the transmitter 130 by receiving the radio waves transmitted from the transmitter 130. In the example shown in Fig. 15, the radio waves received by the receiver 140 are transmitted from the transmitter 130, reflected by one of the objects 120, and then arrive at the receiver 140.

[0012] Fig. 16 is a flowchart showing an example of processing of a radio wave propagation simulation performed by the conventional radio wave propagation simulation device 100 for the target area 110 shown in Fig. 15. The flowchart shown in Fig. 16 is started, for example, when a radio wave propagation simulation is performed by the radio wave propagation simulation device 100.

[0013] In step S101, the radio wave propagation simulator 100 creates, for example, a 3D model of the target area 110. The radio wave propagation simulator 100 creates the 3D model of the target area 110 using, for example, a 3D scanner using a camera, LiDAR, or the like. As a result, the positions, shapes, etc. of the multiple objects 120 are reproduced. On the other hand, the materials of the multiple objects 120 are not reproduced.

[0014] In step S102, the radio wave propagation simulator 100 sets the transmitting and receiving positions, for example, the positions of the transmitter 130 and the receiver 140 shown in FIG.

[0015] In step S103, the radio wave propagation simulator 100 sets simulation parameters. The radio wave propagation simulator 100 sets parameters such as frequency band, transmission power, etc. At this time, the material of each of the plurality of objects 120 has not been reproduced and is unknown, so the radio wave propagation simulator 100 sets the parameters assuming that the material of all of the plurality of objects 120 is metal.

[0016] In step S104, the radio wave propagation simulator 100 performs a radio wave propagation simulation using a computer (not shown). At this time, the radio wave propagation simulator 100 performs the radio wave propagation simulation using, for example, a ray tracing method or electromagnetic field analysis. This completes the processing shown in the flowchart of FIG. 16.

[0017] As described above, according to conventional radio wave propagation simulation techniques, the positions and shapes of the objects (plurality of objects 120) are reproduced and reflected in a 3D model, but the material of the objects is not reproduced, and therefore parameters are set assuming that the material of the objects is metal. On the other hand, in order to obtain more accurate simulation results using radio wave propagation simulation, it is necessary to accurately reflect not only the position and shape of the objects but also the material.

[0018] However, in order to simulate radio wave propagation more accurately using a computer, it is extremely laborious and unrealistic to understand all the materials of the target object and reflect them in a 3D model.

[0019] Therefore, the present disclosure aims to provide a technology that can easily reflect the material of an object in a 3D model by using a camera and machine learning to estimate the material of the object, thereby obtaining radio wave propagation simulation results with higher accuracy than conventional methods.

[0020] One aspect of the radio wave propagation simulation system includes a camera that captures images, a 3D scanner that outputs 3D data based on image data captured by the camera, and a radio wave propagation simulation device that performs radio wave propagation simulation. The radio wave propagation simulation device is characterized by comprising: a 3D model creation unit that creates a 3D model of a target environment based on information acquired from the camera or the 3D scanner; a transmission / reception setting unit that sets the positions of a transmitter and a receiver in the target environment; a parameter setting unit that sets parameters for simulation in the target environment; a material estimation unit that estimates the material of an object in the image based on an image captured by the camera and a learning model created by machine learning; and a material reflection unit that reflects information on the material estimated by the material estimation unit in the 3D model created by the 3D model creation unit.

[0021] A radio wave propagation simulation device according to one aspect is a radio wave propagation simulation device in a radio wave propagation simulation system having a camera that captures images, a 3D scanner that outputs 3D data based on image data captured by the camera, and a radio wave propagation simulation device that performs radio wave propagation simulation. The radio wave propagation simulation device is characterized by comprising: a 3D model creation unit that creates a 3D model of a target environment based on information acquired from the camera or the 3D scanner; a transmission / reception setting unit that sets the positions of a transmitter and a receiver in the target environment; a parameter setting unit that sets parameters for simulation in the target environment; a material estimation unit that estimates the material of an object in the image based on an image captured by the camera and a learning model created by machine learning; and a material reflection unit that reflects information on the material estimated by the material estimation unit in the 3D model created by the 3D model creation unit.

[0022] One aspect of the radio wave propagation simulation method is a radio wave propagation simulation method in a radio wave propagation simulation system having a camera that captures images, a 3D scanner that outputs 3D data based on image data captured by the camera, and a radio wave propagation simulation device that performs radio wave propagation simulation, wherein the radio wave propagation simulation device creates a 3D model of a target environment based on information acquired from the camera or the 3D scanner, sets the positions of a transmitter and a receiver in the target environment, sets simulation parameters for the target environment, estimates the material of an object in the image based on the image captured by the camera and a learning model created by machine learning, and reflects information about the estimated material in the created 3D model.

[0023] A radio wave propagation simulation program according to one aspect causes a computer to execute the processing of the radio wave propagation simulation method described above.

[0024] According to the present disclosure, by estimating the material of an object using a camera and machine learning, the material of the object can be easily reflected in a 3D model, resulting in radio wave propagation simulation results with higher accuracy than conventional methods.

[0025] 1 is a diagram showing an example of the configuration of a radio wave propagation simulation system according to a first embodiment. FIG. 1 is a flowchart showing an example of the operation (processing) of the radio wave propagation simulation device shown in FIG. 1. FIG. 2 is a diagram showing an image in which the material of an object has been estimated. FIG. 3 is a flowchart showing an example of the operation (processing) of the radio wave propagation simulation device when a 3D scanner (camera) is used. FIG. 4 is a diagram showing a specific example of the operation shown in the flowchart of FIG. 5. FIG. 6 is a flowchart showing an example of the operation (processing) of the radio wave propagation simulation device when LiDAR is used. FIG. 7 is a diagram showing a specific example of the operation shown in the flowchart of FIG. 6. FIG. 8 is a diagram showing an example of input information when a learning model creation unit in the radio wave propagation simulation device shown in FIG. 1 creates a learning model. FIG. 9 is a diagram showing an example of input and output information when a material estimation unit in the radio wave propagation simulation device shown in FIG. 1 estimates a material using a learning model. FIG. 10 is a flowchart showing an example of the operation (processing) of a radio wave propagation simulation device according to a second embodiment. FIG. 11 is a diagram showing an image in which material information of an object has been estimated. FIG. 12 is a diagram showing an example of input information when a learning model creation unit in the radio wave propagation simulation device according to the second embodiment creates a learning model. FIG. 13 is a diagram showing an example of input and output information when a material estimation unit in the radio wave propagation simulation device according to the second embodiment estimates a material using a learning model. It is a conceptual diagram showing an example of the hardware configuration of a processing circuit of the radio wave propagation simulation device in the embodiment shown in Figures 1 to 13. It is a diagram showing an example of a target area for which radio wave propagation simulation is performed by a conventional radio wave propagation simulation device. It is a flowchart showing an example of processing of radio wave propagation simulation performed by a conventional radio wave propagation simulation device for the target area shown in Figure 15.

[0026] Hereinafter, embodiments of the radio wave propagation simulation system, radio wave propagation simulation device, radio wave propagation simulation method, and radio wave propagation simulation program disclosed herein will be described with reference to the drawings. Note that in the following embodiments, the radio wave propagation simulation system will be described as an example of the radio wave propagation simulation system, radio wave propagation simulation device, radio wave propagation simulation method, and radio wave propagation simulation program disclosed herein.

[0027] <Configuration of First Embodiment> FIG. 1 is a diagram showing an example of the configuration of a radio wave propagation simulation system 1 according to the first embodiment.

[0028] 1 , the radio wave propagation simulation system 1 includes a camera 11, a 3D scanner 12, a LiDAR 13, and a radio wave propagation simulation device 30. Note that the radio wave propagation simulation system 1 may include at least one of the camera 11, the 3D scanner 12, and the LiDAR 13.

[0029] The camera 11 captures an image of the target environment in which the radio wave propagation simulation is performed, and acquires image data of the target environment (hereinafter also referred to as "image information" or simply "image"). The camera 11 may be built into the 3D scanner 12 or the LiDAR 13.

[0030] The 3D scanner 12 creates 3D data (3D model) of the target environment based on, for example, image data of the target environment captured by the camera 11. The 3D scanner 12 may have the camera 11 built in. In this case, the 3D scanner 12 creates 3D data (3D model) of the target environment by, for example, projecting light from a laser, LED (Light Emitting Diode), or the like onto the target environment and capturing and calculating the reflection time difference and irradiation angle with the built-in camera 11. The 3D model of the target environment may be created by the radio wave propagation simulation device 30 based on the image data output from the camera 11 or the 3D data output from the 3D scanner 12.

[0031] The LiDAR 13 is, for example, a sensor device that irradiates a target environment with laser light and measures the distance to a target object in the target environment and the shape of the target object based on information about the reflected light. The LiDAR 13 may have a built-in camera 11 and a 3D scanner 12. The LiDAR 13 creates 3D data (a 3D model) of the target environment based on, for example, measurement information measured by the LiDAR 13 and image data acquired by the camera 11. The 3D model of the target environment may be created by the radio wave propagation simulation device 30 based on the measurement information output from the LiDAR 13 and the image data output from the camera 11.

[0032] The radio wave propagation simulator 30 performs a radio wave propagation simulation of a target environment by using, for example, a computer 36 (described later) to perform a ray tracing method, an electromagnetic field analysis, or the like.

[0033] The radio wave propagation simulator 30 has a processor 91 (see FIG. 14 ) described below, such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), or GPU (Graphics Processing Unit), which operates by executing a program. The radio wave propagation simulator 30 also has a memory 92 (see FIG. 14 ), described below, and executes a predetermined program stored in the memory 92 to operate the processor 91 and perform radio wave propagation simulation processing.

[0034] The radio wave propagation simulator 30 functions as the following units, for example, by executing a predetermined program stored in a memory 92 (see FIG. 14 ), which will be described later. The radio wave propagation simulator 30 functions as a 3D model creation unit 31, a transmission / reception setting unit 32, a parameter setting unit 33, a material estimation unit 34, a material reflection unit 35, a calculator 36, and a learning model creation unit 37. Note that each of the above functions may be realized by a program executed by a processing unit (not shown) included in the radio wave propagation simulator 30, or by hardware.

[0035] The radio wave propagation simulation device 30 also has a storage unit 40. The radio wave propagation simulation device 30 may also have other configurations or functions, such as an information acquisition unit, a result output unit, an operation unit, a display unit, a control unit, etc., which are not shown in the figures. Other general configurations or functions of the radio wave propagation simulation device 30 will not be described or illustrated.

[0036] The 3D model creation unit 31, the transmission / reception setting unit 32, the parameter setting unit 33, the material estimation unit 34, the material reflection unit 35, the calculator 36, and the learning model creation unit 37 execute predetermined programs to perform the following processing.

[0037] The 3D model creation unit 31 creates a 3D model of a predetermined target environment (simulation environment) based on information acquired from the camera 11, the 3D scanner 12, and the LiDAR 13. If a 3D model has already been created by the above-mentioned 3D scanner 12 or LiDAR 13, the 3D model creation unit 31 acquires the created 3D model and creates the 3D model using the acquired 3D model. The 3D model creation unit may have the functions of the above-mentioned camera 11, the 3D scanner 12, and the LiDAR 13.

[0038] The 3D model creation unit 31 may also label objects included in the created 3D model. The 3D model creation unit 31 may determine and label objects included in the 3D model based on, for example, the shape, size, etc. of the objects included in the created 3D model. The 3D model creation unit 31 may also label objects included in the 3D model based on, for example, information stored in a storage unit 40 (described below), information received from a user via an operation unit (not shown), information learned in advance by machine learning, etc.

[0039] The labeling by the 3D model creation unit 31 may be performed by object detection using, for example, well-known general deep learning. That is, the labeling by the 3D model creation unit 31 may be performed by, for example, a method such as R-CNN (Regional CNN (Convolutional Neural Network)) or YOLO (You Only Look Once). The labeling by the 3D model creation unit 31 may also be performed by, for example, SSD (Single Shot multibox Detector) or DETR (DEtection Transformer). The labeling by the 3D model creation unit 31 may use the results of labeling by the material estimation unit 34 (described later) or other devices, configurations, functions, or the like.

[0040] The transmission / reception setting unit 32 sets the positions of the transmitter and the receiver in the target environment. Note that the set positions of the transmitter and the receiver may be based on, for example, information stored in a storage unit 40 (described later) or information received from a user via an operation unit (not shown).

[0041] The parameter setting unit 33 sets simulation parameters for the target environment. The simulation parameters include, for example, a frequency band, a transmission power, the number of times radio waves are reflected, whether or not diffraction occurs, the gain of a transmitting / receiving antenna, the radiation pattern of a transmitting / receiving antenna, and temperature. The simulation parameters may be based on information stored in a storage unit 40 (described later) or information received from a user via an operation unit (not shown). The simulation parameters may also be calculated by a calculation unit (not shown) based on the information stored in the storage unit 40 (described later) or information received from a user via an operation unit (not shown).

[0042] The material estimation unit 34 estimates the material of an object in the target environment based on an image captured by the camera 11. The material estimation unit 34 may also cause the camera 11 to capture a new image and estimate the material of the object in the target environment based on the newly captured image. It is assumed that the material of the object in the target environment is determined by machine learning performed by the learning model creation unit 37 based on a database in which images and materials are linked in advance, and a model capable of estimating (determining) the material has been created. Therefore, the material estimation unit 34 estimates the material of the object in the target environment based on the image and the created model.

[0043] At this time, the material estimation unit 34 may label objects included in the image. The material estimation unit 34 may determine and label objects included in the image based on, for example, the shape, size, etc. of the objects included in the image. Furthermore, the material estimation unit 34 may label objects included in the image based on, for example, information stored in a storage unit 40 (described below), information received from a user via an operation unit (not shown), information previously learned by machine learning, etc. Then, the material estimation unit 34 estimates the material of the labeled objects.

[0044] The labeling by the material estimation unit 34 may be performed by object detection using, for example, well-known general deep learning. That is, the labeling by the material estimation unit 34 may be performed by, for example, a method such as R-CNN, YOLO, SSD, or DETR. The labeling by the material estimation unit 34 may use the results of labeling by the above-mentioned 3D model creation unit 31 or other devices, configurations, functions, or the like.

[0045] The material reflecting unit 35 reflects the information on the material estimated by the material estimating unit 34 in the 3D model created by the 3D model creating unit 31. The values ​​reflected at this time are the general dielectric constant, conductivity, etc. of each material, and the material reflecting unit 35 reflects, for example, the values ​​of each material stored in the storage unit 40 in the 3D model.

[0046] The computer 36 performs a radio wave propagation simulation of the target environment based on the 3D model created by the 3D model creation unit 31 and on which material information has been reflected by the material reflection unit 35, and on the set parameters. As described above, the computer 36 performs the radio wave propagation simulation of the target environment by, for example, implementing a known ray tracing method or electromagnetic field analysis. Note that the computer 36 may be provided outside the radio wave propagation simulation device 30.

[0047] The learning model creation unit 37 creates, for example, a learning model (hereinafter also simply referred to as a "model") to be used by the material estimation unit 34. The learning model creation unit 37 performs machine learning using a database in which images and materials are linked in advance, and creates a learning model that enables the material estimation unit 34 to estimate materials based on images. The created learning model is used by the material estimation unit 34. The database used for machine learning and the created learning model may be stored in the storage unit 40.

[0048] The storage unit 40 is a volatile or non-volatile storage medium, such as a hard disk drive (HDD), a solid state drive (SSD), a dynamic random access memory (DRAM), or other semiconductor memory. The storage unit 40 is connected to each unit of the radio wave propagation simulation device 30 via a bus (system bus, not shown), for example, so that various types of information can be input and output, and each unit of the radio wave propagation simulation device 30 writes and reads various types of information to and from the storage unit 40.

[0049] The storage unit 40 also stores, for example, information (image information and measurement information) acquired from the camera 11, the 3D scanner 12, the LiDAR 13, etc., information on the created 3D model, information on various parameters received from the user via an operation unit (not shown), etc. The storage unit 40 also stores information on the material estimated by the material estimation unit 34, 3D data (3D model) in which the material information is reflected by the material reflection unit 35, learning data created by the learning model creation unit 37, etc. The storage unit 40 also stores, for example, various arithmetic expressions, coefficients, thresholds, calculation results, etc. used in calculations by the calculator 36 and other units.

[0050] The storage unit 40 may be provided outside the radio wave propagation simulation device 30 and may be connected to the radio wave propagation simulation device 30 by wire or wirelessly. The storage unit 40 may be an external storage medium such as a memory card or a DVD (Digital Versatile Disc), or may be online storage, or may be shared with a memory 92 (see FIG. 14 ) described later.

[0051] <Operation of First Embodiment> Fig. 2 is a flowchart showing an example of the operation (processing) of the radio wave propagation simulation device 30 shown in Fig. 1. The flowchart shown in Fig. 2 starts when a target environment is photographed or measured by the camera 11, the 3D scanner 12, or the LiDAR 13. Note that the flowchart shown in Fig. 2 describes an example in which the target environment is photographed by the camera 11 or the 3D scanner 12 having the camera 11.

[0052] In step S1, the 3D model creation unit 31 of the radio wave propagation simulation device 30 creates a 3D model. The 3D model creation unit 31 creates a 3D model of the target environment (simulation environment) based on information (images) acquired from, for example, the camera 11 or the 3D scanner 12. Note that the 3D model may also be created by the 3D scanner 12 using the camera 11 or the LiDAR 13. The 3D model creation unit 31 also labels objects in the created 3D model.

[0053] In step S2, the transmission / reception setting unit 32 of the radio wave propagation simulator 30 performs transmission / reception setting. For example, the transmission / reception setting unit 32 sets the positions of the transmitter and the receiver in the target environment.

[0054] In step S3, the parameter setting unit 33 of the radio wave propagation simulator 30 sets simulation parameters. The simulation parameters include, for example, frequency band, transmission power, the number of times radio waves are reflected, whether diffraction occurs, gain of the transmitting and receiving antennas, radiation patterns of the transmitting and receiving antennas, temperature, etc. The parameter setting unit 33 sets, for example, at least one of these parameters.

[0055] In step S4, the material estimation unit 34 of the radio wave propagation simulation device 30 estimates materials from the image of the target environment. The material estimation unit 34 estimates the materials of objects in the target environment, for example, by having the camera 11 take a photograph of the target environment or by using the image acquired in step S1. The material estimation unit 34 estimates the materials (e.g., plastic, metal, paper, etc.) of objects in the captured image of the target environment based on the learning model created by the learning model creation unit 37. At this time, the material estimation unit 34 labels the objects included in the image and estimates the materials of the labeled objects.

[0056] Fig. 3 is a diagram showing an image in which the material of an object has been estimated. Fig. 3 shows a state in which objects in an image of a target environment have been labeled and the material of the labeled objects has been estimated.

[0057] 3, a plurality of objects are captured in the captured image of the target environment, and each object is labeled with a corresponding name. That is, the material estimation unit 34 labels each of the objects captured in the image with a name such as speaker, monitor stand leg, monitor leg, monitor stand, tissue box, cup, coaster, and keyboard cover.

[0058] In the image, the material estimation unit 34 estimates the material of the object labeled as a speaker to be plastic, and the material of the object labeled as a monitor stand leg to be metal. Similarly, in the image, the material estimation unit 34 estimates the material of the object labeled as a monitor stand leg to be metal, and the material of the object labeled as a monitor stand to be glass. Similarly, in the image, the material estimation unit 34 estimates the material of the object labeled as a tissue box to be paper, and the material of the object labeled as a cup to be metal. Similarly, in the image, the material estimation unit 34 estimates the material of the object labeled as a coaster to be wood, and the material of the object labeled as a keyboard cover to be plastic.

[0059] Regarding the materials of objects in the target environment, for example, machine learning is performed by the learning model creation unit 37 based on a database in which images and materials are linked in advance, and a learning model capable of determining (estimating) materials is created. Therefore, the material estimation unit 34 estimates the material of each object in the target environment based on the created learning model.

[0060] 2 , in step S5, the material reflecting unit 35 of the radio wave propagation simulation device 30 reflects the material information in the 3D model. That is, the material reflecting unit 35 reflects the material information estimated in step S4 in the 3D model created in step S1. At this time, the material reflecting unit 35 associates (assigns) materials one-to-one with each label in the 3D model and the image whose material has been estimated. Furthermore, the material reflecting unit 35 may reflect the values ​​of each material in the 3D model by, for example, associating the general permittivity, conductivity, etc. of each material stored in the storage unit 40 one-to-one with each label.

[0061] In step S6, the computer 36 of the radio wave propagation simulation device 30 performs a radio wave propagation simulation of the target environment. That is, the computer 36 performs a radio wave propagation simulation of the target environment based on the 3D model created by the 3D model creation unit 31 and on which material information has been reflected by the material reflection unit 35, and on the set parameters. As described above, the computer 36 performs a radio wave propagation simulation of the target environment by, for example, implementing a known ray tracing method or electromagnetic field analysis. Then, the computer 36 (radio wave propagation simulation device 30) displays or outputs the simulation results, or stores them in the storage unit 40, and this flowchart ends.

[0062] As described above, by performing the processing of the flowchart shown in FIG. 2, it is possible to easily set materials in a radio wave propagation simulation, and to realize a radio wave propagation simulation with higher accuracy than conventional methods.

[0063] Fig. 4 is a flowchart showing an example of the operation (processing) of the radio wave propagation simulation device 30 when using the 3D scanner 12 (camera 11). The flowchart shown in Fig. 4 shows an example of the operation (processing) in steps S1, S4, and S5 in the flowchart shown in Fig. 2.

[0064] Fig. 5 is a diagram showing a specific example of the operation shown in the flowchart of Fig. 4. Steps S11 to S14 shown in Fig. 5 correspond to steps S11 to S14 of the process shown in Fig. 4.

[0065] Hereinafter, based on FIG. 4 and with reference to FIG. 5, an example of the operation (processing) of the radio wave propagation simulation device 30 when creating a 3D model to which materials are assigned using the 3D scanner 12 (camera 11) will be described.

[0066] In step S11, the target object (target environment) is photographed by the 3D scanner 12 (camera 11). In step S11, Fig. 5 shows the target object (target environment) photographed.

[0067] In step S12, the 3D model creation unit 31 of the radio wave propagation simulation device 30 creates a 3D model of the target environment based on information from the 3D scanner 12 (camera 11). The 3D model creation unit 31 also labels each object in the created 3D model. Step S12 in Figure 5 shows the created and labeled 3D model.

[0068] In step S13, the material estimation unit 34 of the radio wave propagation simulation device 30 labels each object included in the image and estimates (determines) the material (substance) of each labeled object. The material of each labeled object is estimated, for example, by the material estimation unit 34 based on a learning model that has been machine-learned in advance. Step S13 in Figure 5 shows how each object shown in the image has been labeled and its material estimated. Note that the names of each labeled object and the estimated materials of each object are the same as or similar to those shown in Figure 3, and therefore will not be described here.

[0069] In step S14, the material reflecting unit 35 of the radio wave propagation simulator 30 assigns a material to each object in the 3D model. That is, the material reflecting unit 35 reflects the information about the material estimated in step S13 in the 3D model created in step S12. Step S14 in Fig. 5 shows a 3D model in which materials are assigned (materials are reflected) in one-to-one correspondence with the labels of each object.

[0070] In this way, by performing the processes of steps S11 to S14, the material of the target object (body) is estimated using the camera 11 and machine learning, so that the material of the target object can be easily reflected in the 3D model. As a result, by performing the processes of steps S11 to S14, it is possible to obtain radio wave propagation simulation results with higher accuracy than conventional methods.

[0071] Fig. 6 is a flowchart showing an example of the operation (processing) of the radio wave propagation simulation device 30 when using the LiDAR 13. Similar to the flowchart shown in Fig. 4, the flowchart shown in Fig. 6 shows an example of the operation (processing) in steps S1, S4, and S5 in the flowchart shown in Fig. 2.

[0072] Fig. 7 is a diagram showing a specific example of the operation shown in the flowchart of Fig. 6. Steps S21 to S26 shown in Fig. 7 correspond to steps S21 to S26 of the process shown in Fig. 6.

[0073] Below, based on Figure 6 and with reference to Figure 7, we will explain an example of the operation (processing) of the radio wave propagation simulation device 30 when creating a 3D model to which materials are assigned using LiDAR 13 instead of the 3D scanner 12.

[0074] In step S21, the target object (target environment) is measured by the LiDAR 13. In step S21 of Fig. 7, the state in which the target object (target environment) is measured is shown.

[0075] In step S22, the target object (target environment) is photographed by the camera 11. Fig. 7 shows the target object (target environment) photographed in step S22.

[0076] In step S23, the 3D model creation unit 31 of the radio wave propagation simulation device 30 creates a colored 3D model of the target environment based on the information from the LiDAR 13 and the information from the camera 11. Step S23 in Fig. 5 shows the created colored 3D model.

[0077] In step S24, the material estimation unit 34 of the radio wave propagation simulation device 30 labels each object included in the image and estimates (determines) the material (substance) of each labeled object. The material of each labeled object is estimated, for example, by the material estimation unit 34 based on a learning model that has been machine-learned in advance. Step S24 in FIG. 7 shows how each object shown in the image has been labeled and its material estimated. As in step S13 in FIG. 5, the names of the labeled objects and the estimated materials of each object are the same as or similar to those shown in FIG. 3, and therefore will not be described here.

[0078] In step S25, the material reflection unit 35 of the radio wave propagation simulator 30 identifies the objects in the 3D model with the objects photographed by the camera 11. That is, the material reflection unit 35 identifies each object labeled in step S24 with each object in the colored 3D model created in step S23. Step S25 in Fig. 7 indicates that object identification is performed.

[0079] In step S26, the material reflecting unit 35 of the radio wave propagation simulator 30 assigns a material to each object in the colored 3D model. That is, the material reflecting unit 35 assigns a material to each object in the colored 3D model based on each object in the colored 3D model created in step S23, each object labeled in step S24, and each object identified in step S25. Step S26 in Fig. 7 shows a colored 3D model in which materials have been assigned (materials have been reflected) in one-to-one correspondence with the materials for the 3D models in which objects were identified in step S25.

[0080] In this way, by performing the processes of steps S21 to S26, the material of the object is estimated using the camera 11 and machine learning, so that the material of the object can be easily reflected in the 3D model. As a result, by performing the processes of steps S21 to S26, it is possible to obtain radio wave propagation simulation results with higher accuracy than conventional methods.

[0081] Furthermore, the processing of steps S21 to S26 identifies an object in a colored 3D model created using the LiDAR 13 and an object whose material has been estimated using the camera 11 and machine learning. As a result, the processing of steps S21 to S26 using the LiDAR 13 allows for more precise material assignment (material can be reflected) than the processing of steps S11 to S14 using the 3D scanner 12 (camera 11).

[0082] Fig. 8 is a diagram showing an example of input information when the learning model creation unit 37 in the radio wave propagation simulation device 30 shown in Fig. 1 creates a learning model. As described in Fig. 1, the learning model creation unit 37 creates a learning model (model) to be used in, for example, the material estimation unit 34. Fig. 8 shows an example of information (database) input when the learning model creation unit 37 performs machine learning.

[0083] 8, when the learning model creation unit 37 performs machine learning, an image is input. Then, the material of the object in the input image is input. For example, the material of the object is input.

[0084] Information about the environment and radio waves is also input, such as the frequency band, transmission power, the number of times radio waves are reflected, whether or not there is diffraction, the gain of the transmitting and receiving antennas, the radiation pattern of the transmitting and receiving antennas, and the temperature.

[0085] Based on this input information, machine learning is performed by the learning model creation unit 37, and a learning model to be used in the material estimation unit 34 is created. Note that the information (database) input in the machine learning and the created learning model may be stored in the storage unit 40.

[0086] FIG. 9 is a diagram showing an example of input and output information when the material estimation unit 34 in the radio wave propagation simulator 30 shown in FIG. 1 estimates a material using a learning model.

[0087] 9, when an image is input by the material estimation unit 34, the material of the object shown in the image is output based on the learning model created by the learning model creation unit 37. In this way, the material estimation unit 34 estimates the material of the object in the captured image (determines the material).

[0088] At this time, signal information such as the angle of incidence and frequency is also input. In this case, for example, information corresponding to the input information is output based on the learning model created by the learning model creation unit 37. In other words, the material is output according to the input of the image and signal information.

[0089] <Effects of the First Embodiment> As described above, according to the first embodiment shown in FIGS. 1 to 9 , the material of the object is estimated using the camera 11 and machine learning, so that the material of the object can be easily reflected in the 3D model, and as a result, it is possible to obtain radio wave propagation simulation results with higher accuracy than conventional methods.

[0090] 1 to 9, when the LiDAR 13 and the camera 11 are used, the material of the object can be reflected in the 3D model more precisely than when only the 3D scanner 12 (camera 11) is used. As a result, when the LiDAR 13 and the camera 11 are used, it is possible to obtain radio wave propagation simulation results with higher accuracy than when only the 3D scanner 12 (camera 11) is used.

[0091] 1 to 9, the material of an object in an image is learned by machine learning, and it is possible to output, for example, the material of the object. This makes it easier than ever to reflect the material of the object in a 3D model, and as a result, it is possible to obtain radio wave propagation simulation results that are more accurate and more suited to actual conditions than ever before.

[0092] Second Embodiment Figure 10 is a flowchart showing an example of the operation (processing) of a radio wave propagation simulation device 30 according to a second embodiment. In the second embodiment, the configuration of the radio wave propagation simulation system 1 (radio wave propagation simulation device 30) is the same as or similar to that of the first embodiment shown in Figures 1 to 9, and therefore a detailed description thereof will be omitted. In addition, in the second embodiment, the same or similar configurations or operations as those of the first embodiment shown in Figures 1 to 9 are denoted by the same reference numerals, and detailed description thereof will be omitted or simplified.

[0093] In FIG. 10, steps S1 to S3 are the same as steps S1 to S3 in the first embodiment shown in FIG. 2, and therefore a description thereof will be omitted.

[0094] In step S4A, the material estimation unit 34 of the radio wave propagation simulation device 30 estimates material information from an image of the target environment. The material estimation unit 34 estimates material information of objects in the target environment, for example, by having the camera 11 take a photograph of the target environment or by using the image acquired in step S1. The material estimation unit 34 estimates material information (e.g., numerical values ​​related to the characteristics of the material against radio waves, such as the dielectric constant and conductivity of the object at the target frequency) of objects in the image of the target environment based on the learning model created by the learning model creation unit 37. At this time, the material estimation unit 34 labels objects included in the image and estimates material information of the labeled objects. Note that the material estimation unit 34 may estimate both the material and material information of the objects.

[0095] Fig. 11 is a diagram showing an image in which material information of an object has been estimated, in which objects in an image of a target environment have been labeled and the material and material information of the labeled objects have been estimated.

[0096] 11, a plurality of objects are captured in the captured image of the target environment, and each object is labeled with a corresponding name. That is, the material estimation unit 34 labels each of the objects captured in the image with a name such as speaker, monitor stand leg, monitor leg, monitor stand, tissue box, cup, coaster, and keyboard cover.

[0097] In the image, the material estimation unit 34 estimates the material of an object labeled as a speaker to be plastic, and estimates the permittivity and conductivity as material information. Similarly, in the image, the material estimation unit 34 estimates the material of an object labeled as a monitor stand leg to be metal, and estimates the permittivity and conductivity as material information. Similarly, in the image, the material estimation unit 34 estimates the material of an object labeled as a monitor stand to be metal, and estimates the permittivity and conductivity as material information. Similarly, in the image, the material estimation unit 34 estimates the material of an object labeled as a monitor stand to be glass, and estimates the permittivity and conductivity as material information. Similarly, in the image, the material estimation unit 34 estimates the material of an object labeled as a tissue box to be paper, and estimates the permittivity and conductivity as material information. Similarly, in the image, the material estimation unit 34 estimates that the material of an object labeled as a cup is metal, and the dielectric constant and conductivity are estimated as material information. Similarly, in the image, the material estimation unit 34 estimates that the material of an object labeled as a coaster is wood, and the dielectric constant and conductivity are estimated as material information. Similarly, in the image, the material estimation unit 34 estimates that the material of an object labeled as a keyboard cover is plastic, and the dielectric constant and conductivity are estimated as material information.

[0098] Note that, for example, the learning model creation unit 37 performs machine learning based on a database in which images and material information are linked in advance to create a learning model capable of determining (estimating) material information for the objects in the target environment. Therefore, the material estimation unit 34 estimates the material information of each object in the target environment based on the created learning model.

[0099] Returning to FIG. 10, steps S5 to S6 are the same as steps S5 to S6 in the first embodiment shown in FIG. 2, and therefore a description thereof will be omitted.

[0100] In the flowcharts and specific examples of the operations of the flowcharts shown in Figures 4 to 7 in the first embodiment, the material (substance) of the object is estimated (determined), and the material (substance) is reflected (assigned) to the 3D model. On the other hand, although not shown, in the second embodiment, in Figures 4 to 7, not only the material (substance) but also material information is estimated (determined), and the material information is also reflected (assigned) to the 3D model. In Figures 4 to 7, other operations in the second embodiment are the same or similar to those in the first embodiment, and therefore description thereof will be omitted.

[0101] FIG. 12 is a diagram showing an example of input information when the learning model creation unit 37 of the radio wave propagation simulation device 30 according to the second embodiment creates a learning model.

[0102] As shown in Figure 12, when the learning model creation unit 37 performs machine learning, an image is input. Then, the material of the object in the input image is input. As the material of the object, for example, the type of material is input, as well as numerical values ​​related to the characteristics of the material against radio waves, such as the dielectric constant and conductivity at the target frequency, are input. Note that the rest is the same as or similar to Figure 8, so description thereof will be omitted.

[0103] FIG. 13 is a diagram showing an example of input and output information when the material estimation unit 34 in the radio wave propagation simulation device 30 according to the second embodiment estimates a material using a learning model.

[0104] As shown in FIG. 13 , when an image is input by the material estimation unit 34, the material characteristics of the object in the image are output based on the learning model created by the learning model creation unit 37. The output material characteristics include, for example, reflectivity, transmittance, magnetic permeability, etc., and are numerical values ​​related to the characteristics of the material against radio waves, such as permittivity and conductivity at the target frequency. In this way, the material estimation unit 34 estimates (determines) material information of the object in the captured image. The material estimation unit 34 may estimate (determine) both the material (material) and material information of the object in the captured image. Note that the rest is the same or similar to FIG. 9 , and therefore description thereof will be omitted.

[0105] <Operations and Effects of Second Embodiment> As described above, according to the second embodiment shown in FIGS. 10 to 13, the same effects as those of the first embodiment shown in FIGS. 1 to 9 are achieved.

[0106] Furthermore, according to the second embodiment shown in FIGS. 10 to 13 , material information (material characteristics) of an object in an image is machine-learned, and thus numerical values ​​related to the characteristics of the material with respect to radio waves, such as the dielectric constant and conductivity at a target frequency, as well as the material of the object, are output. That is, in the first embodiment shown in FIGS. 1 to 9 , only the general dielectric constant, conductivity, etc. of the material of the object are reflected in the 3D model. On the other hand, according to the second embodiment shown in FIGS. 10 to 13 , numerical values ​​related to the characteristics of the material with respect to radio waves, such as the dielectric constant and conductivity at a target frequency, as well as the material of the object, are output and reflected in the 3D model. Therefore, according to the second embodiment, the characteristics of the material of the object can be reflected in the 3D model more easily and accurately than in the first embodiment, and as a result, more accurate radio wave propagation simulation results that are more accurate and more suited to actual conditions can be obtained than in the first embodiment.

[0107] <Hardware Configuration Example> Fig. 14 is a conceptual diagram showing an example of the hardware configuration of the processing circuitry 90 included in the radio wave propagation simulation device 30 in the embodiment shown in Figs. 1 to 13. The functions described above are realized by the processing circuitry 90. In one aspect, the processing circuitry 90 includes at least one processor 91 and at least one memory 92. In another aspect, the processing circuitry 90 includes at least one dedicated hardware 93.

[0108] When the processing circuit 90 includes a processor 91 and a memory 92, each function is realized by software, firmware, or a combination of software and firmware. At least one of the software and firmware is written as a program. At least one of the software and firmware is stored in the memory 92. The processor 91 realizes each function by reading and executing the program stored in the memory 92.

[0109] When the processing circuitry 90 includes dedicated hardware 93, the processing circuitry 90 may be, for example, a single circuit, multiple circuits, a programmed processor, or a combination thereof. Each function is implemented by the processing circuitry 90.

[0110] Each function of the radio wave propagation simulator 30 may be partially or entirely configured by hardware, or may be configured as a program executed by a processor. In other words, the radio wave propagation simulator 30 can also be realized by a computer and a program, and the program can be stored in a storage medium or provided via a network.

[0111] <Supplementary Notes on the Embodiments> According to the embodiments shown in Figures 1 to 14, the first embodiment shown in Figures 1 to 9 and the second embodiment shown in Figures 10 to 13 are separated, but the present invention is not limited to this. Some or all of these embodiments may be combined in series or parallel. By combining the embodiments, the combined embodiment can achieve the same effects as the respective embodiments before being combined.

[0112] 1 to 14, the embodiments are divided into cases where the 3D scanner 12 (camera 11) shown in FIGS. 4 and 5 is used and cases where the LiDAR 13 shown in FIGS. 6 and 7 is used, but are not limited to these. Some or all of the operations (processing) in these cases may be combined in series or in parallel. By combining operations (processing), the combined operations (processing) can achieve the respective effects achieved by the operations (processing) before being combined.

[0113] 1 to 14, the radio wave propagation simulation system 1 including the camera 11, the 3D scanner 12, the LiDAR 13, and the radio wave propagation simulation device 30 has been described as an example of one aspect of the present disclosure, but the present disclosure is not limited to this. The form of the system configuration and the like of the present disclosure is not particularly limited.

[0114] 1 to 14, the radio wave propagation simulation system 1 has been described as an example of one aspect of the present disclosure, but the present disclosure is not limited to this. The present disclosure can also be realized as a radio wave propagation simulation method in which processing steps are performed in each part of the radio wave propagation simulation system 1 or the radio wave propagation simulation device 30.

[0115] The present disclosure can also be realized as a radio wave propagation simulation program that causes a computer to execute the processing steps in each part of the radio wave propagation simulation system 1 or the radio wave propagation simulation device 30.

[0116] The present disclosure can also be realized as a storage medium (non-transitory computer-readable storage medium) on which a radio wave propagation simulation program is stored. The radio wave propagation simulation program can be stored and distributed on removable media such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a USB (Universal Serial Bus) memory. The radio wave propagation simulation program may be uploaded to a network via a network interface (not shown) included in the radio wave propagation simulation device 30, or may be downloaded from the network and stored in the memory 92.

[0117] The features and advantages of the embodiments will be apparent from the above detailed description. It is intended that the claims encompass the features and advantages of the above-described embodiments without departing from the spirit and scope of the claims. Furthermore, any improvements and modifications will be readily apparent to those skilled in the art. Therefore, it is not intended that the scope of the inventive embodiments be limited to the above-described embodiments, and appropriate improvements and equivalents within the scope of the disclosed embodiments may be utilized.

[0118] 1...Radio wave propagation simulation system; 11...Camera; 12...3D scanner; 13...LiDAR; 30...Radio wave propagation simulation device; 31...3D model creation unit; 32...Transmission and reception setting unit; 33...Parameter setting unit; 34...Material estimation unit; 35...Material reflection unit; 36...Calculator; 37...Learning model creation unit; 40...Memory unit; 90...Processing circuit; 91...Processor; 92...Memory; 93...Hardware; 100...Radio wave propagation simulation device; 110...Target area; 120...Object; 130...Transmitter; 140...Receiver

Claims

1. A radio wave propagation simulation system having a camera for capturing images, a 3D scanner for outputting 3D data based on image data captured by the camera, and a radio wave propagation simulation device for performing radio wave propagation simulation, wherein the radio wave propagation simulation device comprises: a 3D model creation unit for creating a 3D model of a target environment based on information acquired from the camera or the 3D scanner; a transmission / reception setting unit for setting the positions of a transmitter and a receiver in the target environment; a parameter setting unit for setting parameters for a simulation in the target environment; a material estimation unit for estimating a material of an object in the image based on the image captured by the camera and a learning model created by machine learning; and a material reflection unit for reflecting information on the material estimated by the material estimation unit in the 3D model created by the 3D model creation unit.

2. A radio wave propagation simulation system as described in claim 1, characterized in that, instead of the 3D scanner, it has a LiDAR that measures the object, and the 3D model creation unit creates the 3D model of the target environment based on the image captured by the camera and information obtained from the LiDAR.

3. A radio wave propagation simulation system as described in claim 1, wherein the radio wave propagation simulation device further comprises a learning model creation unit that performs machine learning using a database in which predetermined images and materials are linked, and creates a learning model that can estimate materials based on the image captured by the camera, and the material estimation unit estimates the material of the object in the image based on the image captured by the camera and the learning model created by the machine learning by the learning model creation unit.

4. A radio wave propagation simulation system as described in claim 3, characterized in that the learning model creation unit performs machine learning using a database in which predetermined images are linked to material information to create a learning model capable of estimating material information based on the image captured by the camera, and the material estimation unit estimates material information, which is a numerical value related to the characteristics of the material of the object in the image against radio waves, based on the image captured by the camera and the learning model created by the machine learning by the learning model creation unit.

5. A radio wave propagation simulation device in a radio wave propagation simulation system having a camera that captures images, a 3D scanner that outputs 3D data based on image data captured by the camera, and a radio wave propagation simulation device that performs radio wave propagation simulation, wherein the radio wave propagation simulation device comprises: a 3D model creation unit that creates a 3D model of a target environment based on information acquired from the camera or the 3D scanner; a transmission / reception setting unit that sets the position of a transmitter and the position of a receiver in the target environment; a parameter setting unit that sets parameters for a simulation in the target environment; a material estimation unit that estimates a material of an object in the image based on the image captured by the camera and a learning model created by machine learning; and a material reflection unit that reflects information on the material estimated by the material estimation unit in the 3D model created by the 3D model creation unit.

6. A radio wave propagation simulation method in a radio wave propagation simulation system having a camera that captures images, a 3D scanner that outputs 3D data based on image data captured by the camera, and a radio wave propagation simulation device that performs a radio wave propagation simulation, wherein the radio wave propagation simulation device creates a 3D model of a target environment based on information obtained from the camera or the 3D scanner, sets the position of a transmitter and the position of a receiver in the target environment, sets simulation parameters for the target environment, estimates the material of an object in the image based on the image captured by the camera and a learning model created by machine learning, and reflects information on the estimated material in the created 3D model.

7. A radio wave propagation simulation program for causing a computer to execute the processing of the radio wave propagation simulation method according to claim 6.

Images