Machine learning device, inference device, prediction model generation method, prediction method, and program

Abstract

The purpose of the present invention is to provide a machine learning device capable of improving estimation accuracy of an attenuation map using LOS / NLOS identification. A machine learning device according to the present disclosure comprises: an image generation unit that generates an LOS image on the basis of information on a path in an LOS state in a prescribed space, and generates an NLOS image on the basis of information on a path in an NLOS state in the prescribed space; and a training unit that performs machine learning, by using the LOS image, the NLOS image, and map information indicating the disposition of a shielding object in the prescribed space as teacher data, to thereby generate a prediction model for predicting the disposition of the shielding object from the LOS image and the NLOS image.

Description

Machine learning device, inference device, prediction model generation method, prediction method, and program

[0001] The present disclosure relates to a machine learning device, an inference device, a prediction model generation method, a prediction method, and a program.

[0002] In recent years, there has been a demand to improve efficiency of operations by establishing a wireless network environment at a manufacturing site such as a factory and digitizing the operations at the manufacturing site. On the other hand, it is known that industrial machines, products, etc. are concentrated within a factory, and the radio wave propagation environment within the factory is different from the radio wave propagation environment outdoors or in a general business space. Therefore, it is necessary to appropriately grasp the radio wave propagation environment within the factory and then design and construct a wireless network within the factory.

[0003] Here, as a method for estimating the spatial distribution of signal attenuation inside a three-dimensional space such as within a factory, there is wireless tomography. Wireless tomography may also be referred to as RTI (Radio Tomographic Imaging). Wireless tomography irradiates a wireless signal within a three-dimensional space and uses the observation results. At this time, the spatial distribution of signal attenuation is called an attenuation map.

[0004] Non-Patent Document 1 discloses estimating an attenuation map of a signal within a three-dimensional space by performing wireless tomography using LOS (Line Of Sight) / NLOS (Non Line Of Sight) identification. LOS / NLOS identification is to identify whether there is a line of sight between a transmission device and a reception device that communicate within a three-dimensional space. A state with a line of sight is referred to as LOS, and a state without a line of sight is referred to as NLOS. Further, Non-Patent Document 2 describes performing wireless tomography using information on a path identified as LOS as an input to machine learning. Here, a path refers to a spatial path through which a wireless signal is transmitted.

[0005] Takahiro Matsuda, Yoshiaki Nishikawa, Eiji Takahashi, Takeo Onishi, and Toshiki Takeuchi, “Binary Radio Tomographic Imaging in Factory Environments Based on LOS / NLOS Identification,” IEEE Access, vol. 11, pp. 22418 - 22429, Mar. 2023Yoshiaki Nishikawa, Takahiro Matsuda, Eiji Takahashi, Takeo Onishi, and Toshiki Takeuchi, “Training Data Generation Utilizing LOS Identification for Estimating Spatial Loss Fields,” The 2023 IEEE 97th Vehicular Technology Conference (VTC-Spring 2023), Jun. 2023

[0006] In the attenuation map estimation methods disclosed in Non-Patent Documents 1 and 2, the attenuation map is estimated by setting the values ​​of pixels on paths determined to be LOS to 0 or close to 0. Furthermore, in the attenuation map estimation methods disclosed in Non-Patent Documents 1 and 2, the attenuation map is estimated by setting the values ​​of pixels other than those on paths determined to be LOS to 1 or close to 1. However, in methods that estimate the attenuation map using information on paths determined to be LOS, even pixels where no obstruction actually exists tend to be assigned a value of 1, indicating the presence of an obstruction, for pixels not included in the paths of the transmitting and receiving devices. As a result, there is a problem that the attenuation map cannot be accurately estimated when using the attenuation map estimation methods disclosed in Non-Patent Documents 1 and 2.

[0007] One of the purposes of this disclosure is to provide a machine learning device, inference device, predictive model generator, prediction method, and program that can improve the estimation accuracy of decay maps using LOS / NLOS identification.

[0008] The machine learning device according to this disclosure includes an image generation unit that generates an LOS image based on information about paths in a LOS state within a predetermined space and an NLOS image based on information about paths in a NLOS state within the predetermined space, and a learning unit that generates a prediction model by machine learning, using the LOS image, the NLOS image, and map information showing the arrangement of occluders within the predetermined space as training data, to predict the arrangement of occluders from the LOS image and the NLOS image.

[0009] The inference device according to this disclosure includes: an image generation unit that generates a first LOS image based on information about paths in an LOS state within an observation target space with reference to at least one observation position within the observation target space, and generates a first NLOS image based on information about paths in an NLOS state within the observation target space; and an inference unit that inputs the first LOS image and the first NLOS image to a prediction model that predicts the arrangement of occluders from the second LOS image and the second NLOS image, and outputs second map information indicating the arrangement of occluders within the observation target space, which has been trained using a second LOS image generated based on information about paths in an LOS state within a predetermined space and a second NLOS image generated based on information about paths in an NLOS state within a predetermined space as training data.

[0010] The predictive model generation method according to this disclosure generates an LOS image based on information about the path of an LOS state in a predetermined space with reference to at least one observation position in the predetermined space, generates an NLOS image based on information about the path of an NLOS state in the predetermined space, and generates a predictive model that predicts the arrangement of the occluding objects from the LOS image and the NLOS image by performing machine learning, using the LOS image, the NLOS image, and map information showing the arrangement of occluding objects in the predetermined space as training data.

[0011] The prediction method according to this disclosure generates a first LOS image based on information about the path in the LOS state within the observation target space with reference to at least one observation position within the observation target space, generates a first NLOS image based on information about the path in the NLOS state within the observation target space, inputs the first LOS image and the first NLOS image into a prediction model that predicts the arrangement of the occluding objects from the second LOS image and the second NLOS image, and outputs a second map information that indicates the arrangement of the occluding objects within the observation target space, which has been trained using a second LOS image generated based on information about the path in the LOS state within a predetermined space and a second NLOS image generated based on information about the path in the NLOS state within a predetermined space as training data.

[0012] The program relating to this disclosure causes a computer to generate an LOS image based on information about the path of an LOS state in a predetermined space with reference to at least one observation position in the predetermined space, generate an NLOS image based on information about the path of an NLOS state in the predetermined space, and generate a predictive model that predicts the arrangement of the occluding objects from the LOS image and the NLOS image by performing machine learning, using the LOS image, the NLOS image, and map information showing the arrangement of occluding objects in the predetermined space as training data.

[0013] This disclosure provides a machine learning device, inference device, predictive model generator, prediction method, and program that can improve the estimation accuracy of decay maps using LOS / NLOS identification.

[0014] Figure 1 shows an example configuration of the machine learning device according to this disclosure. Figure 2 shows the flow of the predictive model generation method executed in the machine learning device. Figure 3 shows an example configuration of the machine learning device according to this disclosure. Figure 4 shows a simplified diagram of the map information generated in the acquisition unit. Figure 5 shows an LOS image. Figure 6 shows an NLOS image. Figure 7 shows the input data and filters used in the convolution operation. Figure 8 shows the flow of the learning process executed in the learning phase. Figure 9 shows the flow of the inference process executed in the inference phase. Figure 10 shows an example configuration of the machine learning device. Figure 11 shows an example configuration of the machine learning device. Figure 12 shows an example configuration of the inference device. Figure 13 is a block diagram showing an example configuration of the machine learning device 10, etc.

[0015] (Embodiment 1) Figure 1 shows an example configuration of a machine learning device 10 according to the present disclosure. The machine learning device 10 may be a computer device that operates by having a processor execute a program stored in memory. The machine learning device 10 has an image generation unit 11 and a learning unit 12. The image generation unit 11 may be used as a means for generating images. The learning unit 12 may be used as a means for learning data.

[0016] The image generation unit 11 and the learning unit 12 may be software or modules whose processing is performed by a processor executing a program stored in memory. Alternatively, the image generation unit 11 and the learning unit 12 may be hardware such as circuits or chips. In Figure 1, the configuration in which the image generation unit 11 and the learning unit 12 are included in the machine learning device 10 is shown, but the image generation unit 11 and the learning unit 12 may be distributed and arranged on different computer devices. In other words, the components that make up the machine learning device 10 may be distributed and arranged on multiple computer devices to form a machine learning system.

[0017] The image generation unit 11 generates an LOS image showing a path in the LOS state and an NLOS image showing a path in the NLOS state, based on the LOS and NLOS determination results in a predetermined space. The predetermined space may be an actual existing space or a virtual space for performing simulations. The space may be a closed space surrounded by walls, etc., or an open space without partitions such as walls. Furthermore, the space may be a three-dimensional space.

[0018] LOS and NLOS determination results may also be called line-of-sight determination results. An LOS state is defined as a state where there is a line of sight between two points in space, and an NLOS state is defined as a state where there is no line of sight. LOS and NLOS determination results may also be determination results that indicate whether a specific point or object in space can be seen from an observation position in actual space or virtual space. Multiple locations may be used as the observation position, or only one location may be used. LOS and NLOS determination results may be determined, for example, based on whether a radio signal transmitted from a radio signal transmitting device was received by a receiving device installed at an arbitrary location in space or using an arbitrary location as its travel path, with the observation position being the radio signal transmitting device. Alternatively, the receiving device at the observation position may receive a radio signal transmitted from a transmitting device installed at an arbitrary location in space or using an arbitrary location as its travel path.

[0019] Furthermore, if the receiving device receives a direct wave of a radio signal transmitted from the transmitting device, it may be determined that the connection between the transmitting device and the receiving device is in a LOS (Low-Speed) state. Whether the radio signal received by the receiving device is a direct wave or a reflected wave may be determined, for example, by using the received power and direction of the radio signal received by the receiving device, as well as the transmitted power and direction of the radio signal transmitted by the transmitting device.

[0020] A path in an LOS state may be a path connecting two points in space that are in an LOS state. A path in an NLOS state may be a path connecting two points in space that are in an NLOS state. An LOS image is an image that explicitly shows a path in an LOS state, and an NLOS image is an image that explicitly shows a path in an NLOS state. An LOS image may be an image that shows the same space as an NLOS image.

[0021] The learning unit 12 generates a predictive model that predicts the placement of occluders from LOS images and NLOS images by performing machine learning. When generating the predictive model, the learning unit 12 uses LOS images, NLOS images, and map information showing the placement of occluders in a predetermined space as training data. The training data may also be called training data or learning data. In LOS images, areas where no occluders are placed are represented as LOS states. In NLOS images, areas where occluders are placed are represented as NLOS states.

[0022] Figure 2 shows the flow of the prediction model generation method executed in the machine learning device 10. First, the image generation unit 11 generates an LOS image showing a path in the LOS state and an NLOS image showing a path in the NLOS state based on the LOS and NLOS determination results in a predetermined space (S11).

[0023] Next, the learning unit 12 generates a predictive model that predicts the placement of occluding objects from the LOS image and NLOS image by performing machine learning (S12). When generating the predictive model, the image generation unit 11 uses the LOS image, NLOS image, and map information showing the placement of occluding objects in a predetermined space as training data.

[0024] As described above, the machine learning device 10 generates a predictive model by performing machine learning using LOS images, NLOS images, and map information showing the arrangement of occluders in a predetermined space as training data. By using LOS images and NLOS images, for example, the NLOS image can determine whether a location shown as not being in an LOS state in the LOS image is in an NLOS state. In this way, LOS images and NLOS images are used to complement each other's information. As a result, the prediction accuracy of the arrangement of occluders is higher compared to when only LOS images are used.

[0025] (Embodiment 2) Figure 3 shows an example of the configuration of a machine learning device 20 according to the present disclosure. The machine learning device 20 may be a computer device that operates by having a processor execute a program stored in memory. The machine learning device 20 has an acquisition unit 21, an LOS identification unit 22, an LOS identification image generation unit 23, a learning unit 24, and an observation unit 25. The LOS identification image generation unit 23 corresponds to the image generation unit 11 in Figure 1. The learning unit 24 corresponds to the learning unit 12 in Figure 1.

[0026] The acquisition unit 21 may be used as a means for generating map information. The LOS identification unit 22 may be used as a means for identifying LOS and NLOS. The LOS identification image generation unit 23 may be used as a means for generating LOS images and NLOS images. The learning unit 24 may be used as a means for learning data. The observation unit 25 may be used as a means for observing the space to be observed.

[0027] The acquisition unit 21, LOS identification unit 22, LOS identification image generation unit 23, learning unit 24, and observation unit 25 may be software or modules whose processing is performed by the processor executing a program stored in memory. Alternatively, the acquisition unit 21, LOS identification unit 22, LOS identification image generation unit 23, learning unit 24, and observation unit 25 may be hardware such as circuits or chips.

[0028] The acquisition unit 21 generates map information showing the arrangement of obstacles in a predetermined space. For example, the acquisition unit 21 may generate a wireless signal attenuation map. Figure 4 shows a simplified diagram of the map information 50 generated by the acquisition unit 21. The map information 50 shows obstacles 51. Although the map information 50 in Figure 4 also shows a transmitter 52 and a receiver 53, the transmitter 52 and receiver 53 are shown to illustrate the map information 50 and do not necessarily have to be shown in the actual attenuation map. Figure 4 shows one obstacle 51, two transmitters 52, and six receivers 53, but the number of each object is not limited to the number shown in Figure 4.

[0029] The shielding object 51 is an object that blocks radio waves. The shielding object 51 may be, for example, an object made of metal. Alternatively, the shielding object 51 may be a concrete wall or the like. The transmitting device 52 is a device that transmits radio signals. The shaded area indicating the shielding object 51 is shown as an area where radio signals do not propagate due to the influence of the shielding object 51. The area outside the shaded area is shown as an area where radio signals propagate. In other words, it is presumed that the shielding object 51 is present in the shaded area.

[0030] The transmitting device 52 may be, for example, a base station used for mobile communications. Alternatively, the transmitting device 52 may be an access point used in a wireless LAN (Local Area Network). Alternatively, the transmitting device 52 may be a test device capable of transmitting a wireless signal of a predetermined frequency.

[0031] The receiving device 53 is a device that receives wireless signals transmitted from the transmitting device 52. The receiving device 53 may be a computer device including, for example, a smartphone terminal or an IoT (Internet of Things) terminal. Alternatively, the receiving device 53 may be a UE (User Equipment), which is a general term for terminals used in 3GPP (registered trademark) (3rd Generation Partnership Project). Alternatively, the receiving device 53 may be a slave device used in a wireless LAN. Alternatively, the receiving device 53 may be a test device capable of receiving wireless signals of a predetermined frequency.

[0032] The transmitting device 52 and the receiving device 53 may be movable devices. For example, at least one of the transmitting device 52 and the receiving device 53 may be mounted on a vehicle, a mobile robot, etc. Alternatively, the transmitting device 52 and the receiving device 53 may be fixed in a predetermined location.

[0033] The map information 50 may include positional information for the obstacle 51, the transmitter 52, and the receiver 53. The positional information may be coordinates on a predetermined coordinate axis. In Figure 4, the map information 50 is shown as two-dimensional data, but it may also be shown as three-dimensional data.

[0034] The acquisition unit 21 may simulate the propagation of wireless signals in a closed simulation environment, such as inside a factory or other building. The simulation environment may be referred to as a virtual space. Alternatively, the simulation environment may simulate an actual existing space. An actual existing space may be referred to as a real space. Or, the acquisition unit 21 may identify the propagation of wireless signals based on the results of actually measuring the received power of wireless signals in an actual existing space.

[0035] The simulation environment also simulates the arrangement of shields 51 that block the propagation of wireless signals. For example, the acquisition unit 21 may generate simulation environments for various patterns of shields 51 by changing the position of the shields 51, the material of the shields 51, the structure of the shields 51, the size of the shields 51, etc., in the simulation environment. The acquisition unit 21 may also generate simulation environments for patterns in which the shields 51 move, patterns in which the shields 51 are fixed, or patterns in which moving and fixed shields 51 coexist.

[0036] Furthermore, the acquisition unit 21 determines the position of the transmitting device 52, which transmits a wireless signal set at an arbitrary position within the simulation environment, as the simulation observation position. The acquisition unit 21 determines the value of the received power when the receiving device 53 receives the wireless signal transmitted from the transmitting device 52 at an arbitrary position within the simulation environment, taking into account the position of the obstruction 51. In other words, the acquisition unit 21 may generate an attenuation map within the simulation environment as the wireless propagation environment. The attenuation map may also be information showing the received power distribution. The attenuation map shows the spatial distribution of signal attenuation within the simulation environment.

[0037] An attenuation map may be image data that shows the degree of attenuation or amount of attenuation of a wireless signal by differences in color. The differences in color may be shown by values ​​set for each of the multiple pixels that make up the image data. For example, one pixel may be set with 8-bit values ​​for R (Red), G (Green), and B (Blue). Alternatively, if the grayscale color set for one pixel is shown with an 8-bit value, the area with the highest attenuation may be set to 255, which represents white, and the area with the lowest attenuation may be set to 0, which represents black. The area with the highest attenuation may be rephrased as an area where radio waves do not pass through. The area with the lowest attenuation may be rephrased as an area where radio waves pass through without attenuation.

[0038] The degree of attenuation of a wireless signal may be, for example, a value that represents the ratio of the signal strength received at the measurement location to the signal strength transmitted from the simulation observation location. In other words, the greater the attenuation, the greater the degree of attenuation.

[0039] The LOS identification unit 22 performs LOS / NLOS determination based on the simulation observation position within the simulation environment used by the acquisition unit 21. LOS / NLOS determination refers to LOS determination and NLOS determination.

[0040] The LOS / NLOS determination may be based on whether or not an obstruction is located on the straight line between the transmitter 52 and the receiver 53, and also on the material or structure of the obstruction. For example, if the obstruction is an object that allows radio waves to pass through, it may be determined to be LOS even if the obstruction is located on the straight line between the transmitter 52 and the receiver 53.

[0041] Furthermore, the LOS / NLOS determination may also involve determining whether the receiving device 53 can receive the direct wave transmitted from the transmitting device 52. For example, if the receiving device 53 can receive the direct wave from the transmitting device 52, it is determined to be LOS, and if it cannot receive the direct wave, it is determined to be NLOS. In addition, the LOS / NLOS determination may be performed taking into account the influence of reflected waves received by the receiving device 53. For example, at locations where the value of the received power is greater than a predetermined value, it may be determined to be LOS, and at locations where it is less than the predetermined value, it may be determined to be NLOS. Alternatively, if the value of the received power is large, it may be assumed that the receiving device 53 was greatly affected by reflected waves, and it may be determined to be NLOS. In other words, if the value of the received power is greater than a predetermined value A and less than a predetermined value B (A < B), it may be determined to be LOS, and if it is greater than the predetermined value B, it may be determined to be NLOS.

[0042] Furthermore, LOS / NLOS determination may be performed not only on a straight line connecting the transmitting device 52 and the receiving device 53, but also based on whether or not an obstruction is placed in the area defined as the radio wave propagation path between the transmitting device 52 and the receiving device 53.

[0043] The LOS identification image generation unit 23 generates a LOS image and a NLOS image based on the LOS / NLOS determination result obtained in the LOS identification unit 22. FIG. 5 shows the LOS image. FIG. 5 shows the path determined as LOS among the paths connecting the transmission device 52 and the reception device 53. In the LOS image, the path determined as LOS may be shown using a color different from the background color. In FIG. 5, it is shown that the path passing through the location of the shielding object 51 in the map information 50 of FIG. 4 is not in the LOS state.

[0044] FIG. 6 shows the NLOS image. FIG. 6 shows the path determined as NLOS among the paths connecting the transmission device 52 and the reception device 53. In the NLOS image, the path determined as NLOS may be shown using a color different from the background color. In FIG. 6, the path passing through the location of the shielding object 51 in the map information 50 of FIG. 4 is shown as the path in the NLOS state.

[0045] Returning to FIG. 3, the learning unit 24 generates a prediction model by performing machine learning using, as teacher data, the map information 50 generated in the acquisition unit 21 and the LOS image and the NLOS image generated in the LOS identification image generation unit 23. The prediction model is a learned model used to predict map information from the LOS image and the NLOS image. The map information indicates that the position of the shielding object is a place where the radio signal does not propagate. Therefore, the map information is information indicating the position of the shielding object.

[0046] The learning unit 24 may generate a prediction model using, for example, a neural network. The neural network may be, for example, a convolutional neural network (CNN). Specifically, the learning unit 24 executes a convolution operation using the LOS image and the NLOS image as input data of the neural network. A layer that executes a convolution operation in the neural network may be referred to as a convolution layer.

[0047] Here, the convolutional operation performed in the learning unit 24 will be described using FIG. 7. FIG. 7 shows the input data and filters used in the convolutional operation.

[0048] CH1 and CH2 indicate channels. CH1 shows the LOS image, and CH2 shows the NLOS image. In FIG. 7, for ease of explanation, it is shown that the LOS image and the NLOS image are composed of 9 pixels. D1 to D9 in the LOS image indicate the data of each pixel. D11 to D19 in the NLOS image indicate the data of each pixel. Although D14, D15, D17, and D18 are not shown in FIG. 7, D14, D15, D17, and D18 exist behind D4, D5, D7, and D8, that is, at positions shifted in the Z-axis direction from D4, D5, D7, and D8. Also, D1 and D11 indicate the same position in the space. Similarly, D2 to D9 and D12 to D19 also indicate the same position in the space.

[0049] The LOS image and the NLOS image are arranged in the X-axis direction, Y-axis direction, and Z-axis direction. The Z-axis direction may also be referred to as the channel direction. That is, the input data in FIG. 7 is shown as data of a total of 18 pixels arranged in 3 columns in the X-axis direction, 3 columns in the Y-axis direction, and 2 columns in the Z-axis direction. In other words, the input data is shown as data in the form of 3×3×2.

[0050] FR#1 indicates a filter. The filter is arranged in 2 columns in the X-axis direction, 2 columns in the Y-axis direction, and 2 columns in the Z-axis direction, and has data of a total of 8 pixels. In other words, the filter is shown as data in the form of 2×2×2. FIG. 7 shows N filters of FR#1 to FR#N (N is a positive integer).

[0051] The learning unit 24 performs calculations on the input data while sliding the filter at regular intervals. For example, the learning unit 24 starts from the position where it calculates the F1 and D1 pixels of filter FR#1 and slides it one pixel in the X direction. Furthermore, the learning unit 24 moves filter FR#1 to the position where it calculates the F1 and D4 pixels and slides it one pixel in the X direction. The learning unit 24 may multiply each pixel of the filter by each pixel of the input data corresponding to the pixel of the filter and then calculate the sum of the multiplied values.

[0052] The calculation result at the corresponding position of the F1 pixel and the D1 pixel is denoted as D21, and the calculation result at the corresponding position of the F1 pixel and the D2 pixel is denoted as D22. Furthermore, the calculation result at the corresponding position of the F1 pixel and the D4 pixel is denoted as D23, and the calculation result at the corresponding position of the F1 pixel and the D5 pixel is denoted as D24. D21 to D24 are arranged in a 2x2 format in the X-axis and Y-axis directions, as shown in Figure 7. Filters FR#1 to FR#N are arranged in two columns in the Z-axis direction, which is the channel direction. Therefore, by using the calculation results D21 to D24, the relationships between the same positions in space in the LOS image and NLOS image, such as D1 and D11, are extracted. The relationships between them may be, for example, that either LOS or NLOS is shown at the same position, or that neither LOS nor NLOS is shown.

[0053] The learning unit 24 performs a convolution operation using N (where N is a positive integer) filters, and as a result, obtains N operation results. The operation results in the convolution layer may represent the features of the LOS image and the NLOS image. The learning unit 24 obtains an output result by combining the operation results in the convolution layer in a fully connected layer via a pooling layer. The learning unit 24 may calculate a matrix containing the values ​​of the operation results using FR#1 to FR#N from a matrix containing the pixel values ​​constituting the LOS image of CH1 and the pixel values ​​constituting the NLOS image of CH2. Furthermore, the learning unit 24 may output an attenuation map as map information based on the calculated matrix. Specifically, the learning unit 24 may calculate a one-channel matrix from a two-channel matrix whose elements are the pixel values ​​of each of the two images, by performing operations using the pixels of each of the two images and the filters.

[0054] The learning unit 24 optimizes the parameters used in each layer or node of the neural network so that the image generated based on the output results represents the map information input as training data. The parameters may be, for example, weights or bias values.

[0055] The learning unit 24 generates a predictive model with optimized parameters by performing machine learning using multiple pairs of map information (which serves as training data) and LOS and NLOS images. The processes performed by the learning unit 24 until it generates the predictive model are referred to as the processes performed in the learning phase.

[0056] Next, we will explain the processes performed in the inference phase using the prediction model. The observation unit 25 in Figure 3 performs LOS / NLOS determination in an actual space (hereinafter referred to as "real space") rather than a virtual space. Alternatively, the observation unit 25 may acquire the LOS / NLOS determination result from an observation device that performed the LOS / NLOS determination in real space. The LOS / NLOS determination may be performed using a wireless signal, or it may be performed by observing whether the receiving device 53 etc. can be seen from the observation position. Alternatively, the LOS / NLOS determination result may be performed in a simulation environment in which the real space is simulated by the acquisition unit 21.

[0057] The LOS identification image generation unit 23 generates LOS images and NLOS images based on the LOS / NLOS determination results in real space. The learning unit 24 outputs decay map information by inputting the LOS images and NLOS images into a prediction model.

[0058] Next, we will explain the processing flow executed in the machine learning device 20. Figure 8 shows the flow of the learning process executed in the learning phase.

[0059] First, the acquisition unit 21 generates an attenuation map showing the spatial distribution of signal attenuation of wireless signals in the simulation environment (S21). The simulation environment is a virtual space, or it may be a simulation of a real space that actually exists. The attenuation map shows the shape of the shield 51 because the difference in received power between the shield 51 and its surroundings becomes large. Therefore, the attenuation map is used as data showing the arrangement of the shield 51.

[0060] Next, the LOS identification unit 22 performs LOS / NLOS determination in the simulation environment (S22). The LOS identification unit 22 may perform LOS / NLOS determination based on the received power value when the receiving device 53 receives the wireless signal transmitted from the transmitting device 52 in the simulation environment. Furthermore, the LOS identification unit 22 may receive the LOS / NLOS determination result input from the operator of the machine learning device 20 or the like.

[0061] Next, the LOS identification image generation unit 23 generates an LOS image and an NLOS image based on the LOS / NLOS determination result (S23). The LOS image shows the path between the transmitting device 52 and the receiving device 53 that has been determined to be LOS. The NLOS image shows the path between the transmitting device 52 and the receiving device 53 that has been determined to be NLOS.

[0062] Next, the learning unit 24 generates a predictive model by performing machine learning using the attenuation map, LOS image, and NLOS image as training data (S24). The predictive model is a trained model used to predict map information from the LOS image and NLOS image. The learning unit 24 generates a predictive model in which the parameters used in each layer of the neural network are optimized by performing machine learning using multiple training data.

[0063] Figure 9 shows the flow of the inference process performed in the inference phase. First, the observation unit 25 performs LOS / NLOS determination in real space (S31). Alternatively, the observation unit 25 may obtain the LOS / NLOS determination result from an observation device that performed the LOS / NLOS determination in real space.

[0064] Next, the LOS identification image generation unit 23 generates LOS images and NLOS images based on the LOS / NLOS determination results in real space (S32). The learning unit 24 inputs the LOS images and NLOS images to the prediction model (S33). Next, the learning unit 24 outputs attenuation map information from the prediction model (S34).

[0065] As described above, the machine learning device 20 generates a predictive model that outputs an attenuation map by performing machine learning using LOS images and NLOS images. By performing machine learning using NLOS images in addition to LOS images, information in areas where the LOS state is not shown in the LOS image can be supplemented from the NLOS image. Specifically, information indicating whether an area where the LOS state is not shown in the LOS image is in an NLOS state or an area where observation or measurement has not been performed is supplemented. As a result, the prediction accuracy of the placement of occluders in the predictive model is improved compared to when only LOS images are used.

[0066] (Modified Example of Embodiment 2) Figure 10 shows an example of the configuration of the machine learning device 30. The machine learning device 30 has a configuration in which a noise reduction unit 31 is added to the machine learning device 20 of Figure 3. The noise reduction unit 31 may be a learning model generated as a result of machine learning using a neural network for the purpose of noise reduction. The noise reduction unit 31 may be a learning model that has been machine-learned using the training data used by the learning unit 24 to generate the prediction model. Alternatively, the noise reduction unit 31 may be a learning model that is not limited to the simulation environment etc. used by the learning unit 24 to generate the prediction model, but can be used in general. The neural network used for noise reduction may be, for example, a DnCNN (Denoising Convolutional Neural Network).

[0067] The noise reduction unit 31 may be used to remove noise from the LOS image and NLOS image generated by the LOS identification image generation unit 23. Alternatively, the noise reduction unit 31 may be used to remove noise from the attenuation map output from the prediction model generated by the learning unit 24. Alternatively, the noise reduction unit 31 may be used to remove noise from the LOS image and NLOS image generated by the LOS identification image generation unit 23 and the attenuation map output from the prediction model generated by the learning unit 24.

[0068] The noise reduction unit 31 removes noise from the LOS image and NLOS image, or from the attenuation map, enabling the machine learning device 30 to generate an attenuation map in which the shape of the obstruction 51 is clearly defined.

[0069] (Embodiment 3) Figure 11 shows an example configuration of the machine learning device 60. Furthermore, Figure 12 shows an example configuration of the inference device 70. In Figure 3, an example is shown in which the processing in the learning phase and the inference phase is performed by the machine learning device 20. On the other hand, the machine learning device 60 in Figure 11 performs the processing in the learning phase, and the inference device 70 in Figure 12 performs the processing in the inference phase. The machine learning device 60 and the inference device 70 may be computer devices that operate by having a processor execute a program stored in memory.

[0070] The machine learning device 60 includes an acquisition unit 61, an LOS identification unit 62, an LOS identification image generation unit 63, and a learning unit 64. The acquisition unit 61, LOS identification unit 62, LOS identification image generation unit 63, and learning unit 64 are substantially the same as the acquisition unit 21, LOS identification unit 22, LOS identification image generation unit 23, and learning unit 24 in Figure 3, so a detailed explanation is omitted.

[0071] The inference device 70 includes an observation unit 71, an LOS identification image generation unit 72, and an inference unit 73. The observation unit 71, the LOS identification image generation unit 72, and the inference unit 73 may be software or modules whose processing is performed by a processor executing a program stored in memory. Alternatively, the observation unit 71, the LOS identification image generation unit 72, and the inference unit 73 may be hardware such as circuits or chips.

[0072] The observation unit 71 corresponds to the observation unit 25 in Figure 3. Furthermore, the LOS identification image generation unit 72 corresponds to the LOS identification image generation unit 23 in Figure 3. The inference unit 73 outputs an attenuation map by inputting the LOS image and NLOS image generated by the LOS identification image generation unit 72 into the prediction model. The inference unit 73 may acquire the prediction model from the machine learning device 60 via a network.

[0073] As explained above, the processes executed in the learning phase and the processes executed in the inference phase may be executed on different devices. As a result, the processing load can be distributed compared to when the processes executed in the learning phase and the processes executed in the inference phase are executed on a single device. This makes it possible to achieve faster processing speeds, etc.

[0074] Figure 13 is a block diagram showing an example configuration of machine learning devices 10, 20, 30, 60, and 70 (hereinafter referred to as "machine learning devices 10, etc."). Referring to Figure 13, machine learning devices 10, etc. include a network interface 1201, a processor 1202, and memory 1203. The network interface 1201 may be used to communicate with a network node. The network interface 1201 may include, for example, a network interface card (NIC) compliant with the IEEE 802.3 series. IEEE stands for Institute of Electrical and Electronics Engineers.

[0075] The processor 1202 reads and executes software (computer programs) from the memory 1203 to perform processing such as that of the machine learning device 10 as described using a flowchart. The processor 1202 may be, for example, a microprocessor, an MPU, or a CPU. The processor 1202 may include multiple processors.

[0076] Memory 1203 is composed of a combination of volatile and non-volatile memory. Memory 1203 may include storage located away from the processor 1202. In this case, the processor 1202 may access memory 1203 via an I / O (Input / Output) interface, which is not shown.

[0077] In the example shown in Figure 13, memory 1203 is used to store a group of software modules. The processor 1202 can perform processing on the machine learning device 10, etc., by reading and executing these software modules from memory 1203.

[0078] As explained using Figure 13, each processor in the machine learning device 10, etc., executes one or more programs that include a set of instructions for causing the computer to perform the algorithm described in the diagram.

[0079] In the examples described above, the program includes a set of instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more of the functions described in the embodiments. The program may be stored on a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable medium or a communication medium that includes electrical, optical, acoustic or other forms of propagating signals.

[0080] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure can be made as can be understood by those skilled in the art within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0081] Each drawing is merely illustrative to illustrate one or more embodiments. Each drawing may be associated with one or more other embodiments, rather than being associated with only one specific embodiment. As those skilled in the art will understand, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings, for example, to create embodiments not explicitly shown or described. Not all features or steps shown in any one drawing to illustrate an exemplary embodiment are necessarily required, and some features or steps may be omitted. The order of steps described in any of the drawings may be changed as appropriate.

[0082] Some or all of the above embodiments may also be described as follows, but are not limited to the following: (Note 1) A machine learning device comprising: an image generation unit that generates an LOS image based on information about paths in an LOS state within a predetermined space and an NLOS image based on information about paths in an NLOS state within the predetermined space; and a learning unit that generates a prediction model for predicting the arrangement of occluders from the LOS image and the NLOS image by performing machine learning, using the LOS image, the NLOS image, and map information indicating the arrangement of occluders within the predetermined space as training data. (Note 2) The machine learning device according to Note 1, wherein the learning unit uses the LOS image and the NLOS image as input data for a neural network and the arrangement of occluders as output data for the neural network to perform machine learning on the neural network. (Note 3) The machine learning apparatus according to Note 2, wherein the learning unit uses the LOS image and the NLOS image as input data for a convolutional layer included in the neural network, and uses the arrangement of the obstruction predicted based on the features of the LOS image and the NLOS image extracted in the convolutional layer as output data for the neural network. (Note 4) The machine learning apparatus according to any one of Notes 1 to 3, wherein the learning unit outputs an attenuation map representing the spatial distribution of signal attenuation of the wireless signal as data indicating the arrangement of the obstruction. (Note 5) The machine learning apparatus according to any one of Notes 1 to 4, further comprising: an acquisition unit that acquires information regarding the propagation state of a wireless signal in a predetermined space; and a determination unit that generates LOS and NLOS determination results based on the information regarding the propagation state of the wireless signal. (Note 6) The machine learning apparatus according to Note 5, wherein the acquisition unit acquires information regarding the propagation state based on measurement results. (Note 7) The acquisition unit is a machine learning device as described in Note 5 or 6, which acquires information regarding the propagation state by simulation.(Note 8) The machine learning apparatus according to any one of Notes 1 to 7, wherein the image generation unit generates a first LOS image based on information about the path of an LOS state in the observation target space with reference to at least one observation position in the observation target space, and generates a first NLOS image based on information about the path of an NLOS state in the observation target space, and the learning unit inputs the first LOS image and the first NLOS image to the prediction model and outputs a prediction result of the arrangement of the occluding object in the observation target space from the prediction model. (Note 9) An inference device comprising: an image generation unit that generates a first LOS image based on information about paths in an LOS state within the observation target space with reference to at least one observation position within the observation target space, and generates a first NLOS image based on information about paths in an NLOS state within the observation target space; and an inference unit that inputs the first LOS image and the first NLOS image to a prediction model that predicts the arrangement of the occluding objects from the second LOS image and the second NLOS image, and outputs second map information that indicates the arrangement of the occluding objects within the observation target space, which has been trained using a second LOS image generated based on information about paths in an LOS state within a predetermined space and a second NLOS image generated based on information about paths in an NLOS state within a predetermined space as training data, and outputs second map information that indicates the arrangement of the occluding objects within the observation target space. (Note 10) A method for generating a prediction model, comprising: generating an LOS image based on information about the path of an LOS state in a predetermined space with reference to at least one observation position in the predetermined space; generating an NLOS image based on information about the path of an NLOS state in the predetermined space; and generating a prediction model by machine learning, using the LOS image, the NLOS image, and map information showing the arrangement of occluders in the predetermined space as training data, to predict the arrangement of occluders from the LOS image and the NLOS image.(Note 11) A prediction method comprising: generating a first LOS image based on information about paths in an LOS state within the observation target space with reference to at least one observation position within the observation target space; generating a first NLOS image based on information about paths in an NLOS state within the observation target space; inputting the first LOS image and the first NLOS image into a prediction model that predicts the arrangement of the occluding objects from the second LOS image and the second NLOS image, which has been trained using a second LOS image generated based on information about paths in an LOS state within a predetermined space and a second NLOS image generated based on information about paths in an NLOS state within a predetermined space as training data, and a first map information showing the arrangement of occluding objects within the predetermined space; and outputting a second map information showing the arrangement of occluding objects within the observation target space. (Note 12) A program that causes a computer to perform the following actions: generate an LOS image based on information about the path of an LOS state in a predetermined space with reference to at least one observation position in the predetermined space; generate an NLOS image based on information about the path of an NLOS state in the predetermined space; and generate a predictive model that predicts the arrangement of the occluding objects from the LOS image and the NLOS image by performing machine learning, using the LOS image, the NLOS image, and map information showing the arrangement of occluding objects in the predetermined space as training data. (Note 13) A program that causes a computer to perform the following actions: generate a first LOS image based on information about paths in an LOS state within the observation target space with reference to at least one observation position within the observation target space; generate a first NLOS image based on information about paths in an NLOS state; input the first LOS image and the first NLOS image into a prediction model that predicts the arrangement of the occluding objects from the second LOS image and the second NLOS image, which has been trained using a second LOS image generated based on information about paths in an LOS state within a predetermined space and a second NLOS image generated based on information about paths in an NLOS state within the predetermined space as training data; and output a second map information that indicates the arrangement of the occluding objects within the observation target space.

[0083] Some or all of the elements (e.g., configuration and function) described in Appendices 2 to 8 that are dependent on Appendice 1 may also be dependent on Appendices 10 and 12 in the same way as those described in Appendices 2 to 8. Some or all of the elements described in any appendice may be applied to various hardware, software, recording means, systems, and methods for recording software.

[0084] This application claims priority based on Japanese Patent Application No. 2024-193346, filed on 5 November 2024, and incorporates all of its disclosures herein.

[0085] 10 Machine learning device 11 Image generation unit 12 Learning unit 20 Machine learning device 21 Acquisition unit 22 LOS identification unit 23 LOS identification image generation unit 24 Learning unit 25 Observation unit 30 Machine learning device 31 Noise reduction unit 50 Map information 51 Obstacles 52 Transmitter 53 Receiver 60 Machine learning device 61 Acquisition unit 62 LOS identification unit 63 LOS identification image generation unit 64 Learning unit 70 Inference device 71 Observation unit 72 LOS identification image generation unit 73 Inference unit

Claims

1. A machine learning device comprising: an image generation means for generating an LOS image based on information about paths in a LOS state within a predetermined space, and an NLOS image based on information about paths in a NLOS state within the predetermined space; and a learning means for generating a predictive model that predicts the arrangement of occluding objects from the LOS image and the NLOS image by performing machine learning, using the LOS image, the NLOS image, and map information showing the arrangement of occluding objects within the predetermined space as training data.

2. The machine learning apparatus according to claim 1, wherein the learning means uses the LOS image and the NLOS image as input data to the neural network, and uses the arrangement of the occluders as output data to the neural network, and performs machine learning on the neural network.

3. The machine learning apparatus according to claim 2, wherein the learning means uses the LOS image and the NLOS image as input data to a convolutional layer included in the neural network, and performs machine learning on the neural network using the predicted arrangement of the occluder based on the features of the LOS image and the NLOS image extracted in the convolutional layer and the relationship between the same positions in the LOS image and the NLOS image as output data of the neural network.

4. The machine learning apparatus according to any one of claims 1 to 3, wherein the learning means outputs an attenuation map representing the spatial distribution of signal attenuation of a wireless signal as data indicating the arrangement of the shielding objects.

5. The machine learning apparatus according to any one of claims 1 to 3, further comprising: an acquisition means for acquiring information relating to the propagation state of a wireless signal in the predetermined space; and a determination means for generating LOS and NLOS determination results based on the information relating to the propagation state of the wireless signal.

6. The machine learning apparatus according to claim 5, wherein the acquisition means acquires information regarding the propagation state based on the measurement results.

7. The machine learning apparatus according to claim 5, wherein the acquisition means acquires information regarding the propagation state by simulation.

8. The machine learning apparatus according to any one of claims 1 to 3, wherein the image generation means generates a first LOS image based on information about the path of an LOS state in the observation target space with reference to at least one observation position in the observation target space, generates a first NLOS image based on information about the path of an NLOS state in the observation target space, and the learning means inputs the first LOS image and the first NLOS image to the prediction model and outputs a prediction result of the arrangement of the occluding object in the observation target space from the prediction model.

9. An inference device comprising: an image generation means that generates a first LOS image based on information about paths in an LOS state within the observation target space with reference to at least one observation position within the observation target space, and generates a first NLOS image based on information about paths in an NLOS state within the observation target space; and an inference means that inputs the first LOS image and the first NLOS image to a prediction model that predicts the arrangement of the occluding objects from the second LOS image and the second NLOS image, and outputs second map information indicating the arrangement of the occluding objects within the observation target space, which has been machine-trained using a second LOS image generated based on information about paths in an LOS state within a predetermined space and a second NLOS image generated based on information about paths in an NLOS state within a predetermined space as training data, and outputs second map information indicating the arrangement of the occluding objects within the observation target space.

10. A method for generating a predictive model, comprising: generating an LOS image based on information about the path of an LOS state in a predetermined space with reference to at least one observation position in the predetermined space; generating an NLOS image based on information about the path of an NLOS state in the predetermined space; and generating a predictive model that predicts the arrangement of the occluding objects from the LOS image and the NLOS image by performing machine learning, using the LOS image, the NLOS image, and map information showing the arrangement of occluding objects in the predetermined space as training data.

11. A prediction method comprising: generating a first LOS image based on information about paths in an LOS state within the observation target space with reference to at least one observation position within the observation target space; generating a first NLOS image based on information about paths in an NLOS state within the observation target space; inputting the first LOS image and the first NLOS image into a prediction model that predicts the arrangement of occluders from the second LOS image and the second NLOS image, which has been trained using a second LOS image generated based on information about paths in an LOS state within a predetermined space and a second NLOS image generated based on information about paths in an NLOS state within a predetermined space as training data, and a first map information showing the arrangement of occluders within the predetermined space; and outputting a second map information showing the arrangement of occluders within the observation target space.

12. A program that causes a computer to perform the following actions: generate an LOS image based on information about the path of an LOS state in a predetermined space with reference to at least one observation position in the predetermined space; generate an NLOS image based on information about the path of an NLOS state in the predetermined space; and generate a predictive model that predicts the arrangement of the occluding objects from the LOS image and the NLOS image by performing machine learning, using the LOS image, the NLOS image, and map information showing the arrangement of occluding objects in the predetermined space as training data.

13. A program that causes a computer to perform the following actions: generate a first LOS image based on information about paths in an LOS state within the observed space with reference to at least one observation position within the observed space; generate a first NLOS image based on information about paths in an NLOS state; input the first LOS image and the first NLOS image into a prediction model that predicts the arrangement of occluders from the second LOS image and the second NLOS image, which has been trained using a second LOS image generated based on information about paths in an LOS state within a predetermined space and a second NLOS image generated based on information about paths in an NLOS state within the predetermined space as training data; and output a second map information indicating the arrangement of occluders within the observed space.

Images