Information processing device, information processing method, and program

Abstract

This information processing device is provided with a first generation unit, a second generation unit, and an identification unit. On the basis of each of a plurality of pieces of observation information obtained by using radio waves to observe a target region under different observation conditions, the first generation unit generates a plurality of pieces of reliability information including reliability indicating the accuracy that an object is present at each position of the target region represented using a common coordinate system. The second generation unit generates reliability map information indicating integrated reliability for each position on the basis of a plurality of levels of reliability at each position of the target region. The identification unit identifies an existence region of the object by processing the reliability map information.

Description

Information processing device, information processing method, and recording medium

[0001] The present disclosure relates to an information processing device, an information processing method, and a recording medium.

[0002] Patent Document 1 discloses a technology in which, when different position data are detected for the same target object, the entire device determines a single position data corresponding to the target object as representative position data.

[0003] The surrounding object detection device for a moving body described in Patent Document 1 is mounted on the moving body and detects objects present around the moving body.

[0004] The device described in Patent Document 1 includes a plurality of detection means, each of which includes a sensor means, a position detection means, and a confidence factor calculation means, and an integrated coordinate calculation means.

[0005] The sensor means detects objects present around the mobile body. The position detection means calculates relative position data between the object detected by the sensor means and the mobile body. The certainty calculation means calculates certainty data of the relative position data between the mobile body and the object based on the relative position data calculated by the position detection means.

[0006] The integrated coordinate calculation means weights the relative position data obtained by each detection means based on the confidence data obtained by each detection means, and determines the relative position between the moving body and the object based on each weighted relative position data.

[0007] In the technology described in Patent Document 1, as described in the same document, the likelihood that each detected position actually indicates the position of the target is calculated for each of the obtained multiple value data as a certainty factor, and this is used as an index when calculating representative position coordinate data.

[0008] Japanese Patent Application Laid-Open No. 2006-90957

[0009] However, Patent Document 1 only discloses a technique for determining a representative position of an object detected by a sensor means, but does not disclose a technique for identifying the area in which the object exists, which poses a problem that it is difficult to accurately identify the area in which the object exists.

[0010] The information processing device of the present disclosure includes: a first generating means for generating a plurality of pieces of reliability information including a reliability indicating the likelihood that an object exists at each position in a target area expressed using a common coordinate system, based on each of a plurality of pieces of observation information obtained by observing the target area using radio waves under different observation conditions; a second generating means for generating reliability map information indicating an integrated reliability for each position, based on the plurality of reliabilities at each position in the target area; and an identifying means for identifying the area in which the object exists by processing the reliability map information.

[0011] The information processing method of the present disclosure includes one or more computers generating a plurality of pieces of reliability information, each including a reliability indicating the likelihood that an object exists at each position in the target area expressed using a common coordinate system, based on each of a plurality of pieces of observation information obtained by observing the target area using radio waves under different observation conditions; generating reliability map information indicating an integrated reliability for each position based on the plurality of reliabilities at each position in the target area; and identifying the area in which the object exists by processing the reliability map information.

[0012] The recording medium in the present disclosure has recorded thereon a program for causing one or more computers to perform the following operations: generate, based on each of a plurality of pieces of observation information obtained by observing a target area using radio waves under different observation conditions, a plurality of pieces of reliability information including a reliability indicating the likelihood that an object exists at each position in the target area expressed using a common coordinate system; generate, based on the plurality of reliabilities at each position in the target area, reliability map information indicating an integrated reliability for each position; and identify the area in which the object exists by processing the reliability map information.

[0013] According to the present disclosure, it is possible to accurately identify the area in which an object exists.

[0014] FIG. 1 is a block diagram illustrating a configuration example of a first information processing device according to the present disclosure. FIG. 2 is a flowchart illustrating an example of a processing operation performed by the first information processing device according to the present disclosure. FIG. 3 is a diagram illustrating an example of a configuration of a first information processing system according to the present disclosure. FIG. 4 is a block diagram illustrating an example of a configuration of a moving body according to the present disclosure. FIG. 5 is a block diagram illustrating a detailed configuration example of the first information processing device according to the present disclosure. FIG. 6 is a flowchart illustrating a detailed example of a processing operation performed by the first information processing device according to the present disclosure. FIG. 7 is a diagram illustrating an example of reliability information according to the present disclosure. FIG. 8 is a diagram illustrating an example of processing for generating reliability map information according to the present disclosure. FIG. 9 is a diagram illustrating an example of a target area map according to the present disclosure. FIG. 10 is a block diagram illustrating an example of a physical configuration of a first information processing device according to the present disclosure. FIG. 11 is a block diagram illustrating an example of a configuration of an identification unit according to the present disclosure. FIG. 12 is a flowchart illustrating an example of a processing operation of the identification unit according to the present disclosure. FIG. 13 is a block diagram illustrating an example of a configuration of a second information processing device according to the present disclosure.

[0015] Hereinafter, in this disclosure, the drawings relate to one or more embodiments. In addition, in all drawings, similar components are given similar reference numerals and descriptions thereof will be omitted as appropriate.

[0016] First Embodiment (Overview) As shown in FIG. 1, an information processing device 100 includes a first generating unit 110, a second generating unit 120, and an identifying unit 130.

[0017] The first generation unit 110 generates multiple pieces of reliability information, including a reliability indicating the likelihood that a target object exists at each position in the target area expressed using a common coordinate system, based on each piece of observation information obtained by observing the target area using radio waves under different observation conditions.

[0018] The second generating unit 120 generates reliability map information indicating an integrated reliability for each position in the target area based on a plurality of reliabilities at the positions.

[0019] The identification unit 130 processes the reliability map information to identify the area where the object exists.

[0020] According to the information processing device 100, an integrated reliability for each position can be obtained from the results of observing a target area under different observation conditions, and the target area can be identified using this integrated reliability. Therefore, the target area can be identified with high accuracy.

[0021] The information processing device 100 executes information processing as shown in FIG.

[0022] The first generation unit 110 generates multiple pieces of reliability information, including a reliability indicating the likelihood that an object exists at each position in the target area expressed using a common coordinate system, based on each piece of observation information obtained by observing the target area using radio waves under different observation conditions (step S110).

[0023] The second generating unit 120 generates reliability map information indicating the integrated reliability for each position in the target area based on the multiple reliability at each position (step S120).

[0024] The identification unit 130 processes the reliability map information to identify the area where the object exists (step S130).

[0025] This information processing method obtains an integrated reliability for each position from the results of observing the target area under different observation conditions, and uses this to identify the area where the target exists. Therefore, it is possible to identify the area where the target exists with high accuracy.

[0026] (Detailed Example) Hereinafter, a detailed example of the information processing device 100 etc. will be described.

[0027] (Regarding target area and target object) The target area is an area observed using radio waves and may be predetermined. In the following, an example in which the target area is a three-dimensional area will be described, but it may also be a two-dimensional area (e.g., an area on a plane). Furthermore, for example, the target area may be outdoors or indoors. The target area may include, for example, an area in which a predetermined target object may exist.

[0028] In more detail, for example, the target area may include a predetermined area near the ground surface (i.e., a predetermined range from the ground surface). The predetermined area near the ground surface may be, for example, an area where an object such as a metal object may be placed on the ground surface or may be partially or completely buried underground. By setting such a target area, it is possible to detect an object such as a metal object.

[0029] Furthermore, for example, the target area may be a predetermined area of ​​a building or the like. By setting such a target area, it is possible to detect the state of pipes, reinforcing bars, etc., arranged inside or outside the walls of the building or the like, and to detect abnormalities in the target objects such as pipes, reinforcing bars, etc. Examples of buildings include, but are not limited to, buildings and bridges.

[0030] The target object is preferably an object whose surface includes a material with a high radio wave reflection strength, such as metal, etc. However, the target object is not limited to such an object and may be set as appropriate.

[0031] (Electromagnetic Waves) The radio waves are of a frequency used in general radar, and are millimeter waves with wavelengths of 1 to 10 mm (millimeters).

[0032] The radio waves are not limited to millimeter waves and may be radio waves of various wavelengths, such as microwaves with wavelengths longer than light. Microwaves are radio waves with wavelengths of 1 meter or less, such as ultrashort waves, centimeter waves, millimeter waves, and submillimeter waves. Ultrashort waves, centimeter waves, and submillimeter waves have wavelengths of 0.1 to 1 m (meters), 1 to 10 cm (centimeters), and 0.1 to 1 mm, respectively.

[0033] (Configuration Example of Information Processing System S1) A configuration example of the information processing system S1 including the information processing device 100 is shown in Fig. 3. The information processing system S1 includes, for example, a plurality of mobile objects 180 and the information processing device 100.

[0034] (Regarding the mobile object 180) Each of the multiple mobile objects 180 is an air vehicle such as a drone that moves by remote control or operation by an operator or automatically according to a predetermined algorithm, etc. Each of the multiple mobile objects 180 generates observation information by observing a target area using radio waves under different observation conditions that are determined as appropriate.

[0035] More specifically, for example, each of the plurality of mobile objects 180 includes a transmitting unit 181, a receiving unit 182, a transmitting unit 183, a movement control unit 184, and an observation control unit 185, as shown in FIG.

[0036] The transmitter 181 transmits radio waves to a target area. The receiver 182 receives reflected waves of the transmitted radio waves and generates observation information related to the reflected waves. The transmitter 181 and the receiver 182 each include a sensor (e.g., an antenna) for transmitting and receiving radio waves. Note that the transmitter 181 and the receiver 182 may each include a sensor for transmitting radio waves and a sensor for receiving radio waves.

[0037] Various common methods may be applied to the method of transmitting the radio waves, including frequency-controlled modulation (FMCW), pulse, continuous wave Doppler (CWD), two-frequency CW, and pulse compression, as examples of the method of transmitting the electromagnetic waves of the first frequency.

[0038] The transmitter 183 transmits the generated observation information.

[0039] The movement control unit 184 controls the movement of the moving body 180 .

[0040] The observation control unit 185 controls the observation direction, the time to transmit radio waves, etc. The observation direction is, for example, the direction in which radio waves are emitted. However, the observation direction is not limited to this.

[0041] Each of the multiple mobile bodies 180 may physically include equipment, devices, etc. for realizing the functions of a transmitter 181, a receiver 182, a transmitter 183, a mobility controller 184, an observation controller 185, etc.

[0042] For example, the mobile object 180 moves under the control of the movement control unit 184 while the transmitter 181 transmits radio waves under the control of the observation control unit 185 and the receiver 182 receives the reflected waves. This makes it possible to observe the target area while controlling the observation direction and observation position.

[0043] For example, if the mobile object 180 is a drone, it may fly at a height of about 10 m under the control of the movement control unit 184, transmit radio waves under the control of the observation control unit 185, and receive the reflected waves.

[0044] This makes it possible to observe the target area according to predetermined observation conditions and obtain observation information including the observation results. When the target area includes a predetermined area on the ground surface, the observation information is information indicating the results of observing the vicinity of the ground surface, including underground, using electromagnetic waves of the first frequency.

[0045] Then, when the receiving unit 182 generates observation information related to the received reflected wave, the transmitting unit 183 transmits the observation information to the information processing device 100 via the network NT, for example.

[0046] The network NT is typically a wireless network, but may include at least a wired network. The transmitter 183 may transmit the observation information in real time, or may collectively transmit observation information generated at different times.

[0047] The mobile object 180 may include a storage unit that stores the observation information instead of or in addition to the transmission unit 183. In this case, the observation information stored in the storage unit may be acquired by the information processing device 100 using an appropriate medium that stores or transmits information.

[0048] (Regarding Observation Information and Observation Conditions) The observation information is, for example, information about reflected waves of radio waves transmitted to a target area. In detail, for example, the observation information associates observation identification information, the intensity of the reflected waves, and the observation conditions.

[0049] The observation identification information is information for identifying the sensor used for the observation. The observation identification information is, for example, a code assigned in advance to identify each mobile object 180, each sensor, etc. This code may be composed of one or more numbers, letters, symbols, etc.

[0050] The observation conditions include, for example, at least one of the observation position, the observation direction, and the observation time.

[0051] The observation position is the position in real space where the observation is made.

[0052] The observation position may be obtained by providing the mobile object 180 with a GPS (Global Positioning System) function, a GNSS (Global Navigation Satellite System) function, or the like, for example.

[0053] The observation position may be expressed as a predetermined representative position for the mobile body 180, such as the position of a sensor that transmits and / or receives radio waves, the position of a GPS sensor, etc. The observation position may also be expressed as a three-dimensional position, for example. In more detail, the observation position may be expressed as latitude, longitude, distance from the earth's surface, etc. The distance from the earth's surface may be expressed as a positive value from the surface and a negative value underground.

[0054] Note that the method of acquiring the observation position and the method of representing the observation position are not limited to the above examples. For example, the observation position may be acquired based on the position of a marker (a reference object indicating a specific position) that is installed in advance on the ground surface or the like included in the target area. In this case, the observation position may be represented by a position relative to the marker. Furthermore, for example, when separate sensors are used for transmitting and receiving radio waves, the observation position may be represented by an intermediate position of the sensor. Furthermore, for example, when scanning the target area while moving along a predetermined path with a fixed irradiation direction, the observation position may be represented using a representative position of the target area, a moving path, or the like.

[0055] The observation direction includes, for example, the direction in which the radio waves are emitted and the direction in which the reflected waves are received.

[0056] The direction of radio wave irradiation may be, for example, a direction controlled by the observation control unit 185, and may include a rotation direction around one predetermined axis or two different axes. The direction of reflected wave reception may be, for example, a rotation direction around two different predetermined axes.

[0057] For example, if the target area includes a predetermined area on the Earth's surface and has one rotation axis, the irradiation direction of the radio waves includes rotation directions around a parallel or perpendicular axis. For example, if the target area includes a predetermined area on the Earth's surface and has two rotation axes, the irradiation direction of the radio waves includes rotation directions around each of the parallel axis and the perpendicular axis.

[0058] The direction of radio wave irradiation may be expressed, for example, as an angle relative to a predetermined reference direction along one or two axes.

[0059] Note that the methods for expressing the radiation direction of radio waves controlled by the observation control unit 185 and the radiation direction and reception direction of radio waves are not limited to the above. For example, the radiation direction of radio waves controlled by the observation control unit 185 may use a rotation axis different from the example given here, or may include parallel translation. Furthermore, for example, the method for expressing the radiation direction and reception direction of radio waves may use an angle related to a rotation axis different from the example given here.

[0060] The observation time is information indicating the time of observation, such as the observation time. The observation time may be at least one of the time when radio waves are emitted (e.g., the time of emission), the time when reflected waves are received (e.g., the time of transmission), a time associated with the time of transmission and the time of reception, such as midway between the time of transmission and the time of reception, etc. The observation time may be acquired by providing the mobile object 180 with a timekeeping function.

[0061] Note that one mobile object 180 may, for example, be equipped with multiple sets of transmitters 181 and receivers 182, and may generate multiple pieces of observation information under different observation conditions. Also, for example, one mobile object 180 may scan a target area multiple times, and generate multiple pieces of observation information under different observation conditions. In these cases, the information processing system S1 may be equipped with only one mobile object 180.

[0062] (Detailed Example of Information Processing Apparatus 100) A detailed example of the functional configuration of the information processing apparatus 100 and the information processing executed by the information processing apparatus 100 will be described.

[0063] 5, the information processing device 100 may functionally include an observation information acquisition unit 101, a storage unit 102, an output control unit 140, and a display unit 150 in addition to a first generation unit 110, a second generation unit 120, and an identification unit 130. The information processing device 100 may also execute information processing as shown in FIG.

[0064] Steps S110 to S130 are as described above. The information processing includes, for example, steps S101 to S102 before step S110. The observation information acquisition unit 101 acquires multiple pieces of observation information (step S101). The observation information acquisition unit 101 stores the multiple pieces of observation information acquired in step S101 in the storage unit 102 (step S102).

[0065] The information processing includes, for example, step S140 after step S130. The output control unit 140 outputs an object area map indicating an area where an object exists (step S140).

[0066] (Regarding Observation Information Acquisition Unit 101 and Storage Unit 102) The observation information acquisition unit 101 acquires observation information from each of a plurality of transmitters 183 provided in a plurality of mobile objects 180 that scan a target area under different observation conditions. In this way, the observation information acquisition unit 101 acquires a plurality of pieces of observation information.

[0067] The observation information acquisition unit 101 may acquire multiple pieces of observation information under different observation conditions from the transmitter 183 of one moving object 180 that scans the target area multiple times under different observation conditions.

[0068] The observation information acquisition unit 101 stores the acquired multiple pieces of observation information in a storage unit 102 for storing multiple pieces of observation information.

[0069] (First Generator 110) The first generator 110 acquires a plurality of pieces of observation information from the storage unit 102, and generates a plurality of pieces of reliability information based on each of the acquired pieces of observation information.

[0070] Each piece of reliability information is, for example, information that associates one or more positions in the target area with a reliability that indicates the likelihood that a target object exists at that position.

[0071] In more detail, for example, each piece of reliability information includes three-dimensional point cloud information generated based on the results of observation under one observation condition (observation information), as shown in Fig. 7. That is, each piece of reliability information is generated based on observation information obtained by scanning the target area once, for example. The positions of each point included in the multiple pieces of reliability information are represented using a common coordinate system.

[0072] 7 shows an example in which observation condition OCi is observation time Ti, observation position (xi, yi, zi), and observation direction (θi, Φi). The observation position (xi, yi, zi) is an example that indicates a representative position of an area to which the mobile object 180 has moved for scanning. The observation direction (θi, Φi) is an observation direction adopted in the observation, and is an example that indicates an angle of a rotation direction around two predetermined axes. Here, i is an integer between 1 and N, inclusive. Furthermore, N is an integer of 2 or more.

[0073] The accuracy (resolution) of the observation position and observation direction may be determined as appropriate, and the resolution of the observation information may be reduced in advance depending on this accuracy. When generating reliability information from observation information, the resolution of the observation position may be reduced, and the observation position of the reliability information may include multiple observation points of the observation information. In this case, the reliability included in the reliability information may be the average value, maximum value, minimum value, etc. of the reliability for the multiple observation points included in the observation position.

[0074] The first generating unit 110 may, for example, use the observation information to calculate the distance and angle of the reflection point by fast Fourier transform (FFT) and obtain a 3D point cloud within the target area. Furthermore, for example, the first generating unit 110 may calculate a reliability for each 3D point cloud based on the observation information. This allows the first generating unit 110 to generate reliability information in which each position in the 3D point cloud is associated with a reliability.

[0075] The reliability is a value corresponding to the likelihood that an object exists at the associated position. The reliability may be calculated based on the intensity of the reflected wave at the position, and may be a value indicating, for example, the probability that an object exists at the position. For example, the reliability may have a larger value as the likelihood of an object existing at the position increases.

[0076] (Regarding the Second Generator 120) The second generator 120 generates reliability map information indicating the integrated reliability for each position in the target region by integrating multiple pieces of reliability information having different observation conditions OCi, as shown in Fig. 8. The multiple pieces of reliability information having different observation conditions OCi are the multiple pieces of reliability information generated by the first generator 110.

[0077] Assume that observation condition OCi includes, for example, observation time Ti, observation position (xi, yi, zi), and observation direction (θi, Φi). In this case, different observation conditions OCi mean that, for example, at least one of observation time Ti, observation position (xi, yi, zi), and observation direction (θi, Φi) is different. Observation conditions OC1 to OCN shown in FIG. 8 are examples of such different observation conditions.

[0078] For example, if an object is at least partially buried underground, the object is unlikely to move. However, some objects may move over time. For example, the second generating unit 120 may use multiple pieces of reliability information whose observation times Ti are within a predetermined time range to generate the reliability map information. This makes it possible to accurately identify the area of ​​the object that may move.

[0079] The integrated reliability is a value obtained by integrating the multiple reliabilities included in the multiple pieces of reliability information for each location. When the multiple pieces of reliability information include multiple reliabilities for the same location, the integrated reliability of the location can be calculated by performing statistical processing on the multiple reliabilities associated with the location. Such an integrated reliability can be a simple average, weighted average, median, maximum value, minimum value, top-k average, etc. of the multiple reliabilities.

[0080] Here, the top-k average is the average of k values ​​taken from M values ​​(M is an integer of 2 or greater) in descending order of value. k is a value that can be arbitrarily set between 1 and M. When k=1, the top-k average is the maximum value of the M values. When k=M, the top-k average is the simple average of all M values.

[0081] For example, suppose that there are M reliabilities at a certain position. In this case, the top-k average for that position is, for example, the M reliabilities sorted in descending order, the top k reliabilities extracted, and the average value of the extracted k reliabilities. Furthermore, if it is assumed that the target object is made of a predetermined material such as metal, the integrated reliability may be a value obtained by normalizing the reflection intensity statistics from 0 to 1.

[0082] Furthermore, the integrated reliability may be calculated by performing statistical processing on each observation position (observation point) included in each of the plurality of pieces of reliability information, including a predetermined range of surrounding observation points. In such statistical processing, for example, a moving average filter or a three-dimensional Gaussian filter having a predetermined size, such as 3 units x 3 units x 3 units, in the reliability information may be used for each observation position (observation point).

[0083] Furthermore, the integrated reliability may be corrected based on the degree of variation (e.g., variance v, standard deviation σ, etc.) of multiple reliabilities associated with the same position. Generally, when the variation in multiple reliabilities associated with the same position is large, the signal that is the basis of the reliability is likely to be noise. Therefore, it is preferable to correct the integrated reliability by multiplying it by the reciprocal of the variance v or the standard deviation σ, for example, so that the greater the degree of variation, the smaller the integrated reliability becomes.

[0084] (Target Area Map) As described above, the identification unit 130 identifies the area where the target object exists by processing the reliability map information. Then, the output control unit 140 may, for example, display the target area map on the display unit 150. The target area map is a map that indicates the area where the target object exists in the target area.

[0085] The display unit 150 is an example of an output destination of the output control unit 140. The output destination of the output control unit 140 is not limited to the display unit 150, and may be, for example, a storage unit or the like provided in the information processing device 100 or another device. The display unit 150 may also be provided in a device other than the information processing device 100 that is connected to the information processing device 100 via a network configured, for example, via a wired or wireless connection, or a combination of these, so as to be able to send and receive information to and from the information processing device 100.

[0086] 9 is a diagram showing an example of a target area map. The target area map shown in the figure includes reliability map information. The target area map shown in the figure is expressed in three dimensions, and shows an example in which a contour image is used to indicate the area in which the identified target object exists. Note that the target area map is not limited to this, and may be, for example, a two-dimensional image.

[0087] (Example of Physical Configuration of Information Processing Device 100) The information processing device 100 is, for example, a general-purpose computer. The information processing device 100 includes, for example, a bus 1010, a processor 1020, a memory 1030, a storage device 1040, a network interface 1050, an input interface 1060, and an output interface 1070, as shown in FIG. 10 .

[0088] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, network interface 1050, input interface 1060, and output interface 1070. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0089] The processor 1020 is implemented as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit).

[0090] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0091] The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read only memory (ROM), or the like. The storage device 1040 stores program modules for realizing the functions of the information processing apparatus 100 that includes the storage device 1040. The processor 1020 reads each of these program modules into the memory 1030 and executes them to realize the function corresponding to the program module.

[0092] The network interface 1050 is an interface for connecting the information processing device 100 having the network interface 1050 to the network NT.

[0093] The input interface 1060 is an interface for the user to input information, and is configured from, for example, a touch panel, a keyboard, a mouse, and the like.

[0094] The output interface 1070 is an interface for presenting information to the user, and is configured, for example, by a liquid crystal panel, an organic EL (Electro-Luminescence) panel, or the like.

[0095] Note that the physical configuration of the information processing device 100 is not limited to this. For example, the information processing device 100 may be composed of multiple devices. In this case, each device may be, for example, a computer or the like having a physical configuration similar to that of the information processing device 100 shown in FIG. 10 .

[0096] (Operations and Effects) As described above, according to this embodiment, as outlined above, it is possible to accurately identify the area in which the target object exists.

[0097] According to this embodiment, the information processing device 100 further includes a storage unit 102 that stores a plurality of pieces of observational information. The first generation unit 110 acquires the plurality of pieces of observational information from the storage unit 102, and generates a plurality of pieces of reliability information based on the acquired plurality of pieces of observational information.

[0098] This allows the area where the object exists to be identified with high accuracy by obtaining an integrated reliability for each position from the results of pre-observing the area under different observation conditions.

[0099] According to this embodiment, the observation conditions include at least one of the observation time, the observation direction, and the observation position.

[0100] This allows the presence area of ​​the object to be identified with high accuracy by obtaining an integrated reliability for each position from the results of observing the object area under different conditions, at least one of the observation time, observation direction, and observation position.

[0101] According to this embodiment, the system further includes an observation information acquisition unit 101 that acquires multiple pieces of observation information from at least one flying object equipped with a sensor for transmitting and receiving radio waves, and stores the acquired observation information in a memory unit 102.

[0102] This allows the area where the object exists to be identified with high accuracy by obtaining an integrated reliability for each position from the results of pre-observing the area under different observation conditions.

[0103] According to this embodiment, the observation information includes the observation conditions.

[0104] This allows the area where the object exists to be identified with high accuracy by obtaining an integrated reliability for each position from the results of pre-observing the area under different observation conditions.

[0105] According to this embodiment, the integrated reliability includes at least one of a simple average, a weighted average, a median, a maximum value, a minimum value, and a top-k average of the multiple reliability values.

[0106] This allows the presence area of ​​the target object to be identified with high accuracy by statistically processing the results of pre-observing the target area under different observation conditions to obtain an integrated reliability for each position and using this.

[0107] According to this embodiment, the integrated reliability is a value corrected based on the degree of variation of the multiple reliabilities.

[0108] This allows for a more appropriate integrated reliability to be obtained by taking into account the degree of variation in the likelihood that the target exists. This can then be used to identify the area in which the target exists. Therefore, it becomes possible to identify the area in which the target exists with greater accuracy.

[0109] Second Embodiment The identification unit 130 may identify an area where an object exists in a target area by processing the reliability map information using a threshold value. This threshold value may be set manually or automatically using statistical processing or the like. An example of automatically setting a threshold value and identifying an area where an object exists using this threshold value will be described below.

[0110] 11, the specifying unit 130a includes a threshold setting unit 131 and an existence region specifying unit 132. The specifying unit 130a is a detailed example of the specifying unit 130.

[0111] The threshold setting unit 131 performs statistical processing on the reliability map information to set a threshold according to the area where the object exists.

[0112] The existence region specifying unit 132 specifies the existence region of the object by processing the reliability map information using a threshold value.

[0113] The identification unit 130a executes an identification process (step S130a) as shown in Fig. 12. This identification process (step S130a) is a detailed example of the above-mentioned identification process (step S130).

[0114] The threshold setting unit 131 performs statistical processing on the reliability map information to set a threshold according to the area where the object exists (step S131).

[0115] For example, the threshold setting unit 131 uses statistics such as the average value and variance of the integrated reliability to set a value that falls within a confidence interval with a significance level of 5% as the threshold.

[0116] The method of setting a threshold value by performing statistical processing on the reliability map information is not limited to this example.

[0117] The existence region specifying unit 132 specifies the existence region of the object by processing the reliability map information using a threshold value (step S132).

[0118] For example, the presence region specifying unit 132 may compare the integrated reliability with a threshold and specify the presence region of the object based on the comparison result. In detail, for example, the presence region specifying unit 132 may specify the presence region of the object as a region where the integrated reliability is greater than the threshold or a region where the integrated reliability is equal to or greater than the threshold.

[0119] As described above, according to this embodiment, the identification unit 130a includes a threshold setting unit 131 and an existence region identification unit 132. The threshold setting unit 131 performs statistical processing on the reliability map information to set a threshold according to the existence region of the object. The existence region identification unit 132 processes the reliability map information using the threshold to identify the existence region of the object.

[0120] This allows the threshold to be automatically set and the area where the object exists to be identified using the threshold, thereby eliminating the need for a person to manually set the threshold. As a result, it becomes possible to accurately and easily identify the object from information generated using electromagnetic waves.

[0121] Furthermore, by using statistical processing, it is possible to set an appropriate threshold and more accurately identify the area where the target exists, which makes it possible to more accurately identify the target from information generated using electromagnetic waves.

[0122] [Embodiment 3] The identification unit 130 may identify the presence area of ​​the object in the target area by processing the reliability map information using previously prepared correct answer data instead of the threshold described in embodiment 2. The correct answer data is, for example, data created based on observation information obtained by previously observing an object whose shape, etc. corresponds to that of the object using the transmitting unit 181 and the receiving unit 182. The correct answer data includes, for example, a three-dimensional point cloud based on the observation information of the object.

[0123] The identification unit 130 performs matching using, for example, the ground truth data and the reliability map information. In this matching, for example, the point clouds included in the ground truth data and the reliability map information are aligned so as to best match using a predetermined scale. At this time, it is preferable to perform alignment by, for example, converting the coordinate axis system of the ground truth data. An example of such a technique is the ICP (Iterative Closest Point) 3D point cloud registration technique.

[0124] The identification unit 130 may then identify a match region including a 3D point cloud included in the correct answer data after matching as a region where the object exists. The match region may be represented using, for example, a surface model, a frame model, a polygon model, etc. Furthermore, among the point clouds indicated by the reliability map information, a 3D point cloud within the match region may be identified as a region where the object exists.

[0125] The correct answer data is not limited to data created based on observation information. The correct answer data may include, for example, an approximation model that indicates the external shape of an object whose shape, etc., corresponds to that of the target object. The approximation model may be represented using, for example, a surface model, a frame model, a polygon model, etc. In this case, when matching the correct answer data with the reliability map information, an approximate point cloud may be generated by, for example, sampling from the approximation model, and this pseudo point cloud may be used instead of the 3D point cloud based on the above-mentioned observation information.

[0126] As described above, according to this embodiment, the identification unit 130 identifies the area in which the target object exists by processing the reliability map information using correct answer data created based on observation information obtained by previously observing an object whose shape corresponds to that of the target object.

[0127] In this embodiment, similarly to the first embodiment, an integrated reliability for each position can be obtained from the results of observing a target area under different observation conditions, and the target area can be identified using this integrated reliability. Therefore, it is possible to identify the target area with high accuracy.

[0128] Fourth Embodiment The identification unit 130 may use a machine learning model to identify an area in the target area where an object exists.

[0129] For example, as shown in FIG. 13, the information processing device 100 may include a first generation unit 110, a second generation unit 120, an observation information acquisition unit 101, a memory unit 102, an output control unit 140, a display unit 150, an identification unit 130b, and a learning unit 160.

[0130] The identification unit 130b inputs the reliability map information generated by the second generation unit 120 into an object identification model, which is a machine learning model that has learned to identify an object existence area from the reliability map information, and identifies the object existence area. The identification unit 130b is another detailed example of the identification unit 130, and executes processing equivalent to step S130.

[0131] The learning unit 160 learns the object identification model using training data prepared in advance. This training data is prepared in advance for learning and may include, for example, shape information in addition to confidence map information for learning (position and integrated confidence of the object). The shape information is information about the shape of the object, and is, for example, information indicating at least one of the contour, appearance, and external shape of the object. The information processing performed by the information processing device 100 may further include such processing (learning processing) performed by the learning unit 160.

[0132] As described above, according to this embodiment, the identification unit 130b inputs the reliability map information to an object identification model that has been trained to identify an area where an object exists from the reliability map information, and identifies the area where an object exists.

[0133] This allows the use of a machine learning model to identify the area where an object exists without the need for a human to set a threshold, thereby enabling the object to be identified accurately and easily.

[0134] According to this embodiment, the information processing device 100 includes a learning unit 160 that uses prepared training data to train a machine learning model. The training data includes shape information relating to the shape of an object.

[0135] This allows learning using shape information, eliminating the need for a human to set a threshold value and increasing the likelihood of accurately identifying the area where the object exists. This makes it possible to identify the object more accurately and easily.

[0136] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0137] In addition, although the flowcharts used in the above description show a sequence of steps (processes), the order of steps executed in each embodiment is not limited to the sequence shown in the flowcharts. In each embodiment, the order of steps shown in the diagrams can be changed as long as it does not cause any problems in terms of the content.

[0138] Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0139] 1. An information processing device comprising: a first generation means for generating a plurality of pieces of reliability information including a reliability indicating the likelihood that an object exists at each position in a target area expressed using a common coordinate system, based on each piece of observation information obtained by observing the target area using radio waves under different observation conditions; a second generation means for generating reliability map information indicating an integrated reliability for each position, based on the plurality of reliabilities at each position in the target area; and an identification means for identifying an area where the object exists by processing the reliability map information. 2. The information processing device described in 1., further comprising storage means for storing the plurality of pieces of observation information, wherein the first generation means acquires the plurality of pieces of observation information from the storage means and generates the plurality of pieces of reliability information based on each of the acquired plurality of pieces of observation information. 3. The information processing device described in 1. or 2., wherein the observation conditions include at least one of observation time, observation direction, and observation position. 4. The information processing device described in 2. or 3, further comprising observation information acquisition means for acquiring the plurality of pieces of observation information from at least one flying object equipped with a sensor for emitting and receiving the radio waves, and storing the acquired observation information in the storage means. 5. The information processing device according to any one of 1. to 4., wherein the observation information includes the observation conditions. 6. The information processing device according to any one of 1. to 5., wherein the integrated reliability includes at least one of a simple average, a weighted average, a median, a maximum value, a minimum value, and a top-k average of the multiple reliabilities. 7. The information processing device according to 6., wherein the integrated reliability is a value corrected based on a degree of variation in the multiple reliabilities. 8. The information processing device according to any one of 1. to 7., wherein the identification means includes: threshold setting means for setting a threshold according to an existence region of the object by performing statistical processing on the reliability map information; and existence region identification means for identifying the existence region of the object by processing the reliability map information using the threshold. 9. The information processing device according to any one of 1. to 7., wherein the identification means identifies the existence region of the object by processing the reliability map information using ground truth data created based on observation information obtained by previously observing an object whose shape corresponds to that of the object.10. The information processing device described in any one of 1. to 7., wherein the identification means inputs the reliability map information into an object identification model that has been trained to identify the object's existence region from the reliability map information, thereby identifying the object's existence region. 11. The information processing device described in 10., further comprising learning means that trains a machine learning model using training data prepared in advance, wherein the training data includes shape information related to the shape of the object. 12. An information processing system comprising: one or more mobile bodies that generate a plurality of pieces of observation information obtained by observing a target region using radio waves under different observation conditions; and the information processing device described in any one of 1. to 11, wherein each of the one or more mobile bodies includes: transmitting means that transmits the radio waves to the target region; receiving means that receives reflected waves of the transmitted radio waves and generates the observation information related to the reflected waves; and transmitting means that transmits the generated observation information. 13. 14. An information processing method in which one or more computers generate a plurality of pieces of reliability information, including a reliability indicating the likelihood that an object exists at each position in a target area expressed using a common coordinate system, based on each piece of observation information obtained by observing a target area using radio waves under different observation conditions, generate reliability map information indicating an integrated reliability for each position based on the plurality of reliabilities at each position in the target area, and identify the area where the object exists by processing the reliability map information. 14. The information processing method described in 13., wherein generating the plurality of pieces of reliability information includes acquiring the plurality of pieces of observation information from the storage means that stores the plurality of pieces of observation information, and generating the plurality of pieces of reliability information based on each of the acquired plurality of pieces of observation information. 15. The information processing method described in 13. or 14., wherein the observation conditions include at least one of observation time, observation direction, and observation position. 16. The information processing method described in 14. or 15., further comprising acquiring the plurality of pieces of observation information from at least one aircraft equipped with a sensor for transmitting and receiving the radio waves, and storing the acquired observation information in the storage means. 17. 17. The information processing method according to any one of 13. to 16., wherein the observation information includes the observation conditions.18. The information processing method according to any one of 13. to 17., wherein the integrated reliability includes at least one of a simple average, a weighted average, a median, a maximum value, a minimum value, and a top-k average of the multiple reliabilities. 19. The information processing method according to 18., wherein the integrated reliability is a value corrected based on the degree of variation of the multiple reliabilities. 20. The information processing method according to any one of 13. to 19., wherein specifying the existence region includes performing statistical processing on the reliability map information to set a threshold according to the existence region of the object, and processing the reliability map information using the threshold to specify the existence region of the object. 21. The information processing method according to any one of 13. to 19., wherein specifying the existence region includes processing the reliability map information using ground truth data created based on observation information obtained by previously observing an object whose shape corresponds to that of the object. 22. The information processing method according to any one of 13. to 19., wherein identifying the existence region includes inputting the reliability map information into an object identification model that has been trained to identify the existence region of the object from the reliability map information, thereby identifying the existence region of the object. 23. The information processing method according to 22., further including training the machine learning model using training data prepared in advance, wherein the training data includes shape information related to the shape of the object. 24. A program causing one or more computers to: generate, based on each of a plurality of pieces of observation information obtained by observing the object region using radio waves under different observation conditions, a plurality of pieces of reliability information including a reliability indicating the likelihood that the object exists at each position of the object region expressed using a common coordinate system; generate, based on the plurality of reliabilities at each position of the object region, reliability map information indicating an integrated reliability for each position; and identify the existence region of the object by processing the reliability map information. 25. The generating the plurality of pieces of reliability information includes acquiring the plurality of pieces of observation information from the storage means that stores the plurality of pieces of observation information, and generating the plurality of pieces of reliability information based on each of the acquired plurality of pieces of observation information. The program described in26. The program described in 24. or 25., wherein the observation conditions include at least one of observation time, observation direction, and observation position. 27. The program described in 25. or 26., further comprising acquiring the plurality of pieces of observation information from at least one flying object equipped with a sensor for transmitting and receiving the radio waves, and storing the acquired observation information in the storage means. 28. The program described in any one of 24. to 27., wherein the observation information includes the observation conditions. 29. The program described in any one of 24. to 28., wherein the integrated reliability includes at least one of a simple average, weighted average, median, maximum value, minimum value, and top-k average of the plurality of reliabilities. 30. The program described in 29., wherein the integrated reliability is a value corrected based on the degree of variation of the plurality of reliabilities. 31. The program according to any one of 24. to 30., wherein specifying the existence region includes: setting a threshold value according to the existence region of the object by performing statistical processing on the reliability map information; and processing the reliability map information using the threshold value to specify the existence region of the object. 32. The program according to any one of 24. to 30., wherein specifying the existence region includes processing the reliability map information using ground truth data created based on observation information obtained by previously observing an object whose shape corresponds to that of the object, to specify the existence region of the object. 33. The program according to any one of 24. to 30., wherein specifying the existence region includes inputting the reliability map information into an object identification model that has been trained to specify the existence region of the object from the reliability map information, to specify the existence region of the object. 34. The program according to 33., further including training the machine learning model using training data prepared in advance, wherein the training data includes shape information related to the shape of the object. 35. A recording medium having the program according to any one of 24. to 34. recorded thereon.

[0140] This application claims priority based on Japanese Patent Application No. 2023-137122, filed on August 25, 2023, the disclosure of which is incorporated herein by reference in its entirety.

[0141] REFERENCE SIGNS LIST 100 Information processing device 101 Observation information acquisition unit 102 Storage unit 110 First generation unit 120 Second generation unit 130, 130a, 130b Identification unit 131 Threshold setting unit 132 Existence area identification unit 140 Output control unit 150 Display unit 160 Learning unit 180 Mobile object 181 Transmission unit 182 Reception unit 183 Transmission unit 184 Movement control unit 185 Observation control unit

Claims

1. A first generation means generates multiple confidence information, including confidence levels indicating the likelihood that an object exists at each position in the target region represented using a common coordinate system, based on multiple observational information obtained by observing the target region under different observational conditions using radio waves. A second generation means generates confidence map information showing the integrated confidence level for each location based on a plurality of confidence levels at each location in the target area, The system includes a means for identifying the area where an object exists by processing the aforementioned confidence map information. Information processing device.

2. The system further comprises a storage means for storing the aforementioned multiple observational information, The first generation means acquires the plurality of observational information from the storage means and generates the plurality of confidence information based on each of the acquired plurality of observational information. The information processing apparatus according to claim 1.

3. The aforementioned observation conditions include at least one of the observation time, observation direction, and observation location. The information processing apparatus according to claim 1 or 2.

4. The system further comprises observation information acquisition means for acquiring the plurality of observation information from at least one aircraft equipped with sensors for transmitting and receiving the aforementioned radio waves, and storing the acquired observation information in the storage means. The information processing apparatus according to claim 2.

5. The aforementioned observation information includes the aforementioned observation conditions. The information processing apparatus according to claim 1 or 2.

6. The combined confidence score includes at least one of the following: a simple average, a weighted average, a median, a maximum value, a minimum value, or a top-k mean of the multiple confidence scores. The information processing apparatus according to claim 1 or 2.

7. The aforementioned integrated confidence level is a value corrected based on the degree of variation of the aforementioned multiple confidence levels. The information processing apparatus according to claim 6.

8. The aforementioned specifying means is, A threshold setting means that sets a threshold corresponding to the area where the object exists by performing statistical processing on the confidence map information, The system includes means for identifying the area where the object exists by processing the confidence map information using the threshold value. The information processing apparatus according to claim 1 or 2.

9. One or more computers, Based on multiple observational data obtained by observing the target region under different observational conditions using radio waves, multiple confidence information is generated, including confidence levels indicating the likelihood that an object exists at each position in the target region represented using a common coordinate system. Based on the multiple confidence levels at each location in the target area, confidence map information showing the integrated confidence level for each location is generated. The region where the object exists is identified by processing the aforementioned confidence map information. Information processing methods.

10. On one or more computers, Based on multiple observational data obtained by observing the target region under different observational conditions using radio waves, multiple confidence information is generated, including confidence levels indicating the likelihood that an object exists at each position in the target region represented using a common coordinate system. Based on the multiple confidence levels at each location in the target area, confidence map information showing the integrated confidence level for each location is generated. A program for performing the task of identifying the region where an object exists by processing the aforementioned confidence map information.