Inference device, inference method, and recording medium

Abstract

Provided is an inference device, an inference method, and a recording medium. The inference device (4) includes: an image signal acquisition unit (11) that acquires an image signal representing an inference target image; a feature quantity extraction unit (12) that supplies the image signal acquired by the image signal acquisition unit (11) to a first learning model (1a) in which learning of a learning image has been completed, and acquires an inference-time feature quantity from the first learning model (1a); a three-dimensional position estimation unit (15) that estimates a three-dimensional position of a detection target object appearing in the inference target image on the basis of a representative feature quantity and the inference-time feature quantity acquired by the feature quantity extraction unit (12); and a change analysis unit (16) that analyzes a temporal change in the three-dimensional position of the detection target object appearing in the inference target image on the basis of an estimation result of the three-dimensional position by the three-dimensional position estimation unit (15).

Description

TECHNICAL FIELD

[0001] The present disclosure relates to an inference device, an inference method, and a recording medium. BACKGROUND

[0002] For example, an abnormality detection method that detects occurrence of an abnormality of an autonomous mobile device is disclosed in Patent Literature 1. In the abnormality detection method, using each of a sensor group and a control section, occurrence of an abnormality of the autonomous mobile device is detected.

[0003] The sensor group detects a current state of the autonomous mobile device. The control section acquires, from the sensor group, time series data that is sensor data from a detection start time point to a current time point. The control section generates a plurality of divided data by dividing the time series data by each of a first prescribed interval, and generates a plurality of graphs from the plurality of divided data and the time series data. In addition, the control section provides the plurality of graphs to a learning model, and acquires a detection result of occurrence of an abnormality from the learning model. The detection result of occurrence of an abnormality includes a position of an abnormality of the autonomous mobile device.

[0004] Patent Literature 1: Japanese Patent Application Publication No. 2021-110973 SUMMARY

[0005] Problems to be Solved by the Invention

[0006] If the autonomous mobile device is a flying body, the navigation of the autonomous mobile device is affected by a change in a state of a natural environment. As the state of the natural environment, for example, there are an intensity of wind, a direction of wind, presence or absence of rain, a rainfall amount, presence or absence of snow, or a snowfall amount.

[0007] In the abnormality detection method disclosed in Patent Literature 1, in order to be able to detect occurrence of an abnormality of the autonomous mobile device regardless of a change in the state of the natural environment, the learning model needs to be learned taking into account all the states of the natural environment that can be assumed when the autonomous mobile device is navigating. However, in order to make the learning model learn taking into account all the states of the natural environment, extremely large learning data needs to be prepared, and in practice, sufficient learning data is sometimes unable to be prepared. Therefore, in the abnormality detection method, there is a problem that occurrence of an abnormality of the autonomous mobile device is sometimes unable to be detected depending on the state of the natural environment.

[0008] The present disclosure is achieved to solve the problem as described above, and aims at obtaining an inference device that is able to analyze a time change in a three-dimensional position of a detection target object without making a learning model learn taking into account a state of a natural environment.

[0009] Solution to Problem

[0010] The inference device according to the present disclosure includes an image signal acquisition unit that acquires an image signal representing an inference target image that is an image in which a detection target object is imaged in one or more different cases in which a domain of the image is different from a domain of a learning image and a recognition task is different from a task in which learning has been performed in advance; a feature quantity extraction unit that provides the image signal acquired by the image signal acquisition unit to a first learning model in which learning of a learning image has been completed, acquires an inference-time feature quantity from the first learning model, the inference-time feature quantity being a feature quantity that is used in inference of a three-dimensional position of the detection target object and is obtained by combining a plurality of feature quantities of the detection target object imaged in the inference target image after the plurality of feature quantities are blurred respectively; a three-dimensional position estimation unit that estimates the three-dimensional position of the detection target object imaged in the inference target image on the basis of a representative feature quantity that is a registered feature quantity of the detection target object imaged in a transformed image of an object in which the domain of the image and the recognition task are the same as those of the inference target image, and the inference-time feature quantity acquired by the feature quantity extraction unit; and a change analysis unit that analyzes a temporal change in the three-dimensional position of the detection target object imaged in the inference target image on the basis of an estimation result of the three-dimensional position estimation unit.

[0011] Effects of the Invention

[0012] According to the present disclosure, it is possible to analyze a temporal change in a three-dimensional position of a detection target object without considering a state of a natural environment. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 is a structural diagram representing the inference device 4 according to Embodiment 1.

[0014] Figure 2 is a hardware structural diagram representing hardware of the inference device 4 according to Embodiment 1.

[0015] Figure 3 is a hardware structural diagram of a computer in a case where the inference device 4 is implemented by software or firmware, etc.

[0016] Figure 4 is a structural diagram representing the learning device 7.

[0017] Figure 5 is a hardware structural diagram representing hardware of the learning device 7.

[0018] Figure 6 is a hardware structural diagram of a computer in a case where the learning device 7 is implemented by software or firmware, etc.

[0019] Figure 7 is a flowchart representing a processing procedure of the inference device 4 at the time of domain transformation.

[0020] Figure 8 This is a flowchart illustrating the reasoning method of the reasoning device 4 as a location estimation process.

[0021] Figure 9 It is an explanatory graph showing the time-varying three-dimensional position of the object being probed.

[0022] (Explanation of reference numerals in the attached diagram)

[0023] 1: Model storage unit; 1a: First learning model; 2: Model storage unit; 2a: Second learning model; 3: Camera; 4: Inference device; 5: Display device; 6: Learning data storage unit; 7: Learning device; 11: Image signal acquisition unit; 12: Feature extraction unit; 13: Representative feature registration unit; 14: Representative feature storage unit; 15: 3D position estimation unit; 16: Change analysis unit; 21: Image signal acquisition circuit; 22: Feature extraction circuit; 23: Representative feature registration circuit; 24: Representative feature storage circuit; 25: 3D position estimation circuit; 26: Change analysis circuit; 31: Memory; 32: Processor; 41: Learning data acquisition unit; 42: Learning processing unit; 51: Learning data acquisition circuit; 52: Learning processing circuit; 61: Memory; 62: Processor. Detailed Implementation

[0024] The following description, in order to illustrate the present disclosure in more detail, describes the manner in which the present disclosure is implemented with reference to the accompanying drawings.

[0025] Implementation method 1.

[0026] Figure 1 This is a structural diagram showing the inference device 4 involved in Embodiment 1.

[0027] Figure 2 This is a hardware structure diagram showing the hardware of the inference device 4 involved in Embodiment 1.

[0028] exist Figure 1 In this context, the model storage unit 1 may be implemented, for example, via a hard disk or RAM (Random Access Memory).

[0029] Model storage unit 1 stores the first learning model 1a.

[0030] For example, the first learning model 1a is implemented using deep neural networks (DNNs). DNNs include convolutional neural networks (CNNs).

[0031] The learning model 1a is provided with an image signal representing a learning image as learning data at the time of learning, and performs learning of the learning image. The learning image is, for example, an image used in an image recognition task.

[0032] The kind of image as the domain of the learning image can be arbitrary, and the learning image is, for example, any of an RGB image, a TIR image, or an image generated by a CG simulator.

[0033] In Figure 1 In the inference device 4 illustrated in FIG. 1, the learning image is assumed to be an RGB image for the sake of explanation. The learning model 1a is provided with a large number of RGB images to learn the RGB image.

[0034] In a case where the domain of the image and the recognition task are different from the learning image, the learning model 1a outputs a feature vector representing a feature quantity, which is obtained by combining a plurality of feature quantities of a detection target object, which are blurred respectively, of a transformed image representing the detection target object, to the feature quantity extraction unit 12 when the image signal representing the transformed image is provided from the feature quantity extraction unit 12 described later, and the feature quantity is used in the inference of the three-dimensional position of the detection target object.

[0035] The transformed image can be any one or more images different from the learning image in the domain of the image and the recognition task. In Figure 1 In the inference device 4 illustrated in FIG. 1, the transformed image is assumed to be a TIR image for the sake of explanation.

[0036] The CNNs that realize the first learning model 1a are very deep CNNs. As the very deep CNNs, there are, for example, ResNet having 101 layers. Therefore, for example, at the time of estimation of the three-dimensional position, the feature quantity represented by the feature vector output from the output layer of the first learning model 1a when the image signal is provided to the input layer of the first learning model 1a is a high-dimensional feature quantity. The high-dimensional feature quantity includes a plurality of dimensional feature quantities, and, as the feature vector representing the high-dimensional feature quantity, a Tensor is used, for example.

[0037] The low-dimensional feature quantity output from a shallow layer in the plurality of levels of hidden layers included in the learning model 1a represents, for example, color, brightness, or direction. Therefore, the low-dimensional feature quantity depends on the domain of the image represented by the image signal provided to the input layer. That is, the feature quantity represented by the feature vector output from the shallow layer of the learning model 1a when the image signal of the RGB image is provided to the input layer of the learning model 1a is sometimes greatly different from the feature quantity represented by the feature vector output from the shallow layer of the learning model 1a when the image signal of the TIR image is provided to the input layer of the learning model 1a.

[0038] On the other hand, the high-dimensional features output from the sufficiently deep intermediate layers of learning model 1a represent conceptual features such as whether the detected object is good or bad. Therefore, the high-dimensional features are conceptual information with extremely low dependence on the domain of the image represented by the image signal provided to the input layer. Furthermore, by employing deeper high-dimensional features, general information with low task dependence can be obtained. These can be conceptual object features, such as known "Objectness" or "Informativeness".

[0039] In other words, when the image signal of an RGB image is provided to the input layer of the learning model 1a, the high-dimensional feature quantity represented by the feature vector output from the output layer of the learning model 1a is small in difference from the high-dimensional feature quantity represented by the feature vector output from the output layer of the learning model 1a when the image signal of a TIR image is provided to the input layer of the learning model 1a.

[0040] Therefore, when the learning model 1a is implemented by CNNs and the inference device 3 uses high-dimensional features represented by feature vectors output from sufficiently deep intermediate layers of CNNs, the dependence on the domain of the image represented by the image signal provided to the input layer and the dependence on the recognition task are reduced.

[0041] For example, the model storage unit 2 can be implemented using a hard disk or RAM.

[0042] Model storage unit 2 stores the second learning model 2a.

[0043] For example, the second learning model 2a can be implemented by using RNNs (Recurrent Neural Networks) as recurrent neural networks.

[0044] The second learning model 2a is a free learning model, which is a learning model that learns three-dimensional position to recursively learn the temporal changes of the three-dimensional position.

[0045] When the second learning model 2a receives the estimation result of the three-dimensional position from the three-dimensional position estimation unit 15 (described later) from the change analysis unit 16 (described later), it outputs a signal representing the three-dimensional position of the object to be detected as position data representing the time change of the three-dimensional position of the object to the change analysis unit 16.

[0046] exist Figure 1 In the inference device 4 shown, the first learning model 1a and the second learning model 2a are respectively disposed outside the inference device 4. However, this is only one example. For example, the first learning model 1a can also be built into the feature extraction unit 12, and the second learning model 2a can be built into the change analysis unit 16.

[0047] The detection target object is, for example, a drone, a flying car, a helicopter, a car, or a ship. In Figure 1 In the inference device 4 shown, for the sake of explanation, an example is described in which the detection target object is identified as any one of a drone, a flying car, and a helicopter.

[0048] In addition, in the inference device 4 shown, Figure 1 In the inference device 4 shown, with respect to the three-dimensional position of the detection target object identified by the three-dimensional position estimation section 15, a representative feature amount including the Objectness of the detection target object is acquired from, for example, the representative feature amount storage section 14 in which a feature vector including the Objectness of an object in a high-dimensional feature is registered, the representative feature amount is compared with the inference-time feature amount extracted by the feature amount extraction section 12, and a representative feature amount corresponding to the inference-time feature amount extracted by the feature amount extraction section 12 is determined among the representative feature amounts of the plurality of detection target objects. The class having the most similar representative feature amount becomes the class to which the detection target object belongs. Also, since the inference-time feature amount includes the Objectness of the detection target object included in the image input at the time of inference, by transforming the description method of the high-dimensional feature of the inference-time feature amount from Tensor or the like to two-dimensional space, it is possible to express the spatial position of the object.

[0049] In the case where the detection target object is identified as any one of a drone, a flying car, and a helicopter, the learning data provided to the first learning model la is learning data including an image signal representing a learning image. For the sake of explanation, it is assumed that the learning image is an RGB image.

[0050] The learning data provided to the second learning model 2a is an estimation result of the three-dimensional position estimation section 15 with respect to the three-dimensional position.

[0051] The second learning model 2a, when provided with the estimation result of the three-dimensional position, learns the three-dimensional position to recursively perform the temporal change of the three-dimensional position.

[0052] The camera 3 is realized, for example, by an infrared camera.

[0053] The camera 3 photographs the detection target object.

[0054] When the inference device 4 registers an image of a domain different from that at the time of learning (hereinafter referred to as “at the time of domain transformation”), the camera 3, for example, outputs an image signal representing a TIR image in which the detection target object is visualized as an image signal representing a transformed image in which the detection target object is visualized to the inference device 4.

[0055] When the inference device 4 estimates the position estimation of the three-dimensional position of the detection target object, the camera 3 outputs, for example, an image signal representing a TIR image in which the detection target object is visualized, as an image signal representing an inference target image in which the detection target object is visualized, to the inference device 4.

[0056] In the domain transformation, the image signal acquisition section 11 acquires, from the camera 3, an image signal representing a transformation-use image in which the detection target object is visualized. Figure 1 In the domain transformation, the image signal acquisition section 11 acquires, from the camera 3, an image signal representing a transformation-use image in which the detection target object is visualized.

[0057] The image signal acquisition section 11 outputs the image signal representing the transformation-use image to the feature quantity extraction section 12. Figure 2 The image signal acquisition section 11 outputs the image signal representing the inference target image to the feature quantity extraction section 12.

[0058] In the domain transformation, the image signal acquisition section 11 acquires, from the camera 3, an image signal representing a transformation-use image in which the detection target object is visualized.

[0059] The image signal acquisition section 11 outputs the image signal representing the transformation-use image to the feature quantity extraction section 12.

[0060] In the position estimation, the image signal acquisition section 11 acquires, from the camera 3, an image signal representing an inference target image in which the detection target object is visualized.

[0061] The image signal acquisition section 11 outputs the image signal representing the inference target image to the feature quantity extraction section 12.

[0062] The image signal acquisition section 11 outputs the image signal representing the inference target image to the feature quantity extraction section 12.

[0063] The image signal acquisition section 11 outputs the image signal representing the inference target image to the feature quantity extraction section 12. Figure 2 The feature quantity extraction section 12 is implemented by, for example, a feature quantity extraction circuit 22 illustrated in FIG. 2.

[0064] In the domain transformation, the feature quantity extraction section 12 supplies the image signal acquired by the image signal acquisition section 11 to the first learning model la, and acquires, from the first learning model la, a feature vector representing a representative feature quantity that is a feature quantity in which a plurality of feature quantities of the detection target object visualized in the transformation-use image are combined after being blurred respectively. The representative feature quantity is a feature quantity used in the inference of the three-dimensional position of the detection target object.

[0065] The feature quantity extraction section 12 outputs the feature vector to the representative feature quantity registration section 13.

[0066] At the time of position estimation, the feature quantity extraction section 12 supplies the image signal acquired by the image signal acquisition section 11 to the first learning model la, acquires a feature vector representing an inference-time feature quantity from the first learning model la, the inference-time feature quantity being a feature quantity in which a plurality of feature quantities are combined after the feature quantities of the probe target objects appearing in the inference target image are blurred respectively. The inference-time feature quantity is a feature quantity used in the inference of the three-dimensional position of the probe target object.

[0067] The feature quantity extraction section 12 outputs the feature vector to the three-dimensional position estimation section 15.

[0068] As the process in which a plurality of feature quantities are blurred respectively, "Pooling Operation" is known.

[0069] The representative feature quantity registration section 13 acquires a feature vector representing a feature quantity of a probe target object appearing in the inference target image, and stores the feature vector in the representative feature quantity storage section 14. Figure 2 The representative feature quantity storage section 14 is realized by, for example, a representative feature quantity storage circuit 24 as shown in Fig. 1.

[0070] The representative feature quantity registration section 13 registers the representative feature quantity acquired by the feature quantity extraction section 12.

[0071] That is, the representative feature quantity registration section 13 acquires a feature vector representing a feature quantity of a drone existing in a certain region, and stores the feature vector in the representative feature quantity storage section 14.

[0072] In addition, the representative feature quantity registration section 13 acquires a feature vector representing a feature quantity of a flying car existing in a certain region, and stores the feature vector in the representative feature quantity storage section 14.

[0073] In addition, the representative feature quantity registration section 13 acquires a feature vector representing a feature quantity of a helicopter existing in a certain region, and stores the feature vector in the representative feature quantity storage section 14.

[0074] The representative feature quantity storage section 14 is realized by, for example, a representative feature quantity storage circuit 24 as shown in Fig. 1. Figure 2 The representative feature quantity storage section 14 is realized by, for example, a representative feature quantity storage circuit 24 as shown in Fig. 1.

[0075] The representative feature quantity storage section 14 stores a feature vector representing a representative feature quantity.

[0076] The three-dimensional position estimation section 15 is realized by, for example, a three-dimensional position estimation circuit 25 as shown in Fig. 1. Figure 2 The three-dimensional position estimation section 15 is realized by, for example, a three-dimensional position estimation circuit 25 as shown in Fig. 1.

[0077] The three-dimensional position estimation section 15 acquires a feature vector representing a feature quantity of a probe target object appearing in the inference target image from the feature quantity extraction section 12, and acquires a feature vector representing a representative feature quantity from the representative feature quantity storage section 14.

[0078] The three-dimensional position estimation unit 15 estimates the three-dimensional position of the detection target object represented in the inference target image, based on the representative feature amount and the feature amount of the detection target object represented in the inference target image.

[0079] Specifically, the three-dimensional position estimation unit 15 respectively identifies the kind and the region where the detection target object represented in the inference target image exists, based on the representative feature amount and the inference-time feature amount.

[0080] Specifically, the three-dimensional position estimation unit 15 acquires the representative feature amount including both the Objectness and the kind of the object from the representative feature amount storage unit 14 in which the feature vector including both the Objectness and the kind of the object in the high-dimensional feature is registered, compares the representative feature amount with the inference-time feature amount extracted by the feature amount extraction unit 12, and determines the representative feature amount corresponding to the inference-time feature amount extracted by the feature amount extraction unit 12 among the representative feature amounts of the plurality of detection target objects. The class having the most similar representative feature amount becomes the class to which the detection target object belongs. Also, since the inference-time feature amount includes the Objectness of the detection target object included in the image input at the time of inference, it is possible to represent the position of the object in space by transforming the description method of the high-dimensional feature of the inference-time feature amount from Tensor or the like to two-dimensional space.

[0081] The three-dimensional position estimation unit 15 generates display data representing the recognition result of the detection target object, and outputs the display data to the display device 5. Here, in a case where it is expressed as a rectangle on a two-dimensional space, it becomes an Object Detection task, and in a case where it is expressed as a region on a two-dimensional space, it becomes a Sematic Segmentation task.

[0082] For example, the change analysis circuit 26 shown in FIG. 6 is implemented as the change analysis unit 16. Figure 2

[0083] The change analysis unit 16 analyzes the temporal change in the three-dimensional position of the detection target object represented in the inference target image, based on the estimation result of the three-dimensional position by the three-dimensional position estimation unit 15.

[0084] Specifically, the change analysis unit 16 supplies the estimation result of the three-dimensional position by the three-dimensional position estimation unit 15 to the second learning model 2a, and acquires position data representing the temporal change in the three-dimensional position of the detection target object from the second learning model 2a.

[0085] ​In addition, the change analysis section 16 determines whether or not the detected object is, for example, hovering, ascending, descending, advancing, or retreating, in addition to determining whether or not the detected object has landed, based on the position data, as a process of classifying the detected object into a plurality of classes.

[0086] The change analysis section 16 generates display data of an analysis result indicating a temporal change in the three-dimensional position of the detected object, and outputs the display data to the display device 5.

[0087] In addition, the change analysis section 16 generates display data of a determination result of the determination process described above, and outputs the display data to the display device 5.

[0088] The display device 5 displays the estimation result of the three-dimensional position based on the display data output from the three-dimensional position estimation section 15 on a display not shown.

[0089] In addition, the display device 5 displays the analysis result of the temporal change in the three-dimensional position of the detected object and the determination result of the determination process based on the display data output from the change analysis section 16 on a display not shown.

[0090] In Figure 1 , it is assumed that the image signal acquisition section 11, the feature quantity extraction section 12, the representative feature quantity registration section 13, the representative feature quantity storage section 14, the three-dimensional position estimation section 15, and the change analysis section 16, which are structural elements of the inference device 4, are each realized by a dedicated hardware as shown in Figure 2 . That is, it is assumed that the inference device 4 is realized by an image signal acquisition circuit 21, a feature quantity extraction circuit 22, a representative feature quantity registration circuit 23, a representative feature quantity storage circuit 24, a three-dimensional position estimation circuit 25, and a change analysis circuit 26.

[0091] The representative feature quantity storage circuit 24 corresponds to, for example, a nonvolatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable Programmable Read Only Memory), a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, or a DVD (Digital Versatile Disc).

[0092] The image signal acquisition circuit 21, the feature amount extraction circuit 22, the representative feature amount registration circuit 23, the three-dimensional position estimation circuit 25, and the change analysis circuit 26 each correspond to, for example, a single circuit, a composite circuit, a programmed processor, a parallel-programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0093] The structural elements of the inference device 4 are not limited to being implemented by dedicated hardware, and the inference device 4 can be implemented by software, firmware, or a combination of software and firmware.

[0094] The software or firmware is saved in a memory of a computer as a program. The computer refers to hardware that executes the program, and corresponds to, for example, a CPU (Central Processing Unit), a GPU (Graphical Processing Unit), a central processing device, a processing device, an arithmetic device, a microprocessor, a microcomputer, a processor, or a DSP (Digital Signal Processor).

[0095] Figure 3 is a hardware configuration diagram of a computer in a case where the inference device 4 is implemented by software or firmware and the like.

[0096] In a case where the inference device 4 is implemented by software or firmware and the like, a representative feature amount storage section 14 is constituted on a memory 31 of a computer. A program for causing the computer to execute each processing procedure in the image signal acquisition section 11, the feature amount extraction section 12, the representative feature amount registration section 13, the three-dimensional position estimation section 15, and the change analysis section 16 is saved in the memory 31. Then, a processor 32 of the computer executes the program saved in the memory 31.

[0097] In addition, in Figure 2 is shown an example in which the structural elements of the inference device 4 are each implemented by dedicated hardware, and in Figure 3 is shown an example in which the inference device 4 is implemented by software or firmware and the like. However, this is only an example, and a part of the structural elements of the inference device 4 can be implemented by dedicated hardware, and the remaining structural elements can be implemented by software or firmware and the like.

[0098] Figure 4 is a structural diagram of the learning device 7.

[0099] Figure 5 is a hardware configuration diagram of the hardware of the learning device 7.

[0100] The learning data storage 6 is realized by, for example, a hard disk or a RAM.

[0101] The learning data storage 6 stores image signals representing learning images as learning data.

[0102] The learning device 7 is provided with a learning data acquisition section 41 and a learning processing section 42.

[0103] The learning data acquisition section 41 is realized by, for example, a learning data acquisition circuit 51. Figure 5 The learning data acquisition section 41 is realized by, for example, a learning data acquisition circuit 51.

[0104] The learning data acquisition section 41 acquires learning data from the learning data storage 6.

[0105] The learning data acquisition section 41 outputs the learning data to the learning processing section 42.

[0106] The learning processing section 42 is realized by, for example, a learning processing circuit 52. Figure 5 The learning processing section 42 is realized by, for example, a learning processing circuit 52.

[0107] The learning processing section 42 acquires a large amount of learning data from the learning data acquisition section 41.

[0108] The learning processing section 42 provides each of the learning data to the learning model la, and causes the learning model la to learn learning images represented by image signals included in each of the learning data.

[0109] The learning model la, which has completed learning, outputs a feature vector corresponding to an image signal when the image signal is provided at the time of domain conversion or position estimation.

[0110] In the case where the learning data acquisition section 41 and the learning processing section 42, which are structural elements of the learning device 7, are realized by dedicated hardware, the learning data acquisition section 41 and the learning processing section 42 are realized by, for example, a learning data acquisition circuit 51 and a learning processing circuit 52, respectively. Figure 4 In the case where the learning data acquisition section 41 and the learning processing section 42, which are structural elements of the learning device 7, are realized by dedicated hardware, the learning data acquisition section 41 and the learning processing section 42 are realized by, for example, a learning data acquisition circuit 51 and a learning processing circuit 52, respectively. Figure 5 In the case where the learning data acquisition section 41 and the learning processing section 42, which are structural elements of the learning device 7, are realized by dedicated hardware, the learning data acquisition section 41 and the learning processing section 42 are realized by, for example, a learning data acquisition circuit 51 and a learning processing circuit 52, respectively.

[0111] The learning data acquisition section 41 and the learning processing section 42 correspond to, for example, a single circuit, a composite circuit, a programmed processor, a parallel-programmed processor, an ASIC, an FPGA, or a combination thereof.

[0112] The structural elements of the learning device 7 are not limited to being realized by dedicated hardware, and the learning device 7 can be realized by software, firmware, or a combination of software and firmware.

[0113] Figure 6 is a hardware configuration diagram of a computer in the case where the learning device 7 is realized by software or firmware.

[0114] In a case where the learning device 7 is implemented by software or firmware, or the like, a program for causing a computer to execute each of the processes in the learning data acquisition section 41 and the learning processing section 42 is stored in the memory 61. Then, the processor 62 of the computer executes the program stored in the memory 61.

[0115] In addition, in Figure 5 an example in which the structural elements of the learning device 7 are each implemented by dedicated hardware is shown, and in Figure 6 an example in which the learning device 7 is implemented by software or firmware, or the like is shown. However, this is only an example, and a part of the structural elements of the learning device 7 can be implemented by dedicated hardware, and the remaining structural elements can be implemented by software or firmware, or the like.

[0116] First, the operation of the learning device 7 shown in Figure 4 will be described.

[0117] A large amount of learning data is stored in the learning data storage section 6, and each of the learning data includes an image signal representing a learning image.

[0118] The learning data acquisition section 41 of the learning device 7 acquires a large amount of learning data from the learning data storage section 6.

[0119] The learning data acquisition section 41 outputs each of the learning data to the learning processing section 42.

[0120] The learning processing section 42 acquires each of the learning data from the learning data acquisition section 41.

[0121] The learning processing section 42 supplies each of the learning data to the first learning model la, and causes the learning model la to learn a learning image represented by an image signal included in each of the learning data.

[0122] The learning model la, which has completed learning, for example, outputs a feature vector representing a high-dimensional feature amount of a detection target object appearing in an RGB image from an output layer as a feature vector corresponding to an image signal representing the RGB image when the image signal is supplied to an input layer.

[0123] In a case where a learning image used in learning of the learning model la is, for example, an RGB image without using a TIR image as a learning image, even if a detection target object appearing in the RGB image and a detection target object appearing in the TIR image are the same object which is normal, sometimes a feature vector output from the output layer when an image signal representing the RGB image is supplied to the input layer is different from a feature vector output from the output layer when an image signal representing the TIR image is supplied to the input layer.

[0124] However, the CNNs that implement the learning model la are very deep CNNs, and the feature vector output from the sufficiently deep intermediate layer of the learning model la represents a high-dimensional feature amount. Therefore, the above-described difference is small.

[0125] In addition, the feature amount represented by the feature vector output from the output layer of the learning model la is, as described above, a feature amount in which the dependency of the domain of the image and the dependency of the recognition task are excluded, respectively, after the plurality of feature amounts of the plurality of layers of the sufficiently deep intermediate layer are blurred and combined.

[0126] Next, the operation of the inference device 4 at the time of domain transformation will be described.

[0127] Figure 7 is a flowchart showing the processing procedure of the inference device 4 at the time of domain transformation.

[0128] The camera 3 photographs the detection target object. The detection target object photographed by the camera 3 is any one of a drone, a flying car, and a helicopter. However, the inference device 4 can also classify the detection target object into, for example, 1000. Therefore, classifying the detection target object into three of a drone, a flying car, or a helicopter is an example. In addition, the detection target object photographed by the camera 3 exists in a certain region.

[0129] The camera 3 outputs, for example, an image signal representing a TIR image in which the detection target object is visualized, as an image signal representing a transformed image in which the detection target object existing in a certain region is visualized, to the inference device 4.

[0130] The image signal acquisition unit 11 acquires an image signal representing a transformed image in which the detection target object is visualized from the camera 3 (step ST1 of FIG. 1). Figure 7

[0131] Specifically, the image signal acquisition unit 11 acquires an image signal representing a transformed image in which a drone existing in a certain region is visualized from the camera 3, and outputs the image signal representing the transformed image to the feature amount extraction unit 12.

[0132] In addition, the image signal acquisition unit 11 acquires an image signal representing a transformed image in which a flying car existing in a certain region is visualized from the camera 3, and outputs the image signal representing the transformed image to the feature amount extraction unit 12.

[0133] In addition, the image signal acquisition unit 11 acquires an image signal representing a transformed image in which a helicopter existing in a certain region is visualized from the camera 3, and outputs the image signal representing the transformed image to the feature amount extraction unit 12. ​

[0134] The feature quantity extraction section 12 acquires, from the image signal acquisition section 11, an image signal indicating a transformed image in which a probe target object existing in a certain region is visualized.

[0135] The feature quantity extraction section 12 extracts, from each image signal, a feature quantity of a probe target object visualized in each transformed image (step ST2). Figure 7

[0136] Specifically, the feature quantity extraction section 12 acquires, from the image signal acquisition section 11, an image signal indicating a transformed image in which a drone existing in a certain region is visualized.

[0137] The feature quantity extraction section 12 supplies each image signal to the first learning model la, and acquires, from the first learning model la, a feature vector indicating a representative feature quantity in which a plurality of feature quantities of the drone existing in the certain region are combined after being blurred respectively.

[0138] The feature quantity extraction section 12 outputs the feature vector to the representative feature quantity registration section 13.

[0139] In addition, the feature quantity extraction section 12 acquires, from the image signal acquisition section 11, an image signal indicating a transformed image in which a flying car existing in a certain region is visualized.

[0140] The feature quantity extraction section 12 supplies each image signal to the first learning model la, and acquires, from the first learning model la, a feature vector indicating a representative feature quantity in which a plurality of feature quantities of the flying car existing in the certain region are combined after being blurred respectively.

[0141] The feature quantity extraction section 12 outputs the feature vector to the representative feature quantity registration section 13.

[0142] In addition, the feature quantity extraction section 12 acquires, from the image signal acquisition section 11, an image signal indicating a transformed image in which a helicopter existing in a certain region is visualized.

[0143] The feature quantity extraction section 12 supplies each image signal to the first learning model la, and acquires, from the first learning model la, a feature vector indicating a representative feature quantity in which a plurality of feature quantities of the helicopter existing in the certain region are combined after being blurred respectively.

[0144] The feature quantity extraction section 12 outputs the feature vector to the representative feature quantity registration section 13.

[0145] The representative feature quantity registration section 13 acquires each feature vector from the feature quantity extraction section 12.

[0146] ​The representative feature amount registration unit 13 stores each feature vector in the representative feature amount storage unit 14, thereby registering a representative feature amount (step ST3). Figure 7

[0147] Here, the feature vector representing the representative feature amount is expressed by a Tensor. A Tensor can express information of higher dimensionality than a Vector, and is sometimes referred to as a feature map.

[0148] Since a Tensor can express information of high dimensionality, in the case where the representative feature amount registration unit 13 stores the feature vector expressed by a Tensor as it is in the representative feature amount storage unit 14, a large amount of processing time can be required when the feature vector is collated in the three-dimensional position estimation unit 15.

[0149] In order to shorten the processing time required when the feature vector is collated in the three-dimensional position estimation unit 15, the representative feature amount registration unit 13 can also transform the feature vector representing the representative feature amount into a One-hot-vector of fewer dimensions than a Tensor, and store the One-hot-vector in the representative feature amount storage unit 14.

[0150] The feature vector registered by the representative feature amount registration unit 13 expresses information of high dimensionality such as several hundred dimensions, regardless of whether it is a Tensor or a One-hot-vector. Therefore, even if there is a slight difference between a plurality of detection target objects of the same kind, the feature vector describes the representative features of the detection target object in high dimensionality.

[0151] Next, the operation of the inference device 4 at the time of position estimation will be described.

[0152] Figure 8 is a flowchart showing an inference method representing the process of the inference device 4 at the time of position estimation.

[0153] The camera 3 photographs a detection target object. It is not clear which of a drone, a flying car, and a helicopter the detection target object photographed by the camera 3 is. In addition, it is not clear in which region the detection target object photographed by the camera 3 exists.

[0154] The camera 3, for example, outputs an image signal representing a TIR image in which a detection target object is visualized, as an image signal representing an inference target image in which a detection target object is visualized, to the inference device 4.

[0155] The image signal acquisition unit 11 acquires an image signal representing an inference target image in which a detection target object is visualized, from the camera 3 (step ST11). Figure 8

[0156] ​​The image signal acquisition section 11 outputs an image signal representing an inference target image to the feature quantity extraction section 12.

[0157] The feature quantity extraction section 12 acquires an image signal representing an inference target image from the image signal acquisition section 11.

[0158] The feature quantity extraction section 12 extracts a feature quantity of a detection target object appearing in an inference target image from an image signal (step ST12). Figure 8

[0159] Specifically, the feature quantity extraction section 12 supplies the image signal to the first learning model la, and acquires a feature vector representing a high-dimensional feature quantity of a detection target object appearing in an inference target image from the first learning model la.

[0160] The feature quantity extraction section 12 outputs the feature vector to the three-dimensional position estimation section 15.

[0161] The three-dimensional position estimation section 15 acquires the feature vector from the feature quantity extraction section 12.

[0162] The three-dimensional position estimation section 15 acquires a plurality of feature vectors representing representative feature quantities from the representative feature quantity storage section 14.

[0163] The three-dimensional position estimation section 15 estimates a three-dimensional position of a detection target object appearing in an inference target image on the basis of the plurality of feature vectors representing representative feature quantities and the feature vector acquired from the feature quantity extraction section 12 (step ST13). Figure 8

[0164] Specifically, the three-dimensional position estimation section 15 calculates the similarity of the plurality of feature vectors representing representative feature quantities and the feature vector acquired from the feature quantity extraction section 12, respectively.

[0165] The three-dimensional position estimation section 15 determines the highest similarity in the similarity of the representative feature quantity registered in the number of objects to be identified and the inference-time feature quantity, and determines the representative feature quantity corresponding to the highest similarity. The three-dimensional position estimation section 15 is able to discriminate which class by performing the determination.

[0166] In the three-dimensional position estimation section 15, the class having the most similar representative feature quantity becomes the class to which the detection target object belongs. For example, in a case where both the representative feature quantity and the inference-time feature quantity are input with a TIR image, class recognition beyond the domain at the time of learning is able to be performed.

[0167] ​​Further, since the feature amount at the time of inference includes the presence range (Objectness) of the detection target object included in the image input at the time of inference, it is possible to express the presence position of the object in space by transforming the description method of the high-dimensional feature of the feature amount at the time of inference from Tensor or the like to two-dimensional space. Thus, in the case where the task at the time of learning is image classification (Image Classification), it is possible to perform recognition beyond the task.

[0168] If the representative feature amount with the highest similarity is, for example, a representative feature amount of a drone, the three-dimensional position estimation unit 15 identifies that the kind of the detection target object appearing in the inference target image is a drone, and identifies the region in which the detection target object exists.

[0169] If the representative feature amount with the highest similarity is, for example, a representative feature amount of a flying car, the three-dimensional position estimation unit 15 identifies that the kind of the detection target object appearing in the inference target image is a flying car, and identifies the region in which the detection target object exists.

[0170] If the representative feature amount with the highest similarity is, for example, a representative feature amount of a helicopter, the three-dimensional position estimation unit 15 identifies that the kind of the detection target object appearing in the inference target image is a helicopter, and identifies the region in which the detection target object exists.

[0171] The three-dimensional position estimation unit 15 outputs the estimation result of the three-dimensional position to the change analysis unit 16.

[0172] Further, the three-dimensional position estimation unit 15 generates display data expressing the estimation result of the three-dimensional position, and outputs the display data to the display device 5.

[0173] Further, each time the image signal acquisition unit 11 acquires an image signal, the three-dimensional position estimation unit 15 outputs the estimation result of the three-dimensional position to the change analysis unit 16. If the sampling time at which the image signal acquisition unit 11 acquires an image signal is t n , the three-dimensional position estimation unit 15 outputs the estimation result of the three-dimensional position at the sampling time t n to the change analysis unit 16.

[0174] The change analysis unit 16 acquires the estimation result of the three-dimensional position at the sampling time t n from the three-dimensional position estimation unit 15, and provides the estimation result of the three-dimensional position to the second learning model 2a.

[0175] Further, the change analysis unit 16 provides the estimation result of the three-dimensional position of the drone to the second learning model 2a in a case where it is necessary to acquire position data expressing the temporal change of the three-dimensional position of the drone.

[0176] The change analysis section 16 supplies the estimation result of the three-dimensional position of the air car to the second learning model 2a in a case where position data indicating a time change of the three-dimensional position of the air car is required to be acquired.

[0177] The change analysis section 16 supplies the estimation result of the three-dimensional position of the helicopter to the second learning model 2a in a case where position data indicating a time change of the three-dimensional position of the helicopter is required to be acquired.

[0178] The second learning model 2a is a learning model that learns a three-dimensional position to recursively perform a time change of the three-dimensional position. Therefore, the second learning model 2a, when supplied with the estimation result of the three-dimensional position at the sampling time t n , outputs position data indicating a time change of the three-dimensional position of the detected object at a future sampling time corresponding to the estimation result to the change analysis section 16.

[0179] The change analysis section 16 acquires, from the second learning model 2a, for example, position data indicating a time change of the three-dimensional position of the detected object at the future sampling time t n+1 ~ t n+3 , as position data indicating a time change of the three-dimensional position of the detected object at a future sampling time.

[0180] In the inference device 4 illustrated in FIG. Figure 1 , the change analysis section 16 acquires position data indicating a time change of the three-dimensional position of the detected object using the second learning model 2a. However, this is only an example, and the change analysis section 16 can also supply the estimation result of the three-dimensional position of the detected object at the sampling time t n to a prediction function for predicting the three-dimensional position of the detected object, thereby acquiring position data indicating a time change of the three-dimensional position of the detected object.

[0181] The change analysis section 16 determines whether or not the detected object lands based on the position data.

[0182] For example, if the future position of the detected object indicates a position on the ground, the change analysis section 16 determines that the detected object lands. If the future position of the detected object does not indicate a position on the ground, the change analysis section 16 determines that the detected object does not land.

[0183] In addition, the change analysis section 16 determines, based on the position data, whether or not the detected object is, for example, hovering, ascending, descending, advancing, or retreating, as a process of classifying the detected object into a plurality of classes.

[0184] The change analysis section 16 generates display data of an analysis result indicating a time change in the three-dimensional position of the detected object, and outputs the display data to the display device 5.

[0185] In addition, the change analysis section 16 generates display data of a determination result indicating the above-described determination processing, and outputs the display data to the display device 5.

[0186] The display device 5 displays the estimation result of the three-dimensional position on a display not shown in accordance with the display data output from the three-dimensional position estimation section 15. The estimation result of the three-dimensional position indicates the kind of the detected object and the three-dimensional position of the detected object.

[0187] In addition, as shown in Figure 9 , the display device 5 displays the analysis result of the time change in the three-dimensional position of the detected object and the determination result such as whether the detected object landed or not on a display not shown in accordance with the display data output from the change analysis section 16.

[0188] Figure 9 is an explanatory view indicating a time change in the three-dimensional position of the detected object.

[0189] Figure 9 The x-direction position of the detected object at time t and the y-direction position of the detected object at time t are shown.

[0190] In Figure 9 , the x-direction is a direction orthogonal to the photographing direction of the camera 3, for example, a direction horizontal to the ground.

[0191] The y-direction is a direction parallel to the photographing direction of the camera 3. The z-direction is a direction orthogonal to the photographing direction of the camera 3, for example, a direction vertical to the ground.

[0192] In Figure 9 the example, the z-direction position of the detected object becomes the position on the ground at a certain time, and thus the determination result indicating the meaning that the detected object landed is explicitly shown. In Figure 9 , the determination result indicating the meaning that the detected object landed can be displayed as a "message".

[0193] In the above Embodiment 1, the inference device 4 is configured to include: an image signal acquisition section 11 that acquires an image signal representing an inference target image that is an image in which a detection target object is imaged in one or more different cases in which a domain of the image is different from that of a learning image and a recognition task is different from that of a task in which learning has been performed in advance; and a feature amount extraction section 12 that supplies the image signal acquired by the image signal acquisition section 11 to the first learning model la in which learning has been completed for the learning image, and acquires an inference-time feature amount from the first learning model la, the inference-time feature amount being a feature amount that is used in the inference of the three-dimensional position of the detection target object and is obtained by combining a plurality of feature amounts of the detection target object imaged in the inference target image after the plurality of feature amounts are blurred respectively. In addition, the inference device 4 includes: a three-dimensional position estimation section 15 that estimates the three-dimensional position of the detection target object imaged in the inference target image on the basis of a representative feature amount that is a registered feature amount of the detection target object imaged in a transformed image of an object in which the domain of the image and the recognition task are the same as those of the inference target image, and the inference-time feature amount acquired by the feature amount extraction section 12; and a change analysis section 16 that analyzes the temporal change in the three-dimensional position of the detection target object imaged in the inference target image on the basis of the estimation result of the three-dimensional position by the three-dimensional position estimation section 15. Thus, the inference device 4 can analyze the temporal change in the three-dimensional position of the detection target object without causing the first learning model to learn in consideration of the state of the natural environment.

[0194] In addition, in Embodiment 1, the inference device 4 is configured such that the image signal acquisition section 11 acquires an image signal representing a transformed image, and the feature amount extraction section 12 supplies the image signal representing the transformed image to the first learning model la, and acquires a representative feature amount that is a feature amount obtained by combining a plurality of feature amounts of a detection target object imaged in the transformed image after the plurality of feature amounts are blurred respectively. In addition, the inference device 4 includes a representative feature amount registration section 13 that registers the representative feature amount acquired by the feature amount extraction section 12. Thus, the inference device 4 can register the representative feature amount that can be used in the estimation of the three-dimensional position of the detection target object.

[0195] In Figure 1 In the inference device 4 illustrated in FIG. 1, the feature amount extraction section 12 supplies an image signal to the first learning model la implemented by a very deep CNN, and acquires a feature amount obtained by combining a plurality of feature amounts of a detection target object after the plurality of feature amounts are blurred respectively from the first learning model la.

[0196] In a case where the first learning model 1a is implemented by a very deep CNN, as described above, even if the domain of the inference target image represented by the image signal supplied to the input layer of the first learning model 1a is different from the learning image and even if the detection target object is different, the difference in the feature vector output from the output layer is small.

[0197] On the other hand, in a case where the first learning model 1a is implemented by a general neural network or the like, if the domain of the inference target image represented by the image signal supplied to the input layer of the first learning model 1a or the detection target object represented in the inference target image is different from the learning image, sometimes the difference in the feature vector output from the output layer becomes large.

[0198] However, the domain of the transformed image is the same as the domain of the inference target image. Therefore, even in a case where the first learning model 1a is implemented by a general neural network or the like, if the detection target object represented in the inference target image is a normal object, the representative feature amount acquired by the feature amount extraction section 12 at the time of domain transformation and the inference-time feature amount acquired by the feature amount extraction section 12 at the time of position estimation become approximately the same value.

[0199] On the other hand, if the detection target object represented in the inference target image is an abnormal object, the representative feature amount acquired by the feature amount extraction section 12 at the time of domain transformation and the inference-time feature amount acquired by the feature amount extraction section 12 at the time of position estimation become largely different values.

[0200] Thus, even in a case where the first learning model 1a is implemented by a general neural network or the like, the three-dimensional position estimation section 15 is able to estimate the three-dimensional position of the detection target object with high precision.

[0201] Further, the present disclosure can be implemented with deformation of any of the structural elements of the embodiments or omission of any of the structural elements of the embodiments.

[0202] Industrial Applicability

[0203] The present disclosure is suitable for an inference device, an inference method, and an inference program.

Claims

1. An inference device comprising: an image signal acquisition section that acquires an image signal representing an inference target image that represents an image of a detection target object in one or more different cases from a case where a domain of the image is different from a learning image and a case where a recognition task is different from a task that has been learned in advance; a feature quantity extraction section that provides the image signal acquired by the image signal acquisition section to a first learning model in which learning of the learning image has been completed, acquires an inference-time feature quantity from the first learning model, the inference-time feature quantity being a feature quantity that is used in inference of a three-dimensional position of the detection target object and is obtained by combining a plurality of feature quantities of the detection target object that are blurred respectively, the plurality of feature quantities being represented in the inference target image; a three-dimensional position estimation section that estimates the three-dimensional position of the detection target object represented in the inference target image based on a representative feature quantity and the inference-time feature quantity acquired by the feature quantity extraction section, the representative feature quantity being a registered feature quantity of the detection target object represented in a transformed image that is an object whose domain and recognition task are the same as those of the inference target image; and a change analysis section that analyzes a temporal change in the three-dimensional position of the detection target object represented in the inference target image based on an estimation result of the three-dimensional position by the three-dimensional position estimation section. 2.The inference device according to claim 1, wherein the image signal acquisition section acquires an image signal representing the transformed image, the feature quantity extraction section provides the image signal representing the transformed image to the first learning model, acquires the representative feature quantity from the first learning model, the representative feature quantity being a feature quantity that is used in inference of the three-dimensional position of the detection target object and is obtained by combining a plurality of feature quantities of the detection target object that are blurred respectively, the plurality of feature quantities being represented in the transformed image, and the inference device comprises a representative feature quantity registration section that registers the representative feature quantity acquired by the feature quantity extraction section. 3.The inference device according to claim 1, wherein the three-dimensional position estimation section compares representative feature quantities of a plurality of detection target objects whose kinds and regions where they exist are different from each other with the inference-time feature quantity acquired by the feature quantity extraction section, determines a representative feature quantity corresponding to the inference-time feature quantity acquired by the feature quantity extraction section among the representative feature quantities of the plurality of detection target objects, and estimates the kind and the three-dimensional region where the detection target object exists respectively as recognition of the detection target object represented in the inference target image based on a determination result of the representative feature quantity. 4.The inference device according to claim 1, wherein ​ ​ ​ ​ ​ ​ ​ ​ ​ The change analysis section obtains position data indicating a temporal change in the three-dimensional position of the detection target object from a second learning model that learns a temporal change in a three-dimensional position by recursively learning a three-dimensional position, by providing the second learning model with an estimation result of a three-dimensional position by the three-dimensional position estimation section.

5. An inference method comprising: An image signal acquisition section acquires an image signal indicating an inference target image that represents an image of a detection target object in one or more different cases out of a case where a domain of the image is different from a learning image and a case where an identification task is different from a task that has been learned in advance, A feature amount extraction section obtains an inference-time feature amount from a first learning model whose learning of a learning image has been completed, by supplying the first learning model with an image signal acquired by the image signal acquisition section, the inference-time feature amount being a feature amount that is used in inference of a three-dimensional position of the detection target object and is obtained by combining a plurality of feature amounts of the detection target object that are blurred respectively after being represented in the inference target image, A three-dimensional position estimation section estimates a three-dimensional position of the detection target object represented in the inference target image, based on a representative feature amount that is a registered feature amount of the detection target object represented in a transformed image of an object that is the same as the inference target image in terms of a domain of the image and an identification task, and the inference-time feature amount obtained by the feature amount extraction section, A change analysis section analyzes a temporal change in the three-dimensional position of the detection target object represented in the inference target image, based on an estimation result of a three-dimensional position by the three-dimensional position estimation section.

6. A computer-readable recording medium recording an inference program for causing a computer to execute the following processes: An image signal acquisition process in which an image signal acquisition section acquires an image signal indicating an inference target image that represents an image of a detection target object in one or more different cases out of a case where a domain of the image is different from a learning image and a case where an identification task is different from a task that has been learned in advance, A feature amount acquisition process in which a feature amount extraction section obtains an inference-time feature amount from a first learning model whose learning of a learning image has been completed, by supplying the first learning model with an image signal acquired by the image signal acquisition section, the inference-time feature amount being a feature amount that is used in inference of a three-dimensional position of the detection target object and is obtained by combining a plurality of feature amounts of the detection target object that are blurred respectively after being represented in the inference target image, A three-dimensional position estimation process in which a three-dimensional position estimation section estimates a three-dimensional position of the detection target object represented in the inference target image, based on a representative feature amount that is a registered feature amount of the detection target object represented in a transformed image of an object that is the same as the inference target image in terms of a domain of the image and an identification task, and the inference-time feature amount obtained by the feature amount extraction section, and The change analysis process analyzes a temporal change in the three-dimensional position of the detected object present in the inference target image, based on the estimation result of the three-dimensional position by the three-dimensional position estimation unit.

Images