Information processing device, information processing method, and program

Abstract

To provide an information processing apparatus, an information processing method, and a program.SOLUTION: In an information processing system in which an information processing apparatus 100 is communicatively connected via a network 110, the information processing apparatus has a control unit. The control unit acquires first depth information having depth as information based on an image of a first moving object, acquires a degree of similarity between the first moving object and a second moving object based on the first depth information, and predicts a first parameter for the first moving object based on a second parameter and the similarity for the second moving object.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 discloses a structure for improving air resistance without impairing the appearance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-151303 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, since it was necessary to build an actual vehicle in order to know the air resistance of the structure disclosed in Patent Document 1, there was a need for a technology that could more efficiently predict the physical quantities of a moving body. [Means for solving the problem]

[0005] According to one aspect of the present invention, there is provided an information processing device. The information processing device includes a control unit. The control unit acquires first depth information having depth information based on an image of a first moving object. The control unit acquires a similarity between the first moving object and a second moving object based on the first depth information. The control unit predicts a first parameter related to the first moving object based on a second parameter related to the second moving object and the similarity. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a diagram illustrating an example of a system configuration of an information processing system. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device. [Figure 3]FIG. 3 is an activity diagram illustrating an example of information processing according to the first embodiment. [Figure 4] FIG. 4 is a conceptual diagram showing an example of processing for acquiring synthesis information according to the first embodiment. [Figure 5] FIG. 5 is an image diagram showing an example of overlapping silhouettes according to the first embodiment. [Figure 6] FIG. 6 is a conceptual diagram showing an example of information processing according to the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0007] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the accompanying drawings. Various features shown in the following embodiments can be combined with each other.

[0008] Incidentally, the program for realizing the software appearing in this embodiment may be provided as a non-transitory computer-readable medium, or may be provided so that it can be downloaded from an external server, or may be provided so that the program is started on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0009] In this embodiment, the term "unit" may also include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In addition, various types of information are handled in this embodiment, and this information may be represented by, for example, physical values ​​of signal values ​​representing voltages and currents, high and low signal values ​​as a binary bit set consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculations may be performed on a circuit in the broad sense.

[0010] In addition, a circuit in the broad sense is a circuit realized by at least appropriately combining a circuit, circuitry, a processor, a memory, etc. That is, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0011] [Embodiment 1] 1. System Configuration FIG. 1 is a diagram illustrating an example of a system configuration of an information processing system. As shown in FIG. 1, the information processing system includes an information processing device 100 and a network 110. The information processing device 100 is communicably connected via the network 110. The information processing device 100 executes the process according to the embodiment. For example, the information processing device 100 generates first depth information D having depth information based on an image of a first moving object. A The information processing device 100 acquires the similarity between the first moving body and the second moving body based on the first depth information. The information processing device 100 predicts the first parameter related to the first moving body based on the second parameter related to the second moving body and the similarity. This makes it possible to more efficiently predict the physical quantity of the moving body.

[0012] For the sake of simplicity, FIG. 1 illustrates only one information processing device 100 in the information processing system. However, the information processing system includes as many information processing devices 100 as there are users. An example of the information processing device 100 is a personal computer (PC). However, this does not limit the present embodiment, and the information processing device 100 may be a smartphone, a tablet computer, or the like.

[0013] 2. Hardware Configuration In this section, the hardware configuration of the present embodiment will be described. The hardware configuration of the information processing device 100 will be described below.

[0014] FIG. 2 is a diagram illustrating an example of a hardware configuration of the information processing device 100. As shown in FIG. The information processing device 100 has a control unit 201, a storage unit 202, a communication unit 203, an input unit 204, and an output unit 205, and these components are electrically connected via a communication bus inside the information processing device 100. Each component will be further described below.

[0015] The control unit 201 processes and controls the overall operations related to the information processing device 100. The control unit 201 is, for example, a central processing unit (CPU). The control unit 201 reads out a predetermined program stored in the storage unit 202 and executes processing based on the program, thereby realizing various functions related to the information processing device 100, for example, the processing shown in FIG. 3 described below. Note that the number of control units 201 is not limited to one, and the information processing device 100 may be implemented with multiple control units 201 for each function. Alternatively, a combination of these may be used. In another embodiment, a predetermined program stored in a storage unit of another information processing device on the network 110 may be read via the communication unit 203 (described later) and the network 110, and processing may be executed based on the program.

[0016] The storage unit 202 stores various pieces of information defined above. This can be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs and the like related to the information processing device 100 executed by the control unit 201, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to the program operations. The storage unit 202 stores various programs and variables related to the information processing device 100 executed by the control unit 201, and data and the like used when the control unit 201 executes processing based on the programs.

[0017] The communication unit 203 is preferably a wired communication means such as USB, IEEE1394, Thunderbolt, or wired LAN network communication, but may also include wireless LAN network communication, mobile communication such as 3G / LTE / 5G, Bluetooth (registered trademark) communication, etc. as necessary. In other words, it is more preferable to implement it as a collection of multiple communication means. In other words, the information processing device 100 may communicate various information from the outside via the communication unit 203 and the network 110.

[0018] The input unit 204 may be included in the housing of the information processing device 100 or may be externally attached. For example, the input unit 204 may be integrated with the output unit 205 and implemented as a touch panel. The touch panel allows the user to input tapping, swiping, and the like. Of course, a switch button, a mouse, a QWERTY keyboard, or the like may be used instead of the touch panel. That is, the input unit 204 accepts an operation input made by the user. The input is transferred as a command signal to the control unit 201 via a communication bus, and the control unit 201 can execute predetermined control or calculation as necessary.

[0019] The output unit 205 may be included in the housing of the information processing device 100, or may be externally attached. The output unit 205 displays a graphical user interface (GUI) screen that can be operated by a user. This is preferably implemented by selectively using display devices such as a CRT display, a liquid crystal display, an organic EL display, and a plasma display depending on the type of information processing device 100.

[0020] 3. Information Processing Method In this section, an information processing method executed by the above-described information processing device 100 will be described.

[0021] 3.1 Overview of Information Processing FIG. 3 is an activity diagram illustrating an example of information processing according to the first embodiment.

[0022] In A301, the control unit 201 of the information processing device 100 receives input of input shape data 401 via the input unit 204. In this embodiment, the control unit 201 uses images of a vehicle drawn from multiple viewpoints as the input shape data 401.

[0023] In A302, the control unit 201 receives an instruction to start calculation via the input unit 204.

[0024] When an instruction to input input shape data 401 and start calculation is given, in A303 the control unit 201 analyzes the given input shape data 401 to obtain depth information, normal information, and silhouette information.

[0025] In A304, the control unit 201 acquires composite information by synthesizing the acquired depth information, normal information, and silhouette information. Acquisition of the depth information, normal information, silhouette information, and composite information will be described in detail with reference to FIG. 4.

[0026] In A305, the control unit 201 compares the silhouette information included in the synthesis information with the silhouette information of other vehicles stored in the storage unit 202. The control unit 201 acquires non-overlapping and overlapping portions of the input vehicle silhouette information and the stored vehicle silhouettes. Details will be described using FIG. 5.

[0027] In A306, the control unit 201 acquires the similarity between the input vehicle silhouette information and the stored vehicle silhouette in the non-overlapping portion. This will be described in detail later using Equation 1.

[0028] In parallel with A306, in A307, the control unit 201 acquires the similarity of the overlapping portion between the input vehicle silhouette information and the stored vehicle silhouette. This will be described in detail later using Equation 2.

[0029] When two types of similarity are acquired in A306 and A307, in A308 the control unit 201 inputs the two types of similarity and the stored air resistance of the vehicle into the machine learning model.

[0030] In A309, the control unit 201 weights and adds two types of similarity in the machine learning model. At this time, the weighting is optimized in the machine learning model. The similarity obtained by this addition represents the closeness between the input vehicle and the stored vehicles. The control unit 201 predicts the air resistance of the input vehicle by performing regression between the similarity obtained by the addition and the air resistance of the stored vehicles. The detailed flow from A308 to A309 will be described in detail using Figure 6.

[0031] 3.2 Details of information processing Next, the above-mentioned information processing will be described in detail.

[0032] FIG. 4 is a diagram illustrating an example of an image of a process for acquiring composite information according to the first embodiment. FIG. 4 shows input shape data 401 and first depth information DA and the first normal information N A and the first silhouette information S A and first synthesis information 402. The input shape data 401 includes a vehicle drawn from multiple viewpoints. In this embodiment, the input shape data 401 is an image of the vehicle captured on software such as CAD. Here, the depth information is information that indicates the degree of depth when an object is projected onto a surface. In this embodiment, the first depth information D A The higher the vehicle is three-dimensionally when viewed from the projection surface, the darker the displayed image. Here, the normal information is information representing a line that intersects at right angles with the tangent line (or tangent plane) at the contact point when the object is projected onto the image. In this embodiment, the first normal information N A When viewed from the projected surface, the inclination of the normal line of the vehicle becomes closer to being parallel to the surface, and the displayed image becomes darker. Here, the silhouette information is information that represents the silhouette of an object when it is projected onto a surface. In this embodiment, the first silhouette information S A The first composite information 402 is displayed as the first depth information D A and the first normal information N A and the first silhouette information S A The combined information includes parameterized depth information, normal information, and silhouette information. When the input shape data 401 and the start of calculation are instructed via the input unit 204, the control unit 201 generates first depth information D having depth information as information based on the input shape data 401. A and first normal information N having information on the normal of the vehicle. A and first silhouette information S A Specifically, the control unit 201 acquires first depth information D for the plurality of viewpoints based on images of the vehicle drawn from the plurality of viewpoints. A and the first normal information N for multiple viewpoints A and the first silhouette information S for multiple viewpoints. A Next, the control unit 201 acquires the first depth information D A and the first normal information N Aand the first silhouette information S A Then, the control unit 201 acquires the similarities F1 and F2 based on the acquired first synthesis information 402. In other words, the control unit 201 acquires the first depth information D A and the first normal information N A and the first silhouette information S A Specifically, the control unit 201 obtains the similarities F1 and F2 based on the first depth information D A and the first normal information N for multiple viewpoints A and the first silhouette information S for multiple viewpoints. A The similarities F1 and F2 are obtained based on the above. This allows the similarities to be obtained from various information such as depth information, normal information, and silhouette information, making it possible to obtain more accurate similarities.

[0033] FIG. 5 is an image diagram showing an example of overlapping silhouettes according to the first embodiment. The Venn diagram information 500 includes first silhouette information S of the first vehicle. A and the second silhouette information S of the second vehicle B The first silhouette information S of the first vehicle is expressed as a set. A and second silhouette information S of the second vehicle B The overlapping part is the intersection set (A∩B), and the first silhouette information S of the first vehicle A and second silhouette information S of the second vehicle B The non-overlapping part is the symmetric difference set ((A∪B)-(A∩B)). The control unit 201 determines whether or not the first combined information 402 has been acquired. When the control unit 201 determines that the first combined information 402 has been acquired, the control unit 201 calculates the first silhouette information S of the first vehicle. A and the second silhouette information S of the second moving object. BBased on this, the control unit 201 acquires the overlapping portion (A∩B) of the silhouettes of the first moving body and the second moving body and the non-overlapping portion ((A∪B)-(A∩B)). Next, the control unit 201 acquires an overlap measure F1 and a depth similarity F2 for the overlapping portion (A∩B) of the silhouettes and the non-overlapping portion ((A∪B)-(A∩B)). Note that the overlap measure F1 and the depth similarity F2 are each an example of similarity. As a result, different methods of acquiring similarity are used depending on the overlapping of the silhouettes, and more accurate similarity can be acquired.

[0034] The control unit 201 determines whether or not a portion of the silhouettes where the silhouettes do not overlap ((A∪B)-(A∩B)) has been acquired. When the control unit 201 determines that a portion of the silhouettes where the silhouettes do not overlap ((A∪B)-(A∩B)) has been acquired, the control unit 201 acquires an overlap measure F1 for the portion of the silhouettes where the silhouettes do not overlap ((A∪B)-(A∩B)) based on the ratio of the areas of the silhouettes. Specifically, the control unit 201 calculates the overlap measure F1 for the portion of the silhouettes where the silhouettes do not overlap ((A∪B)-(A∩B)) based on the area ratio of the silhouettes. A and the second silhouette information S of the second vehicle B It can be obtained by dividing the intersection (A∩B) of A and B by the union (A∪B). More specifically, it can be obtained using the following equation 1. Here, the overlap measure F1 is also called the Jacquard coefficient. In equation 1, Area represents the area. This makes it possible to obtain the similarity between vehicles even when there is little overlap between the silhouettes.

[0035]

number

[0036] Furthermore, the control unit 201 determines whether or not the overlapping portion of the silhouettes (A∩B) has been acquired. When the control unit 201 determines that the overlapping portion of the silhouettes (A∩B) has been acquired, the control unit 201 calculates the depth similarity F2 of the overlapping portion of the silhouettes (A∩B) as the first depth information D A and second depth information D of the second vehicle BThe difference between the first normal information N A and second normal information N of the second moving body B Specifically, the control unit 201 calculates the depth similarity F2 of the overlapping silhouette portion (A∩B) based on the first depth information D A and second depth information D of the second vehicle B The difference between the first normal information N A and second normal information N B The average of and the product of are integrated over the overlapping portion (A∩B) of the silhouettes. More specifically, it is calculated by the following equation 2. In other words, the control unit 201 calculates the first depth information D A Based on this, the depth similarity F2 between the first vehicle and the second vehicle is obtained. In this way, since depth information is used, it is possible to obtain the similarity taking into account finer details of the vehicles.

[0037]

number

[0038] FIG. 6 is a conceptual diagram showing an example of information processing according to the first embodiment. FIG. 6 includes input shape data 401, first synthesis information 402, a machine learning model 610, and parameters 620. The input shape data 401 includes a moving object drawn from one or more viewpoints. The input shape data 601 may be an image in which only a portion of the moving object is drawn. The first synthesis information 402 is obtained by analyzing depth information of the input shape data 401. The machine learning model 610 performs processing based on the training data and the input synthesis information 602 to obtain parameters 620. An example of the machine learning model 610 is a convolutional neural network (CNN). In this embodiment, the training data is data in which synthesis information of other moving objects and the air resistance of the moving object are paired. The parameters 620 are physical quantities obtained as a result of regression analysis by the machine learning model 610. The parameters 620 may be any physical quantity related to the moving object, and in this embodiment, the parameters 620 are air resistance. The control unit 201 determines whether the overlap measure F1 and the depth similarity F2 have been acquired. If the control unit 201 determines that the overlap measure F1 and the depth similarity F2 have been acquired, the control unit 201 weights and adds up the two types of similarities in the machine learning model. At this time, the similarity F3 obtained by this addition represents the closeness between the input vehicle and the stored vehicles. Next, the control unit 201 predicts the air resistance of the input vehicle by regressing the similarity F3 obtained by the addition and the air resistance of the stored vehicles. That is, the control unit 201 predicts the first air resistance of the first vehicle based on the second air resistance of the second vehicle and the similarity between the first vehicle and the second vehicle. The control unit 201 predicts the first air resistance of the first vehicle based on the second air resistance of the second vehicle and the similarity, and acquires the reliability of the first air resistance. For example, the control unit 201 outputs 0.3 as the air resistance and a result that "the air resistance is 0.28 to 0.32 with an 80% probability" as the reliability. That is, the control unit 201 inputs the second parameter related to the second vehicle and the similarity to the machine learning model, and predicts the first parameter related to the first vehicle based on the output value from the machine learning model. Here, the machine learning model of this embodiment uses a Gaussian process. By using the Gaussian process, the control unit 201 acquires an expected value as a predicted value of the air resistance of the input shape data and a variance as a reliability based on the air resistance of the training data and the similarity between the input shape data and the training data. Furthermore, at this time, the similarity F3 functions as a kernel function in the Gaussian process. As a result, since a Gaussian process is used, parameters can be predicted with high accuracy when data similar to the training data is input. As described above, according to one aspect of the present invention, the physical quantity of a moving body can be predicted more efficiently.

[0039] [Variation 1] In the embodiment, an example has been described in which the entire vehicle is given as input shape data, but in Modification 1, only a part of the moving object may be given as input shape data. In this case, first, the control unit 201 receives an image of a portion of the moving object as input shape data. Next, the control unit 201 references the entire shape from the storage unit 202 and replaces only the input portion. Thereafter, the control unit 201 performs processing using the shape resulting from replacing only the input portion as input shape data. In this way, parameters for the entire shape can be obtained by preparing only partial data of the input shape.

[0040] Note that a vehicle is an example of a moving object. Other examples of a moving object include amusement park rides, roller coasters, airplanes, ships, trains, saddle-type vehicles such as motorcycles, bicycles, taxis, and buses. In addition, a moving object may be a vehicle that is not directly controlled by a user, such as a radio-controlled car or a drone.

[0041] It may be provided in the following manner. In the information processing device, the control unit acquires first silhouette information having information on the silhouette of the first moving body and first normal information having information on the normal of the first moving body based on the image of the first moving body, and acquires the similarity based on the first depth information, the first silhouette information, and the first normal information. In the information processing device, the control unit acquires first synthesis information that combines the first depth information, the first silhouette information, and the first normal information, and acquires the similarity based on the first synthesis information. In the information processing device, the control unit obtains overlapping portions of the silhouettes of the first moving body and the second moving body and non-overlapping portions of the silhouettes based on the first silhouette information and the second silhouette information of the second moving body, and obtains the similarity for each of the overlapping portions of the silhouettes and the non-overlapping portions of the silhouettes. In the information processing device, the control unit obtains the similarity of the overlapping portion of the silhouettes based on the product of the difference between the first depth information and the second depth information of the second moving body and the average of the first normal information and the second normal information of the second moving body. In the information processing device, the control unit acquires the similarity of the portions where the silhouettes do not overlap based on a ratio between an area of ​​the portions where the silhouettes overlap and an area of ​​the portions where the silhouettes do not overlap. In the information processing device, the image of the first moving body includes the first moving body drawn from multiple viewpoints, and multiple pieces of first depth information are obtained based on the images of the first moving body drawn from multiple viewpoints, and the control unit obtains the similarity based on the multiple pieces of first depth information. In the information processing device, the first parameter and the second parameter are air resistance. In the information processing device, the control unit predicts a first parameter related to the first moving body based on a second parameter related to the second moving body and the similarity, and obtains the reliability of the first parameter. In the information processing device, the control unit inputs a second parameter related to the second moving body and the similarity into a machine learning model, and predicts the first parameter related to the first moving body based on an output value from the machine learning model. In the information processing device, a Gaussian process is used in the machine learning model. In the information processing device, the first moving body and the second moving body are vehicles. An information processing method executed by an information processing device, which acquires a first depth image based on an image of a first moving body, and predicts a first parameter related to the first moving body based on the first depth image, a second depth image related to a second moving body at the same angle as the first depth image, and a second parameter related to the second moving body. A program for causing a computer to function as a control unit of the information processing device. Of course, this is not the case.

[0042] Finally, while various embodiments of the present invention have been described, they are presented by way of example only and are not intended to limit the scope of the invention. The novel embodiments may be embodied in various other forms, and various omissions, substitutions, and modifications may be made without departing from the spirit of the invention. Such embodiments and modifications are intended to be included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the accompanying claims. [Explanation of symbols]

[0043] 100: Information processing device 110: Network 201: Control unit 202: Storage section 203: Communications Department 204: Input section 205: Output section 401: Vehicle image 402: 1st synthesis information 500: Venn diagram information 610: Machine Learning Models 620: Parameter D A :1st depth information D B :Second depth information F1: Overlap measure F2: Depth similarity F3: Similarity N A :First normal information N B :First normal information S A :First silhouette information S B : Second silhouette information

Claims

1. An information processing device, A control unit is provided. The control unit acquiring first depth information having depth information based on the drawn image of the first moving object; acquiring first silhouette information having information on a silhouette of the first moving body and first normal information having information on a normal of the first moving body based on the image; acquiring a similarity between the first moving body and the second moving body based on the first depth information, the first silhouette information, and the first normal information; predicting a first parameter related to the first moving object based on a second parameter related to the second moving object and the similarity; the first parameter and the second parameter are parameters indicating predetermined physical quantities, acquiring an overlapping portion of the silhouettes of the first moving body and the second moving body and an overlapping portion of the silhouettes of the second moving body based on the first silhouette information and the second silhouette information of the second moving body; The similarity is obtained for each of the overlapping portions of the silhouettes and the non-overlapping portions of the silhouettes. Information processing device.

2. An information processing device, A control unit is provided. The control unit acquiring first depth information having depth information based on the drawn image of the first moving object; acquiring first silhouette information having information on a silhouette of the first moving body and first normal information having information on a normal of the first moving body based on the image; acquiring a similarity between the first moving body and the second moving body based on the first depth information, the first silhouette information, and the first normal information; predicting a first parameter related to the first moving object based on a second parameter related to the second moving object and the similarity; the first parameter and the second parameter are parameters indicating predetermined physical quantities, Information processing device.

3. 3. The information processing device according to claim 2, The control unit obtaining first combined information obtained by combining the first depth information, the first silhouette information, and the first normal information; obtaining the similarity based on the first combined information; Information processing device.

4. In the information processing device according to claim 3, The control unit acquiring an overlapping portion of the silhouettes of the first moving body and the second moving body and an overlapping portion of the silhouettes of the second moving body based on the first silhouette information and the second silhouette information of the second moving body; The information processing device acquires the similarity for a portion where the silhouettes overlap and a portion where the silhouettes do not overlap.

5. In the information processing device according to claim 1 or claim 4, The control unit An information processing device that obtains the similarity of the overlapping portion of the silhouettes based on the product of the difference between the first depth information and the second depth information of the second moving body and the average of the first normal information and the second normal information of the second moving body.

6. 6. The information processing device according to claim 4, The control unit The information processing device acquires the similarity of the portion where the silhouettes do not overlap based on a ratio between an area of ​​the portion where the silhouettes overlap and an area of ​​the portion where the silhouettes do not overlap.

7. 7. The information processing device according to claim 1, the image of the first moving object includes the first moving object rendered from a plurality of viewpoints; acquiring a plurality of pieces of first depth information based on images of the first moving object drawn from a plurality of viewpoints; The control unit acquires the similarity based on a plurality of pieces of first depth information.

8. 8. The information processing device according to claim 1, The information processing device, wherein the first parameter and the second parameter are air resistance.

9. 9. The information processing device according to claim 1, The control unit An information processing device that predicts a first parameter related to the first moving object based on a second parameter related to the second moving object and the similarity, and obtains a reliability of the first parameter.

10. 10. The information processing device according to claim 1, The control unit An information processing device that inputs a second parameter related to the second moving body and the similarity into a machine learning model and predicts the first parameter related to the first moving body based on an output value from the machine learning model.

11. 11. The information processing device according to claim 10, The information processing device, wherein the machine learning model uses a Gaussian process.

12. 12. The information processing device according to claim 1, The information processing device, wherein the first moving body and the second moving body are vehicles.

13. An information processing method executed by an information processing device having a control unit, The control unit executes the process according to any one of claims 1 to 12. Information processing methods.

14. A program, A program for causing a computer to function as a control unit of the information processing device according to any one of claims 1 to 12.

Images