Combined Injection and Ejection Device

Abstract

This invention provides a composite injection / injection device that efficiently arranges multiple types of devices on a single substrate. [Solution] The composite energy injection / exiting device comprises a plurality of first-type devices that inject or emit energy, a plurality of second-type devices that inject or emit energy and are of a different type from the first-type devices, and a substrate. The device synthesizes information measured by the injection or emission of energy from the plurality of first-type devices, and synthesizes information measured by the injection or emission of energy from the plurality of second-type devices. The plurality of first-type devices and the plurality of second-type devices are arranged on the substrate while satisfying both of the following two constraints: Constraint 1: Each of the plurality of first-type devices has at least one second-type device that is closest to it in azimuthal space; Constraint 2: Each of the plurality of second-type devices has at least one first-type device that is closest to it in azimuthal space.

Description

Technical Field

[0001] The present invention relates to a composite input / output device, and particularly to a technology for arranging a plurality of types of devices on a single substrate.

Background Art

[0002] In recent years, devices such as portable information terminals such as mobile phones and smartphones have been miniaturized, and at the same time, various additional functions have been installed, thereby achieving differentiation. In particular, the camera function has been enhanced, and it has become possible to capture a wide range of images and videos by mounting a wide-angle lens. For example, in Patent Document 1, "The surrounding cameras are each equipped with a camera on each constituent surface of a polyhedron such as a regular dodecahedron, and imaging images over the entire circumference can be obtained by each camera. By sequentially connecting the imaging images of adjacent cameras, a single panoramic image can be obtained. However, it is not possible to assemble the surrounding cameras so that the projection centers of each camera completely coincide. Therefore, when connecting the imaging images, depending on the perspective of the subject, by dynamically adjusting the connection positions of adjacent imaging images, the discontinuities and seams near the boundaries between the images are eliminated, and a smooth panoramic view is generated. (Abstract excerpt)" The configuration is disclosed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The distance measurement method according to Patent Document 1 involves connecting captured images, assuming the distance to the subject. The distance to the subject is estimated by dynamically adjusting the position and smoothly connecting the points. Because this is an indirect method, the accuracy of the obtained distance is not sufficient. Therefore, In addition to the imaging function, it would be desirable to have a separate distance measuring function to directly measure the distance to the subject. i. The frame used in the ambient camera of Patent Document 1 is wired and fixed in place. Therefore, a relatively large frame can be used, and in such cases, the imaging device By increasing the field of view and overlapping the imaging area, it is also possible to implement a distance measurement function using the stereo method. However, miniaturized devices like mobile information terminals require a full solid angle distance measurement function and a camera. When incorporating imaging capabilities, is it possible to implement the distance measurement function using a stereo method with an imaging device? There is a problem in that the distance measurement accuracy does not improve because the baseline length is shortened. Therefore, Patent Document 1 There are still challenges in directly applying the described ambient camera technology to small devices.

[0005] This invention was made to solve the above problems, and it allows multiple types of devices to be used in one The objective is to provide a composite injection / injection device efficiently arranged on a substrate. [Means for solving the problem]

[0006] To solve the above problems, the present invention comprises the configuration described in the claims. An example thereof. To give an example, the present invention comprises a plurality of first type devices that inject or emit energy, and Multiple Type 2 devices, which are of a different type from the Type 1 device, are injected or ejected with energy. The present invention comprises a base on which the plurality of first-type devices and the plurality of second-type devices are arranged. Combine the information measured by the incidence or emission of energy of the plurality of first-type devices, combine the information measured by the incidence or emission of energy of the plurality of second-type devices, The plurality of first-type devices and the plurality of second-type devices are arranged on the substrate while satisfying both of the following two constraint conditions; Constraint condition 1: Each of the plurality of first-type devices is such that the closest device in the azimuth space is at least one of the second-type devices, Constraint condition 2: Each of the plurality of second-type devices is such that the closest device in the azimuth space is at least one of the first-type devices. It is characterized by the above.

Advantages of the Invention

[0007] According to the present invention, a composite input / output device in which multiple types of devices are efficiently arranged on a single substrate can be provided. Other objects, configurations, and effects than those described above will be clarified in the following embodiments.

[0008]

Brief Description of the Drawings

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The same configuration will be described throughout the drawings. The symbol "1" is used to indicate the omission of redundant explanations. In the following explanation, we will refer to combinations of different devices. The imaging device (corresponding to a Type 1 device) and the distance measuring sensor (corresponding to a Type 2 device) The following is an example, but it is not limited to this.

[0010] (First Embodiment) In the first embodiment, in addition to the substrate 2 (see Figure 1) on which the device is mounted, there is a first type device and And a direction definition that defines the central direction of the Type 2 device (the direction that points towards the center of the input / output range). Consider polyhedron 5 (see Figure 3A). Here, "orientation" refers to the direction of the center of the input / output range. It refers to the orientation in real space. Also, "azimuth space" is the set of directions in real space and It is a space defined as follows. For example, the real space is defined by the angle (θ, φ) of the polar coordinate system (r, θ, φ). The direction between them is represented. And, when vectors in real space are translated in parallel, overlapping points are of the same direction. It is assumed to have a position. "Substrate 2" actually exists in real space and is a Type 1 device and It is an object equipped with a Type 2 device. In contrast, "orientation-defining polyhedron 5" is a base 2 An approach is introduced to determine the central orientation of the Type 1 and Type 2 devices installed in the device. It is a conceptual shape and not an actual object. "Orientation-Defining Polyhedron 5" is its geometric pair Based on symmetry, a group of directions consisting of multiple directions with good symmetry is defined within an "directional space." .

[0011] An orientation group is, for example, a group of faces of the same shape from the center position of "Orientation Definition Polyhedron 5". This is a group of directions that face the heart (more details will follow). "Directional Definition Polyhedron 5" is the first The shape has symmetry in at least two directions, including symmetry and a second symmetry. The first type of debits corresponding to each of the multiple orientations that the polyhedron 5 contains which have the first symmetry. The chair is placed on the base 2 with the corresponding orientation as the central orientation of the device. A second type of device corresponding to each of the multiple orientations having the second symmetry contained within facet 5. The corresponding orientation is placed on the base 2 with the device's central orientation. For both Type 2 devices, the placement position ensures that base unit 2 does not fall within the input / output range of each device. Set it up as follows. However, even if base 2 interferes with the input / output range, the individual devices The input / output range is sufficiently wide, and when the input / output ranges of each type of device are combined, the total solid angle If it is covered, interference is acceptable.

[0012] Furthermore, even if the direction being interfered with does not require input or output, the direction is still affected. Interference may be present in the input / output range of some devices. The device will have less interference from base 2. The arrangement of the chairs is such that if base 2 has the same shape as or is close to the shape of azimuthal polyhedron 5, then the center of base 2 is... This is the case when the orientation of the device as seen is close to the orientation of the device's center. In other words, the central orientation of the device is, on average, close to the direction perpendicular to the surface of the substrate 2. Furthermore, in this arrangement, the central orientations of the Type 1 and Type 2 devices are in the azimuth space. Because they are nested, the placement positions on the base 2 are also nested, and the device The chair placement efficiency will improve. If the shape of base 2 is different from the shape of orientation-defining polyhedron 5, The basic configuration is defined as the position where the orientation of the device as viewed from the center of body 2 is the orientation of the device's center. Therefore, taking into account the shape of base 2 and implementation constraints, the center position of the device is maintained, Adjust the position on body 2 by moving it from its basic position. In this case as well, the adjustment is made from the basic position. Therefore, the device placement efficiency is good.

[0013] Figure 1 is an external view of the composite injection device 1 according to the first embodiment. The composite injection device 1 is A mobile information terminal, such as a smartphone, is considered the base body 2, and the directional polyhedron 5 contains it. With the orientation satisfying the first symmetry as the central orientation, multiple Type I devices 1, 2, ..., M( 11-1, 11-2, ..., 11-M) are arranged. Furthermore, azimuthal polyhedron 5 is contained within. With the orientation satisfying the second symmetry as the central orientation, multiple Type II devices 1, 2, ... N(11-1, 11-2, ..., 11-N) is arranged. On the substrate 2 of each device The placement position is determined considering the implementation constraints of base 2. Orientation-defining polyhedron 5, first symmetry Sex and second symmetry will be discussed later.

[0014] A controller 3 is provided inside the base 2. The controller 3 controls multiple Type 1 devices. Each of chairs 1, 2, ..., M (11-1, 11-2, ..., 11-M), and multiple chairs Each of the Type 2 devices 1, 2, ..., N (11-1, 11-2, ..., 11-N) It is connected. Controller 3 consists of a computer including a processor and circuits. The number of Type 1 devices, M, and the number of Type 2 devices, N, are 2 or greater. It is an integer. M and N may be the same number or different numbers. Also, multiple first kind Vices 1, 2, ..., M (11-1, 11-2, ..., 11-M) are grouped together as Class 1. Device group 11, Type 2 devices 1, 2, ..., N(11-1, 11-2, ..., 1 Devices 1-N are collectively referred to as the Type 2 device group 12.

[0015] Each of the Type 1 devices 1, 2, ..., M (11-1, 11-2, ..., 11-M) is , wide-angle lens, CCD (Charge-Coupled Device) sensor or C MOS(Complementary metal-oxide-semiconductor) It is an imaging device composed of sensors, etc. Type 1 devices 1, 2, ..., M(1 Each of 1-1, 11-2, ..., 11-M) is positioned in the appropriate location and orientation on base 2. By positioning them accordingly, it becomes possible to image the entire solid angle.

[0016] Each of the Type 2 devices 1, 2, ..., N (11-1, 11-2, ..., 11-N) is For example, a TOF sensor that measures the distance to a person or object. Type 2 devices 1, 2, Each of the ...N(11-1, 11-2, ..., 11-N) is positioned appropriately on the substrate 2. By positioning them in the correct direction, it becomes possible to measure the distance of objects across the entire solid angle. .

[0017] Figure 2 is an overall block diagram of the combined inlet / outlet device 1 according to the first embodiment. Controller 3 in position 1 controls the first type device processor 2 which controls the first type device group 11. 11. Second-type device processor 212, CPU (Ce (Intral Processing Unit) 16, CPU 16 processing, Type 1 device Programs and various settings for controlling the S group 11, the second type device group 12, etc. ROM (Read Only Memory) 13 for retention, Type 1 device group 1 Image data output from 1 and distance measurement data output from the second type device group 12 (hereinafter The imaging data and distance measurement data (collectively referred to as "measurement data") are temporarily stored and executed. RAM14 provides the work area for the program, measurement data, imaging data, and also pre-configured External memory 15 for storing created or captured video data and image data, and system battery Includes S17.

[0018] Figure 3A shows the first type devices 1, 2, M(11-1, 11-2) according to the first embodiment. Each of the following, and the second type of device 1, 2, ...N (11-1, 11- The following shows examples of arrangements based on the respective orientation-defining polyhedra 5 of 2, ..., 11-N).

[0019] As azimuthal polyhedra 5, regular polyhedra, semi-regular polyhedra, or Catalan solids (semi-regular polyhedra) (The dual polyhedron) is used. In Figure 3A, the shape of orientation-defined polyhedron 5 is a cuboctahedron.

[0020] Each of the first type device group 11 and the second type device group 12 contains the orientation-defining polyhedron 5. They are arranged using the symmetry described above. Specifically, the center of the azimuthal polyhedron 5 is used as the reference point, and the center The directional group consisting of directions viewed from the center of the plane (or the centroid) is called the first symmetry, and from the center The orientation group consisting of orientations viewed from the edge center is the second symmetry, and the orientation group consisting of orientations viewed from the center to the vertex is the second symmetry. The orientation group is defined as the third symmetry. It contains 5 identical orientation-defining polyhedra with good symmetry. With the orientation of the orientation group as the central orientation, Type 1 devices 1, 2, ..., M(11-1, 11-2, ..., 11-M), Type 2 device 1, 2, ..., N (11-1, 11- If you arrange 2, ..., 11-N) respectively, they will not interfere with each other and will be in azimuth space This allows for efficient nesting of all solid angles. For example, the orientation of the face center (1st With symmetry) as the central orientation, Type 1 devices 1, 2, ..., M(11-1, 11-2, ...) ..., 11-M) are arranged, and the vertex-center orientation (third symmetry) is used as the central orientation for the second kind of Debye Arrange S1, 2, ..., N(11-1, 11-2, ..., 11-N). Since you can place as many different types of devices as there are groups, there can be three or more types of devices. That's fine.

[0021] Furthermore, imaging devices that use ultra-wide-angle lenses such as fisheye lenses are one imaging device When the field of view is 180 degrees or more using only the directional elements, all directions within the directional group are intentionally used. It is not necessary to use this method to position the devices. However, even in this case, when combining different types of devices... In this case, the orientation group where the different types of devices are placed and the nested orientation group are part of the same orientation group. The image device is positioned. This results in an efficient device arrangement with good overall symmetry. This allows for reduced interference. In this case, near a small number of devices... In this case, the arrangement becomes nested in azimuth space. The arrangement is such that, taking mounting errors into consideration, the first proximity device of the device with fewer units This can be described as the second nearest device being a different type of device.

[0022] Figure 3A is also an example of a device arrangement when the base 2 is equal to the orientation-defining polyhedron 5. Figure 3 In A, the direction from the center of the cuboctahedron toward the center of the quadrilateral faces is defined as the central direction, and there are 6 points in 6 locations. Each of the individual Type 1 devices 11-1, 11-2, and 11-3 (the remaining three are omitted in the diagram) These are arranged. The remaining three, which are not shown in the illustration, are on the square faces in the opposite, unseen positions. It is located there.

[0023] Furthermore, the direction from the center of the cuboctahedron toward the center of the triangular faces is defined as the central direction, and there are 8 points at 8 locations. Type 2 devices 12-1, 12-2, 12-3, 12-4 (the remaining four are omitted in the diagram) Each of the following is placed. The remaining four, which are not shown in the diagram, are placed in the correct position on the opposite side where they are not visible. They are arranged on a triangular surface.

[0024] Imaging range of Type 1 devices 11-1, 11-2, and 11-3, and Type 2 device 12-1. The ranging ranges for 12-2, 12-3, and 12-4 are, respectively, the input / output angle ranges of each individual device. These are combined to cover the entire solid angle. From the perspective of a single Type I device, the nearest in azimuthal space. A contact device is a Type II device, and from the perspective of one Type II device, in azimuthal space... The nearest nearest device is a Type 1 device.

[0025] If the input / output angle range of individual devices is greater than the minimum required value, strictly the azimuth It does not need to match the symmetry orientation contained within the defined polyhedron 5; it must cover the entire solid angle. There is some tolerance for the orientation in which the chair is installed.

[0026] For example, as a Type 1 device, it is used in combination with a CCD or CMOS sensor with a wide-angle lens. Multiple imaging devices are arranged to enable imaging of the entire solid angle. Distancing as a Type 2 device. For example, multiple TOF (Time of Flight) sensors can be placed. The entire solid angle is measured. A TOF sensor is used to illuminate the object with light from the image sensor side and reflect it off the object. By measuring the time lag for each pixel until it returns, and measuring the distance, a two-dimensional distribution can be achieved. When used in conjunction with a row of sensors (CMOS or CCD), it allows for two-dimensional determination of the distance to the subject. ru.

[0027] However, while it is possible to grasp the data in two dimensions, this requires arranging the TOF sensor two-dimensionally on the base 2. It is necessary. Even if a wide-angle lens is installed, the widest angle lens is 120-14 It is around 0 degrees. However, since the pixel size is about 10 μm, the size of the sensor itself The size can be kept to just a few millimeters square. Therefore, it is possible to mount multiple units on base 2. Therefore, if the number, placement, and angle of TOF sensors are appropriately determined, the distance measurement across the entire solid angle will be It is easy.

[0028] The placement of the devices is such that, as long as the orientation of the devices is the same, base 2 (in this case) Not a specific position (face center position, edge center position, vertex position) of the azimuthal-defining polyhedron 5 That's good too.

[0029] Furthermore, the shape of base 2 may differ from the shape of the orientation-defining polyhedron 5. In that case, The direction that Vice points is the direction of the orientation group contained within the orientation-defining polyhedron 5. For example In Figure 3A, the cuboctahedron imaging device and the distance measuring sensor are used, but the orientation of the device is not changed. Alternatively, a rectangular prism can be used as the base body 2, and the faces of the rectangular prism can be positioned at the center and vertex positions. This is shown in Figure 3B.

[0030] In Figure 3B, an imaging device as a Type 1 device is placed on each face of a rectangular parallelepiped, and a Type 2 device is placed on each face of the rectangular parallelepiped. Distance sensors, as devices, are placed at the vertices of each rectangular prism. In this case, the imaging device is The direction it is facing is the same as the direction from the center of the rectangular prism towards the center of the face, but the distance measuring sensor is The direction is different from the direction from the center of the rectangular prism to the vertices. This is the direction that the equilateral triangular faces of the octahedron face. Figure 3B shows that the shape of base 2 is a smartphone. Since it has the same shape as a tablet device (rectangular prism), it is actually a portable composite injection device 1. It can also be used in this way. Note that the "direction the imaging device is facing" refers to the direction in which the image is taken. This refers to the orientation of the center line of the image device's field of view, and "the orientation the distance measuring sensor is facing" means the direction the distance measuring sensor is facing. This is the direction of the center line of the scan range.

[0031] Figures 4A, 4B, and 4C show the above-mentioned devices as regular polyhedra, semi-regular polyhedra, and Catalan polyhedra. This figure shows an example of arrangement using the symmetry inherent in each three-dimensional object. The example here is a substrate. This is the case where the shape of polyhedron 2 and the shape of azimuthal polyhedron 5 are the same. Figure 4A shows the view from the center of the cube. Figure 4B shows the arrangement of devices using orientation groups in the direction of the center of the face (centroid). Figure 4C shows the arrangement of devices using point orientation groups. This diagram shows the devices arranged using positional groups.

[0032] Even within the same cube, the number of devices installed differs depending on the orientation group. Whether to adopt this approach depends on the input and output angles of the device being mounted. See below for details. As stated below.

[0033] The combination of each installed device enables actual distance measurement and shooting across the entire solid angle. The required input and output angles for each device are explained using Figures 5 and 6.

[0034] Figure 5A shows the first type of device (11-1)(11-2)(11) on the square faces of a cuboctahedron. -3) is positioned (reverse side not shown), and a Type 2 device (12-1) is on the equilateral triangular surface. (12-2)(12-3)(12-4) are arranged (reverse side diagram omitted). Example here This also applies when the base 2 and the orientation-defining polyhedron 5 are the same. A first-type device mounted on a square face For a chair to cover the entire solid angle (to measure the entire solid angle), it must be within the group of Type 1 devices. The azimuthal range within the pyramid is composed of the azimuthal vectors of the nearest subgroups at the input and output angular range. The enclosure should cover it. That is, the azimuth range within the triangular pyramid formed by the X, Y, and Z axes in Figure 5A (this This constitutes one unit of the measurement range of a Type 1 device) with margin (ingress / outgression of adjacent Type 1 devices) It is necessary that the area, including the overlapping region with the firing angle range, be covered by the input and output firing angle range of the Type 1 device. It is essential.

[0035] Figure 5B shows the orientation of all devices in polar coordinates, expressed as polar declination. Type 1 devices The direction of (11-1) is the direction of θ=0 degrees, so in Figure 5B, the circle in Figure 5A is represented as a band. It is noted. The required input / output angle range is shown in Figure 5B, for one device of type 1 devices (e.g., For example, if the input / output angle range of the first type device 11-2) covers up to the central axis of the pyramidal apex angle, This means that in this case, "central axis direction" refers to the direction of the Type 2 device (12-1) in Figure 5A. This corresponds to the direction. That is, the direction of V in Figure 5A. This corresponds to the value of α shown in Figure 5B. The required input / output angle range is the angle α formed by the direction of the Type 1 device and the central axis direction of the pyramidal apex angle. It doubles.

[0036] In Figure 5B, this is the area indicated by the dashed line around the Type 1 device 11-2. Therefore, since α becomes 54.7°, the required entry and exit angles become twice that, 109.5°. First, the number of directions in each direction group for regular tetrahedrons, cubes, regular octahedrons, and cuboctahedrons. Figure 6 shows the required input / output angle range (which can also be expressed as the number of mounted devices). Like the orientation groups of the faces of a cuboctahedron, the orientation groups are determined by the type of face, edge, or vertex. These can sometimes be further subdivided. This is just one example, and the same may be true for other polyhedra. The required input / output angle range can be determined using a similar approach. The above applies to Type 1 devices. The description is as follows, but naturally the same thinking applies to Type 2 devices as well, and the required input / output angle range. It is possible to decide.

[0037] If the above required entry and exit angles are met, and a TOF sensor is used as the distance measuring sensor, adjacent The measurement ranges of the TOF sensor and the other sensor will overlap, but this is handled by complementing each other's data. Completing the process enables highly accurate measurements.

[0038] In other words, with a TOF sensor, the measurement accuracy is higher for data closer to the center of the field of view than for data from the periphery. Since the value will be higher, compare the data from adjacent TOF sensors and use the measurement from the sensor closer to the center of the field of view. Using distance data enables more accurate measurement of the total solid angle distance. The required input shown in Figure 6 The emission angle is determined by placing a TOF sensor in each directional group of the polyhedron. This indicates the required field of view. For example, in a regular hexahedron, TOF sensors are placed at each vertex. This means we need 8 devices, but if each TOF sensor has a field of view of 109.5 degrees, then we can use 8 of them. This makes it possible to measure the distance across the entire solid angle. Similarly, in a cuboctahedron, if we place them at each vertex, then 12 This requires several TOF sensors, but the required field of view is 90 degrees, which is a small field of view for TOF sensors. However, it is possible to measure the distance of the entire solid angle. The overlap of the field of view between the nearest identical devices. If such a device exists, as mentioned above, more accurate measurements can be obtained by supplementing the measurement data from adjacent devices. It can obtain a fixed image. The above is a description of a TOF sensor as a distance measuring sensor, Similarly, for example, with regard to imaging devices, if there is overlap with adjacent images, the image closer to the center of the field of view will be the one that is closest to the center. Since the distortion is less, it is best to select the image closer to the center of the field of view. In this way, The input / output angle range of a device is a crucial factor in determining the number of devices that can be mounted.

[0039] Figure 6 is a table summarizing the orientation groups of various polyhedra, the number of orientations, and the required range of entry and exit angles. Figure 6 shows the cases of a regular tetrahedron, cube, regular octahedron, and cuboctahedron, mainly focusing on the number of polyhedra. While this is listed, it is not limited to these, and includes other regular polyhedra, semi-regular polyhedra, and Catalan solids. The same approach can be used to determine the required input and output angle range for various polyhedra, and for devices. The number of units that can be installed can also be determined.

[0040] In other words, this embodiment uses a distance measuring sensor with a small solid angle measurement range to measure the entire solid angle and a camera. Regarding the combination of full solid angle imaging, in order to efficiently cover the entire solid angle, the rangefinder system, A key feature is the selection of a combination of directions that exhibit good symmetry in both the imaging system and the imaging system. In light of this, Using the symmetry inherent in regular polyhedra, semi-regular polyhedra, and Catalan solids as defined polyhedra 5 Specifically, this refers to the view from the center of various regular polyhedra, semi-regular polyhedra, and Catalan solids. Using orientation groups for each type, such as the face center (centroid) orientation, edge center orientation, and vertex orientation, each The orientations of the orientation groups contained within a seed regular polyhedron (or semi-regular polyhedron or Catalan polyhedron), By arranging the distance measuring sensor and imaging device, they can be efficiently nested together without interfering with each other. It can cover the entire solid angle.

[0041] Furthermore, if the imaging device has a wide-angle lens with a field of view of 180 degrees or more, two opposing directions By positioning the device in this way, imaging of the entire solid angle becomes possible. In this case, all of the azimuth groups The device does not need to be positioned in that direction. However, even in this case, it must be combined with a different type of device. In this case, the orientation group where the different types of devices are placed and the nested orientation group are part of the same orientation group. By arranging the image devices, an efficient device arrangement with good overall symmetry can be achieved. This allows for the reduction of interference between different devices.

[0042] Furthermore, as will be discussed later, as a variation, the axis passing through the center of each device must change direction. They don't need to be concentrated at one point. This increases the degree of freedom in placement. Also, another variation For example, if the input and output angles covered by the vertical and horizontal directions of the device are different, the face centers of the rhombic polyhedron. Using directions also improves placement efficiency.

[0043] Next, regarding the mounting positions of each imaging device and distance measuring sensor mentioned above, we will specifically discuss the mounting arrangement. An example of the method will be explained using Figures 7A and 7B. However, the explanation below is just one example, so the arrangement examples are not provided. Not limited to the following.

[0044] Figure 7A shows an example of a portable information terminal with a roughly rectangular (hexahedral) shape on which the composite input / output device 1 is implemented. A roughly rectangular prism is the base body 2. And for example, a regular octahedron is defined as the orientation-defining polyhedron 5, and its orientation Use a group to position the distance sensor and imaging device. The center orientation of the distance sensor is... If a plane-center orientation group is adopted, eight distance measuring sensors must be placed. However, since a mobile information terminal is hexahedron-shaped, mounting eight distance measuring sensors on each face is not feasible. It is not possible. To solve this, eight distance measuring sensors are used as shown in Figure 7A, for example, above Place 4 on the faces and 4 on the base, oriented towards the orientation group of the face center orientation group of a regular octahedron.

[0045] Furthermore, when placing distance sensors on the edges of a portable information terminal, grooves (or chamfers) should be made on each edge. (This can be done and the sensor may be mounted there.) The field of view of the distance sensor at this time is the azimuth of the regular octahedron in Figure 6. The group's face-center orientation must be 109.5 degrees or greater. The above refers to grooves on each side of the mobile information terminal. This describes a method for creating a device and placing a distance measuring sensor, but it is not limited to this method. The method is irrelevant as long as the center orientation of the distance sensor can be positioned in the aforementioned orientation.

[0046] Meanwhile, the imaging devices 110 and 111 are positioned in the direction of the edge center orientation group of the regular octahedron. Here, assuming the imaging device has a field of view of 180 degrees or more, from among the side-center orientation groups... By selecting two opposing directions and mounting them on two faces of the hexahedron of the mobile information terminal in symmetrical positions... This makes it possible to capture the entire solid angle.

[0047] Figure 7B shows the distance sensor connected to each vertex orientation group of a regular octahedron, with orientation-defining polyhedron 5. An example of arranging the positions with the central orientation is shown. In this case, 3 are placed on the top surface and 3 on the bottom surface. Specifically, the top of the mobile device is cut off diagonally so that it faces the direction of the aforementioned directional group. It may also be mounted on the vehicle. In this case as well, the field of view of the distance measuring sensor only needs to be 109.5 degrees or more.

[0048] On the other hand, imaging devices 110 and 111 use the face-center orientation group orientation of a regular octahedron as the central orientation. Arrange them as follows. Here, assuming that the field of view of the imaging device is 180 degrees or more, the plane center orientation is Select two opposing directions from within the loop, and the two symmetrical positions on the hexahedron of the mobile device By mounting it on a surface, it becomes possible to capture the entire solid angle. In this example, the surface center orientation is Although imaging devices are not placed in all directions of the loop, all distance measuring sensors are in the azimuth space. The nearest device in this configuration is the imaging device.

[0049] Thus, the same regular polyhedra, semi-regular polyhedra, and Catalan solids as azimuthal-defining polyhedra 5 Using the orientation of the enclosed orientation group as the central orientation, the imaging device and distance measuring sensor are efficiently mounted on the base 2. By positioning them correctly, it becomes possible to measure distances and take photographs across the entire solid angle.

[0050] Additionally, considering nighttime or dark-room photography, it may be desirable to install lighting for the entire solid angle. However, this can also be addressed by mounting multiple lights as emission devices using the same approach. In this case, the central orientation of the imaging device and the central orientation of the illumination device are nested in azimuthal space. By arranging them in this way, direct interference from the lighting device to the imaging device is reduced. It is effective.

[0051] Figure 8 shows, for example, the four distance measuring sensors 12-1, 12-2, 12-3, 12- This example shows how the distance measurement data from each of the four devices is combined into a single distance measurement data set.

[0052] Each distance measuring sensor 12-1, 12-2, 12-3, and 12-4 includes a measurement range. The measurement range will have a larger error in distance measurement in the peripheral areas. This is the case with TOF sensors, for example. In cases like this, because the measurement system uses an optical lens, the angle of view in the camera sensor This is because the distortion increases towards the periphery. Therefore, as shown in Figure 8, the center Measurement accuracy is worse in the peripheral areas compared to the central areas.

[0053] Therefore, it is best to avoid using the peripheral area as much as possible. Therefore, multiple distance measuring sensors 12- The respective measurement ranges of 1, 12-2, 12-3, and 12-4 are adjacent to each other in azimuthal space. The sensor is positioned to overlap with the measurement range, and the area with the best measurement accuracy is used preferentially. This allows for accurate measurements even when performing full solid angle distance measurement. This choice can be made, for example, beforehand. The relative positions of adjacent distance measuring sensors are known during the design phase, so the distance measurement responsibility for each distance measuring sensor is determined. You just need to define the range. The remaining distance measuring sensors 12-5 to 12-8 shown in Figure 7A The same procedure is applied to the sensors, enabling highly accurate distance measurement across the entire solid angle.

[0054] Figure 9 shows an example of combining multiple distance measurement data into a single distance measurement data. In Figure 9, distance information The information is represented using three patterns, but this is not the only way; in reality, there are even finer variations in shade. The distance can be calculated accurately, and color-coding is also possible.

[0055] The CPU 16 reads distance measurement data from each of the distance measuring sensors 12-1 to 12-4, and These may be combined and trimmed to generate the full solid angle distance measurement data 800. Distance measurement data The synthesis process may be performed using only the distance measurement data from each distance measuring sensor, or using multiple imaging devices. You can use the image data from the chair to supplement the distance measurement data and perform a synthesis process on the distance measurement data. Yes. More details will follow later.

[0056] When combining distance measurement data, discrepancies may occur in the data at the connection points of the measurement range. This is because the geometric focal points of the light incident on each distance sensor do not necessarily coincide, Depending on the distance to the target object, the azimuth range covered by each distance measuring sensor will shift. This is the case. A method for correcting data in such cases will be explained using Figure 10.

[0057] The imaging data shown in Figure 10A is, for example, based on imaging device 11-1 in Figure 2. Figure 10B shows two distance measuring sensors 12-1 and 12-2 that measure the range of the imaging data described above. This document describes the data synthesis method. In reality, the proximity device of imaging device 11-1 The chair has four distance sensors, so the data will be a combination of four, but to simplify the explanation... To simplify things, we will describe the two proximity distance measuring sensors. Also, the two distance measuring sensors 12 The measurement ranges of -1 and 12-2 include areas outside the imaging range of imaging device 11-1. However, this explanation describes measurements within the imaging range of the imaging device 11-1. If the distance measuring sensors 12-1 and 12-2 are positioned in a geometrically accurate position, angle, and direction... Assuming this is the case, the correction will only be for distortion in the peripheral areas. In this case, the synthesis of the distance measurement data is shown in Figure 8. Therefore, you should use the data with higher accuracy, whichever is more precise, to combine the data, based on the measurement range and its positional relationship.

[0058] However, in actual products, the distance measuring sensor is positioned at a location that is offset from its geometric position. In other words, the "focusing point" (the point where the incident signal geometrically converges) of each device aligns to a single point. If it is not present, or due to distortion and issues with the mounting position and accuracy of the distance measuring sensor itself, even if the field of view is Even if they overlap, there is a possibility that the data cannot be connected cleanly. Therefore, the subject The data connection needs to be dynamically adjusted according to the distance. This is what we use for this purpose. This is imaging data. For example, when an image is taken of a target object, that target object is measured by the distance measuring sensor 12-1 When measured with 12-2, the range of the distance measurement data is as shown in Figure 10B, due to the shift in the focal point and the handling Due to mounting precision, there will be a slight misalignment. Therefore, this misalignment must be corrected to match the aforementioned imaging data. The distance measurement data is corrected and the connections between regions are shaped (see Figure 10C). This correction is applied to the object It is also possible to calculate this from the distance, device characteristics, and arrangement, but using imaging data as a reference. Further reshaping can improve the accuracy of data in boundary regions.

[0059] According to the first embodiment, when arranging multiple types of devices on a single substrate 2, the orientation definition The device is placed in azimuthal space using different symmetry orientation groups contained within polyhedron 5. Because they are arranged in a child-like manner, the central orientation of the base 2 is affected by whether the devices are of different or the same type. Without doing so, multiple types of devices can be efficiently arranged to enable measurement of the entire solid angle. ru.

[0060] (Second Embodiment) The second embodiment is an embodiment in which the combined input / output device 1a is applied to the portable information terminal 4. Figure 11A is an external view of the composite inlet / outlet device 1a according to the second embodiment. Figure 11B is a composite inlet This figure shows an example of the display on the injection device 1a.

[0061] The combined input / output device 1a uses a shooting device and a distance measuring sensor on a portable information terminal 4, which serves as the base 2. It is equipped with a display 20 for checking the distance and image of measured objects or people. The configuration of the device and distance sensor is the same as shown in Figure 5B. It is hidden in Figure 11A. The parts that are shown are not indicated.

[0062] As shown in Figure 11B, the combined inlet / outlet device 1a displays text or other information to indicate that measurement is in progress. You may do so. Additionally, after measurement and shooting, you may display the full solid angle images and distance information. At this time, by rotating the mobile information terminal 4 in directions such as left, right, up, or down, the display 20 will appear. The image of the full solid angle is displayed. At this time, the gyro sensor of the mobile device (see Figure 12) and The system is linked to the mobile information terminal 4 to check its direction and display the corresponding image.

[0063] Figure 12 shows the overall block of the combined injection / injection device 1a.

[0064] The portable information terminal 4 includes a wide-angle lens and an image sensor (e.g., CCD or) as a Type 1 device. A camera 110 including a CMOS sensor, a wide-angle lens and an image sensor, and a display The front camera 111, camera 110, and front camera 111 have the play 20 side as their imaging range. Image processing processor 112 and Type 2 device perform image processing based on the imaging data from each. N TOF sensors as 1, 2, ..., N(12-1), (12-2), ... (12-N), ROM13, RAM14, External memory15, CPU16, System bus1 7. Gyroscope 18, accelerometer 19, display 20, LAN communication device 21, power Network communication device 22, and GPS (Global Positioning System) It is equipped with 23.

[0065] The image processing processor 112 corrects the angle and rotation of the captured image, or wide-angle lenses. Therefore, a distortion correction unit 114 corrects the distorted image to make it appear as it should, including a person. A subject recognition unit 116 recognizes faces and objects, and based on the subject recognition unit 116, it processes image data. A trimming unit 115 for cutting out faces and object parts, and an image for combining images from the multiple cameras mentioned above. Includes a composite section 117.

[0066] In this embodiment, at least two or more cameras are mounted. For example, as described above, It features two cameras, one for the front and one for the regular camera, with a wide-angle lens providing at least a 180-degree field of view. By mounting them on different surfaces, it becomes possible to capture images of the entire solid angle. Yes, there are cameras. You can install any number of cameras as long as there are two or more.

[0067] The image processing processor 112 acquires data from either the camera 110 or the front camera 111. First, distortion correction is performed on the captured image data by the distortion correction unit 114, and then the subject recognition unit 116... After recognizing people or objects, the trimming unit 115 trims the people or objects, This is not the only way; the distortion caused by the ultra-wide-angle lens is corrected in a simplified manner before recognition. You can perform more precise distortion correction after trimming, or, as will be discussed later, use a rangefinder group. The detailed distances to people and objects obtained from 12 are then displayed on the display 20. That's good too.

[0068] Figure 13 shows the full solid angle imaging and distance measurement algorithm executed by the image processing processor 112. show.

[0069] The image processing processor 112 receives imaging data from the camera 110 and wide-angle lens The first distortion correction unit 114-1 corrects the distortion, and the image data from the in-camera 111 It includes a second distortion correction unit 114-2 that receives input and corrects the distortion of the wide-angle lens. The distortion correction unit 114-1 includes a first aberration correction unit and a first positive image conversion unit. Similarly, the second distortion correction unit... The positive section 114-2 includes a second aberration correction section and a second positive image conversion section. As described above, with one camera... In contrast, there is one distortion processing unit, but this is not the only one, and high-speed processing is possible despite time constraints. If none exist, a single processing function can perform the processing by switching between functions.

[0070] The first distortion correction unit 114-1 and the second distortion correction unit 114-2 each perform distortion correction processing. The combined image is output to the image synthesis unit 117. The image synthesis unit 117 processes the first distortion correction unit 114- The distortion-corrected images acquired from the first and second distortion correction units 114-2 are combined to form a full 3D model. An image of the corners (hereinafter referred to as the "full solid angle image") is created and output to the subject recognition unit 116.

[0071] The subject recognition unit 116 performs subject recognition processing based on the full solid angle image and outputs the result to Output to trimming unit 115.

[0072] The trimming unit 115 processes the subject recognized by the subject recognition process within the entire solid angle image. The image area is cropped. The subject area is completely cropped, leaving only the subject area. You can either extract the image, or you can add a frame that encloses the subject area. (Cropping) You may apply further distortion correction to the resulting image, or enlarge the cropped portion. You may also highlight the location and, as described later, display the distance to that point.

[0073] Meanwhile, the CPU 16 processes the distance measurement data of each TOF sensor 12-1, 12-2, ... 12-n. The data is acquired and combined with the total solid angle distance measurement data. The CPU 16 measures the field of view of the distance sensor. The collected distance measurement data is complemented by each other to create a total solid angle distance measurement data of 800 (Figure 8, Figure 8). See 9).

[0074] The CPU 16 acquires the full solid angle image from the image synthesis unit 117 (LA in Figure 13), and then... This is associated with the body angle ranging data 800. There are various methods for this, but for example, a pre-composite input / output firing device The subject is placed at a known position around position 1a, and a full solid angle image is created. Using group 12, the distance from the combined input / output device 1a to the same subject is measured, and the total solid angle distance measurement is performed. Create a -800. Then, the subject area within the full solid angle image in which the same subject was captured and The distance information to the same subject read from the total solid angle distance measurement data 800 is associated with This allows us to determine the distance between the imaging area within the full solid angle image and the subject captured within that imaging area. Calibration data, associated with the information, is created and stored.

[0075] When the user uses the combined input / output device 1a, the CPU 16 creates a full solid angle image and Referencing calibration data, read out distance information corresponding to the imaging area of ​​the full solid angle image. The CPU 16 reads the distance information and the corresponding solid angle image of the imaging area. The position (for example, coordinates within the full solid angle image) is output to the distance information addition unit 118.

[0076] The distance information addition unit 118 uses the subject area information acquired from the trimming unit 115 to perform the following: Distance information obtained from CPU16 is added to the subject area in the full solid angle image.

[0077] Figure 14 shows an example of displaying a full solid angle image with distance information added. In Figure 14, the display The display range of Play 20 is only a part of the total solid angle, but when displaying the total solid angle, the gyro The display 20 is rotated vertically and horizontally based on the data from the sensor 18. You can either let it display as is, or you can scroll the display 20 left, right, forward, and backward to show the content. Alternatively, distance information can be displayed for a specific object, showing the distance to that object as shown in the diagram.

[0078] Figure 15 shows the full solid angle image and full solid angle distance measurement data 800 in the combined input / output device 1a. This is a flowchart showing the creation process flow. The process flow described below is explained in the first embodiment. This also applies to the combined injection / injection device 1.

[0079] If a user of the combined input / output device 1a wants to perform full solid angle imaging and full solid angle distance measurement, the user Switch the operating mode of the combined input / output device 1a to full solid angle shooting and full solid angle distance measurement mode. (S100). The combined injection device 1a displays a mode select screen on the display 20. The system accepts mode selection operations from the user. The combined input / output device 1a performs full-solid angle imaging and full-solid angle imaging. When switching to solid angle distance measurement mode, the display 20 shows "Currently shooting in full solid angle & distance measurement system" The system displays messages such as "System in operation" and prompts users to take precautions such as not shaking the device as much as possible. You can.

[0080] Camera 110 and front camera 111 each start up and begin shooting. Also, each rangefinder... Sensors 12-1, 12-2, ..., 12-N are activated and distance measurement begins (S101).

[0081] The image data generated by camera 110 and in-camera 111 respectively is processed by the image processing processor. The output is sent to 112. The image processing processor 112 performs distortion correction caused by the wide-angle lens. (S102) Create a full solid angle image (S103).

[0082] The CPU 16 (Type 2 device processor 212 in the first embodiment) controls the distance measuring sensor 12 The distance measurement data measured at -1, 12-2, ...12-N are combined to obtain the full solid angle distance measurement data. Create 800 (S104). Steps S103 and S104 are executed simultaneously. Alternatively, step S104 may be performed first.

[0083] The image processing processor 112 detects the subject area from the full solid angle image (S105), The subject area is cropped and extracted. Then, the total solid angle is created using S104. Distance information from the combined input / output device 1a to the subject is added using the distance measurement data 800 (S 106).

[0084] The image processing processor 112 processes the full solid angle image with distance information added, and the full solid angle distance measurement data The data is recorded to at least one of RAM 14 or external memory 15 (S107), and the process terminates.

[0085] Figure 16 shows the full solid angle image and full solid angle distance measurement data 800 in the combined input / output device 1a. This is a flowchart showing the playback process flow. The following process flow will be explained in the first embodiment. This also applies to the combined injection / injection device 1.

[0086] The combined input / output device 1a switches to playback mode for full solid angle images and full solid angle distance measurement data. Perform the operation (S110). The combined inlet / outlet device 1a displays the mode selector on the display 20. A screen is displayed, and mode selection operations are accepted from the user. Furthermore, the combined inlet / outlet device 1a This refers to the data to be played back from the data recorded by the user in RAM14 or external memory15. The system accepts the operation to select a file and plays that file (S111).

[0087] The files to be played back include image information and distance information covering a wider area than display 20, i.e., the entire solid angle. The report is included (see Figure 14). Therefore, the combined inlet / outlet device 1a uses a gyro sensor 18 Based on position information from the accelerometer 19 and GPS 23, the system calculates azimuth information and elevation angle information in the horizontal plane. The data is calculated and includes an image of a specific orientation of the full solid angle image (a partial image of the full solid angle image) and distance information. Display it.

[0088] Subsequently, the combined input / output device 1a provides the user with a scrolling function for the display 20, or I request that the combined injection device 1a be operated (S112).

[0089] The combined injection device 1a responds to user movements (for example, scrolling operations or the combined injection device 1a itself). When movement is detected (S113 / Yes), the image and distance in that direction are displayed according to the instructions. Display the information (S114).

[0090] If there is no user activity (S113 / No), or after step S114, the termination condition is met. For example, if there is no user instruction or action for a certain period of time, or if the user requests to end playback (S 115 / Yes), the playback process is terminated. If the termination condition is not met (S115 / No), Return to step S112.

[0091] Figure 17A shows the combined input / output device 1 as an HMD (Head Mounted Display). An example of its application to 50 is shown.

[0092] A venue for displaying AR (Augmented Reality) using an HMD (Head-Mounted Display). In conjunction with this, the system captures and measures the distance of the HMD user's real-world surroundings, and the captured data and distance data are used. This may be used for AR display. This allows for the superimposition of virtual objects onto real space. The control becomes more precise.

[0093] As shown in Figure 17A, the HMD50 consists of a temporal attachment 50-1 and a top attachment 50- 2, and a transparent or opaque display 50-3. A first camera 51-1, which has a wide-angle lens at the center of its upper edge, is positioned in the occipital region of the temporal head attachment 50-1. It is equipped with a second camera 51-2 in the center, which has a wide-angle lens. Also, the left and right of the display 50-3 Each of the vertices is equipped with a first distance measuring sensor 52-1 and a second distance measuring sensor 52-2. Furthermore, the top of the head is equipped The head-mounted body 50-2 is equipped with a third distance measuring sensor 52-3 on the top of the head, and the side head-mounted body 50-1 A fourth distance measuring sensor 52-4 is provided in the center of the back of the head. That is, as shown in Figure 17B, in this example... In the HMD50, a regular tetrahedron is used as the azimuthal-defining polyhedron 5. The camera has a field of view of 180°. Using values ​​of degrees or greater, two opposing directions (51-1, 51-) are selected from the side-center direction group. 2) The distance measuring sensor is positioned in the direction of the vertex center.

[0094] When creating full solid angle images and full solid angle distance measurement data with HMD50, the full solid angle image It is desirable that the image attachment point be located directly below the HMD50, that is, on the side of the HMD50 wearer. This is because, for HMD50 wearers, their own images are often unnecessary. The same applies to distance measurement data; there is almost no demand for knowing the distance to oneself. Therefore, it is desirable that the junction of the distance measurement data also be located on the side of the HMD50 wearer. Therefore, as a modified example of the sensor arrangement, as shown in Figures 18A and 18B, adjacent sensors While ensuring the necessary incident angle range with respect to the sensor, the camera's orientation is shifted from the orientation of the side center. Then, position the junction of the imaging data and distance measurement data so that it is directly below the HMD50, and adjust the direction of each sensor accordingly. Adjust the orientation. Figure 18B shows the orientation of each device in azimuthal space. When 51-1 and 51-2 are positioned in the direction of the side center, 51-1 and 51-2 adjust their orientation. This is a modified version. In the modified version, if the positive direction of the Z axis is selected to be vertically upward, the user's body Since the Z-axis becomes negative and the camera's optical axis becomes horizontal, both the imaging data and the distance measurement data are In this modified form, the joint will be located on the side of the HMD50 wearer. Therefore, the nearest device in the azimuthal space of each individual device is a heterogeneous device.

[0095] According to this embodiment, even small portable information terminals like smartphones can have multiple types of devices. The chairs were positioned so that the placement of different and similar devices did not overlap, and the entire solid angle was measured within the range. It can be placed within an enclosure.

[0096] The above embodiments are merely examples of embodiments of the present invention, and the present invention is not limited to those described above. It is not a means to an end. For example, it can also be used as a video conferencing system in another embodiment. Yes, it is. By utilizing distance information in the entire solid angle and object recognition, for example, in a video conferencing system... By using facial recognition images and distance information of all attendees, the system can determine the location of each person attending the meeting. In combination with a directional microphone, it identifies the person speaking and displays them enlarged. Alternatively, it becomes possible to clearly record the conversation after identifying the content and attendees. .

[0097] Other examples of injecting and ejecting devices include radio wave transmitting antennas and receiving antennas. There is an antenna. As the frequency used by mobile phones increases and its directivity strengthens, It is necessary to efficiently position antennas to cover the entire solid angle as the range of radio wave input and output. The present invention is also effective in this case.

[0098] Furthermore, other examples of injector and ejector devices include highly directional microphones and directional There is a speaker with strong sound. In audio input / output devices such as smart speakers, the surrounding area This also applies when enabling voice communication targeting specific users from among existing users. The invention is valid.

[0099] Furthermore, the hardware configuration of the combined injection / injection devices 1 and 1a is just one example, and one CPU 16 The first type device processor 211, the second type device processor 212, and the image processing processor You may perform each of the functions of 112. [Explanation of symbols]

[0100] 1, 1a: Composite input / output device 2: Base 3: Controller 4: Mobile information terminals 5: Orientation-defining polyhedron 11: Type 1 Device Group 12: Type 2 Device Group 13: ROM 14: RAM 15: External memory 16:CPU 17: System bus 18: Gyro sensor 19: Accelerometer 20: Display 21:LAN communication device 22:Telephone network communication device 23: GPS 50: HMD 50-1: Temporal head attachment 50-2: Head-mounted device 50-3: Display 51-1: Camera 1 51-2: Second camera 52-1: First distance measuring sensor 52-2: Second distance measuring sensor 52-3: Third distance measuring sensor 52-4: Fourth distance measuring sensor 110: Camera 111: Front camera 112: Image processing processor 114: Distortion Correction Section 114-1: Correction section 114-2: Correction section 115: Trimming section 116: Subject recognition section 117: Image Synthesis Unit 118: Distance information addition unit 211: Type 1 device processor 212: Type II Device Processor 800: Total solid angle ranging data

Claims

1. Multiple Type I devices that inject or emit energy, Multiple Type 2 devices, which are of a different type from the Type 1 device, are injected or emitted energy. A chair and The system comprises a base on which the plurality of Type 1 devices and the plurality of Type 2 devices are arranged. 、 Information measured by the incidence or emission of energy from the plurality of Type 1 devices is synthesized The information measured by the injection or emission of energy from the plurality of Type 2 devices is combined accomplish, The plurality of first-type devices and the plurality of second-type devices are subject to both of the following two constraints. The form is filled and placed on the substrate; Constraint 1: Each of the plurality of Type 1 devices is the nearest device in azimuthal space. S is at least one of the above-mentioned second type devices. Constraint condition 2: Each of the plurality of Type 2 devices is in the nearest device in azimuthal space. The device is at least one of the aforementioned Type 1 devices. A composite injection device characterized by the following features.

2. In the composite inlet / outlet device according to claim 1, An azimuthal polyhedron is defined, The aforementioned orientation-defining polyhedron has a first symmetry with respect to a point within the orientation-defining polyhedron as the reference point and This shape incorporates a second symmetry distinct from the first symmetry, Each of the plurality of first type devices satisfies the first symmetry in the orientation-defining polyhedron. Arranged in the direction of addition, Each of the plurality of Type 2 devices satisfies the second symmetry in the orientation-defining polyhedron. It is placed in the direction of the plus sign. A composite injection device characterized by the following features.

3. In the composite inlet / outlet device according to claim 2, The aforementioned azimuthal polyhedron is a regular polyhedron, a semi-regular polyhedron, or a Catalan solid. A composite injection device characterized by the following features.

4. In the composite inlet / outlet device according to claim 3, The first symmetry and the second symmetry are defined by the distance from the reference point to the face of the orientation-defining polyhedron. Directional group directed towards the center or centroid, direction from the reference point toward the side center direction of the base Either a group, or an orientation group directed from the reference point toward the vertex orientation of the base. ru, A composite injection device characterized by the following features.

5. In the composite inlet / outlet device according to claim 1, Each of the plurality of first type devices is an injector device, and each of the plurality of second type devices Each of the above is an injection device, or each of the above-mentioned multiple first-type devices is an injection device, and the Each of the multiple Type II devices is an injector device. The energy emitted from the emission device is directly injected into the injection device. The plurality of first-type devices and the plurality of second-type devices are positioned in an orientation and location where there is no such orientation and location Placed on the substrate, A composite injection device characterized by the following features.

6. In the composite inlet / outlet device according to claim 1, The aforementioned substrate is a portable information terminal. A composite injection device characterized by the following features.

7. In the composite inlet / outlet device according to claim 1, The information of the total solid angle is synthesized from the information measured by the aforementioned multiple Type 1 devices, A processor that synthesizes information on the total solid angle from data measured by multiple Type II devices. It also has, A composite injection device characterized by the following features.

8. In the composite inlet / outlet device according to claim 1, One of the first type device and the second type device is an imaging device, and the other is a distance measuring device. It is a sensor, A composite injection device characterized by the following features.

9. In the composite inlet / outlet device according to claim 8, The imaging data measured by the imaging device and the distance measurement data measured by the distance measuring sensor It further includes a processor that uses this to display the image information of the entire solid angle as a composite. A composite injection device characterized by the following features.

10. In the composite inlet / outlet device according to claim 9, When the processor synthesizes the distance measurement data as distance measurement information for the entire solid angle, the distance measurement Distance measurement data obtained from the azimuth angle, which is measured with higher precision within each measurement range of the sensor. Extract and synthesize the 'ta'. A composite injection device characterized by the following features.

Images