Information processing system, information processing method, and program

Abstract

To provide an information processing system, an information processing method, and a program for arranging elements in consideration of an aesthetic view.SOLUTION: In an information processing system 1000, an information processing device 100 divides a picked-up image acquired by an image pickup device 110 communicably connected through a network 150 into a plurality of cells, sets an element for each of the plurality of cells, and optimizes arrangement of the plurality of cells of the picked-up image in which the elements are set according to restriction conditions by using a model of a cell automaton method. The restriction conditions include conditions indicating restrictions between elements related to an aesthetic view and line-of-sight conditions related to the picked-up image.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 discloses an architectural space layout planning support device that can obtain a wide variety of architectural space layout variations, including clear boundaries and passageways and partition walls, in a short period of time. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2007-156891 A Summary of the Invention [Problem to be solved by the invention]

[0004] The technology of Patent Document 1 does not take into consideration the aesthetic layout of the elements. [Means for solving the problem]

[0005] According to one aspect of the present invention, there is provided an information processing system. The information processing system acquires a captured image. The captured image is divided into a plurality of cells. An element is set for each of the plurality of cells. The arrangement of the plurality of cells of the captured image in which the elements have been set is optimized using a cellular automaton model according to constraint conditions. The constraint conditions include conditions indicating constraints between elements regarding aesthetic scenery and line-of-sight conditions regarding the captured image. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a diagram illustrating an example of a system configuration of an information processing system. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device. [Diagram 3] FIG. 3 is a diagram showing an example of an equation for creating a beautiful landscape. [Figure 4] FIG. 4 is a conceptual diagram showing an example of a method for creating an aesthetic space. [Diagram 5] FIG. 5 is a sequence diagram illustrating an example of information processing in the information processing device. [Figure 6] FIG. 6 is a diagram showing an example of a screen including a captured image. [Figure 7] FIG. 7 is a diagram showing an example in which elements are set in each of a plurality of cells that make up an image. [Figure 8] FIG. 8 is a diagram showing an example of elements set in each cell. [Figure 9] FIG. 9 shows the constraints (part 1) implemented in the cellular automaton. [Figure 10] FIG. 10 shows the constraints (part 2) implemented in the cellular automaton. [Figure 11] FIG. 11 shows the third constraint implemented in the cellular automaton. [Figure 12] FIG. 12 shows the fourth constraint implemented in the cellular automaton. [Figure 13] FIG. 13 is a diagram showing an example of a calculation process for layout optimization. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the drawings. Various characteristic features shown in the following embodiments can be combined with each other.

[0008] In this specification, the term "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In addition, various information is handled in this embodiment, and communication and calculation can be performed on a circuit in the broad sense regardless of whether this information is represented by a high or low signal value as a binary bit collection consisting of 0 or 1, represented by a physical numerical value of a signal value, or represented by quantum superposition.

[0009] In addition, a circuit in the broad sense is a circuit realized by at least appropriately combining a circuit, circuitry, a processor, a memory, etc. In other words, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0010] In addition, the program for realizing the software appearing in the embodiments may be implemented in a form that is downloadable from a server, the program may be executed on a cloud computer, or the program may be stored in a non-volatile or volatile non-transitory storage medium and distributed.

[0011] <Embodiment 1> 1. System Configuration FIG. 1 is a diagram showing an example of a system configuration of an information processing system 1000. The information processing system 1000 includes an information processing device 100 and an imaging device 110 as a system configuration. The information processing device 100 and the imaging device 110 are communicably connected via a network 150. The imaging device 110 captures an image of a subject and transmits the captured image to the information processing device 100 or the like. In the first embodiment, the imaging device 110 captures an image of a garden and generates a captured image including the garden. Note that, in the example of FIG. 1, a security camera is shown as an example of the imaging device 110, but the imaging device 110 is not limited to a security camera. The imaging device 110 may be a camera mounted on an unmanned aerial vehicle (e.g., a drone) or the like. The imaging device 110 may be any device that can capture an image of a garden. In addition, a plurality of imaging devices may be included in the information processing system 1000. The information processing device 100 acquires a captured image and executes the process of the first embodiment described below on the captured image.

[0012] Here, the information processing system described in the claims may be composed of multiple devices or may be composed of one device. When the information processing system described in the claims is composed of one device, an example of the device is the information processing device 100. When the information processing system described in the claims is composed of multiple devices, an example of the multiple devices is the information processing device 100 and the imaging device 110, or the information processing device 100 and a server device that can communicate with the information processing device 100 and executes part of the processing of the information processing device 100 described later, etc.

[0013] 2. Hardware Configuration 2 is a diagram showing an example of a hardware configuration of the information processing device 100. The information processing device 100 includes, as the hardware configuration, a control unit 210, a storage unit 220, an input unit 230, an output unit 240, and a communication unit 250.

[0014] The control unit 210 is a CPU (Central Processing Unit) or the like, and controls the information processing device 100 as a whole.

[0015] The storage unit 220 is any one of a ROM (Read Only Memory), a RAM (Random Access Memory), and an SSD (Solid State Drive), or any combination thereof, and stores a program and data (e.g., captured image, set line of sight information, setting data between image information and elements, optimization results, etc.) used when the control unit 210 executes processing based on the program. The control unit 210 executes processing based on the program stored in the storage unit 220, thereby realizing the functions of the information processing device 100 and the processing of the information processing device 100 in the sequence diagram shown in FIG. 5 described later. The storage unit 220 is an example of a storage medium. Note that, in this embodiment, the data used when the control unit 210 executes processing based on the program is described as being stored in the storage unit 220, but the data may be stored in a storage unit of another device that can communicate with the information processing device 100. In other words, the data may be stored in a storage unit of any device as long as the control unit 210 can refer to the data.

[0016] The input unit 230 is a mouse, a keyboard, or the like, and inputs information based on user operations. The output unit 240 is a display or the like, and outputs the information input by the input unit 230, the results of processing by the control unit 210, and the like. The communication unit 250 connects the information processing device 100 to a network 150 or the like, and controls communication with other devices.

[0017] 3. Information Processing The information processing according to this embodiment will be described below. (Processing Overview) This paper discloses a technology that uniquely determines the target space by establishing rules between components. Rules are an example of constraints. The specification describes a method for designing spaces with excellent aesthetic scenery. Japanese gardens are a concentrated expression of the aesthetic sense of the Japanese people, and the idea was to utilize knowledge of Japanese gardens in spatial design. To this end, documents written for the construction and management of gardens in the Edo period ("Sakuteiki" and "Tsukiyama Teizoden") were deciphered, and rules that appeal to the aesthetic sensibilities of the Japanese were established. The specific steps are as follows: (1) Deciphering the cursive script of "Sakuteiki" and "Tsukiyama Teizouden" and translating them into modern Japanese (2) Extraction of items related to the spatial arrangement of Japanese garden components (3) Narrowing down the applicable items and broadening the concepts (4) Extraction of prohibitions learned from Japanese gardens

[0018] From the above books, items related to spatial arrangement were extracted. These items necessary for creating a beautiful landscape could be associated with three aesthetic sensibilities held by the Japanese. In other words, these are the three values ​​of "imitating nature (hou)," "deliberately breaking with nature (kuzure)," and "creating space (ma)." Therefore, the equation shown in Figure 3 is the secret to creating a Japanese garden, and it was thought that this secret could be included among the prohibitions. Figure 3 shows an example of an equation for creating a beautiful landscape.

[0019] As a method of implementing the equation shown in Fig. 3 in real space, the control unit 210 abstracts each of the artificial and non-artificial elements (trees, stones, waterfalls, etc.) that make up a Japanese garden into blocks as shown in Fig. 4, and solves an optimization problem of allocating them to a target space divided into a two-dimensional grid. Fig. 4 is a conceptual diagram showing an example of a method for creating an aesthetic space. The control unit 210 limits the layout conditions to three elements (constraint conditions): "imitation," "space," and "break," and repeats calculations so that the entire system follows the constraint conditions, thereby making it possible to unequivocally derive an aesthetic space that is optimized based on the design theory of Japanese gardens.

[0020] (Processing details) FIG. 5 is a sequence diagram showing an example of information processing in the information processing device 100. As shown in FIG. In sequence SQ501, the control unit 210 receives a captured image including a garden transmitted from the imaging device 110. When the captured image is received, in sequence SQ502, the control unit 210 stores the received captured image in the storage unit 220 or the like. As another example, the captured image may be stored in a storage medium or the like and input to the information processing device 100. The storage medium is a medium for storing data such as captured images, such as a USB flash drive. The control unit 210 stores the captured image in the USB flash drive connected to the information processing device 100 in a selected storage area of ​​the storage unit 220 in response to a predetermined user operation such as drag and drop.

[0021] In sequence SQ503, the control unit 210 acquires a captured image from the storage unit 220, and displays a screen including the captured image on the output unit 240 or the like. FIG. 6 is a diagram showing an example of a screen including a captured image. A captured image 650 is included in the screen 600. As described above, the captured image in the first embodiment is a captured image of a garden. The user operates an arrow GUI component 660 displayed on the screen 600 to indicate from which direction the garden included in the captured image is to be viewed. The control unit 210 sets a line of sight for the garden included in the captured image based on a user operation via the screen. 6 is just an example, and any method may be used as long as it is possible to set the line of sight to the garden included in the captured image. For example, the control unit 210 may set the line of sight based on a description of the direction to look from, etc., described in a setting file or the like. The set line of sight is used as one of the constraints in the optimization process using a cellular automaton model, which will be described later.

[0022] In sequence SQ504, the control unit 210 divides the captured image into a plurality of cells. In sequence SQ505, the control unit 210 sets an element for each of the plurality of cells. More specifically, the control unit 210 sets an element for each of the plurality of cells based on the pixel value of each of the plurality of cells in the captured image. The pixel value is an example of image information. FIG. 7 is a diagram showing an example in which elements are set in each of a plurality of cells constituting an image. FIG. 8 is a diagram showing an example of elements set in each cell. The numerical values ​​attached to each cell shown in FIG. 7 indicate elements as shown in FIG. 8. The numerical value 0 indicates a tree (low) as an element. The tree (low) indicates a tree that is lower than the first threshold. The numerical value 1 indicates a tree as an element. The tree indicates a tree that is higher than the first threshold and lower than the second threshold. The numerical value 2 indicates a tree (tall) as an element. The tree (tall) indicates a tree that is higher than the second threshold. Note that the first threshold is a value smaller than the second threshold. The numerical value 3 indicates a waterfall as an element. The numerical value 4 indicates a pond as an element. The numerical value 5 indicates a road as an element. The numerical value 6 indicates a sidewalk as an element. The numerical value 7 indicates a stone as an element. The numerical value 8 indicates a house as an element. The number of elements indicates the number of cells included in the image. [%] indicates the percentage of each element included in the image.

[0023] In sequence SQ506, the control unit 210 optimally arranges the quantified two-dimensional matrix data using a numerical calculation model called a Cellular Automaton. The control unit 210 repeats the process of optimal arrangement a predetermined number of times (for example, 100 times). Here, the Cellular Automaton is a model in which a number of cells arranged in a lattice in space undergo state transitions over time depending on their relationships with adjacent cells. Based on the Cellular Automaton model, the control unit 210 changes the quantified garden elements using state transition rules as constraints, and attempts to optimally arrange the space. The constraints implemented in the Cellular Automaton are shown in Figures 9 to 12 below.

[0024] Figure 9 shows the constraint condition (part 1) implemented in the cellular automaton. For the sake of simplicity, the specification shows an example in which the optimal placement is limited to trees, and the line of sight is set to diagonally upward to the right. The constraint condition in Figure 9 is "place taller trees toward the rear."

[0025] Figure 10 shows the constraint (part 2) implemented in the cellular automaton. The constraint in Figure 10 is that "trees should be placed on the sides of the structure, but the front should not be blocked."

[0026] Figure 11 shows the constraint (part 3) implemented in the cellular automaton. The constraint in Figure 11 is that "the heights of trees should not be uniform in the depth and horizontal directions."

[0027] Figure 12 shows the fourth constraint implemented in the cellular automaton. The constraint in Figure 12 is that "the proportion of trees should be kept at the original ratio."

[0028] The constraint conditions shown in Figures 9 to 12 are examples of conditions showing constraints between elements related to aesthetic scenery. The constraint conditions shown in Figures 9 to 12 can also be said to be conditions showing constraints between elements related to aesthetic scenery that constitutes a garden.

[0029] FIG. 13 is a diagram showing an example of the calculation process of placement optimization. FIG. 13(a) is a diagram showing an example of the result of the first placement optimization. FIG. 13(b) is a diagram showing an example of the result of performing placement optimization 20 times. FIG. 13(c) is a diagram showing an example of the result of performing placement optimization 40 times. FIG. 13(d) is a diagram showing an example of the result of performing placement optimization 60 times. FIG. 13(e) is a diagram showing an example of the result of performing placement optimization 80 times. FIG. 13(f) is a diagram showing an example of the result of performing placement optimization 100 times. From Figure 13, we can see that tall trees were gradually placed in the lower left area, far from the line of sight, while avoiding height uniformity and occlusion by structures, and that the pattern started to converge after about 80 times.

[0030] The processing of sequence SQ506 is an example of a process that uses a cellular automaton model to optimize the arrangement of multiple cells in a captured image in which elements have been set according to constraint conditions. The constraint conditions include conditions that indicate constraints between elements regarding aesthetic scenery and line-of-sight conditions for the captured image. The constraint conditions that indicate constraints between elements regarding aesthetic scenery are the constraint conditions shown in Figures 9 to 12. The line-of-sight conditions for the captured image are the line-of-sight constraint conditions set in sequence SQ503. The line-of-sight constraint conditions are constraint conditions on from where and in what direction the garden is viewed.

[0031] In sequence SQ507, control unit 210 outputs the result of the processing of sequence SQ506. Control unit 210 may generate a screen shown in Fig. 13(g) showing the result of the processing of sequence SQ506 and display it on output unit 240, or may generate a file or the like including the result of the processing of sequence SQ506 and store it in storage unit 220 or the like. The processing of sequence SQ507 is an example of a process of outputting the result of the optimized cell placement.

[0032] As described above, according to the process of the first embodiment, it is possible to arrange elements taking into consideration the aesthetic landscape. Furthermore, according to the process of the first embodiment, it is possible to specify from which direction and how the aesthetic landscape of a garden or the like should be viewed by setting a line of sight. Furthermore, according to the process of the first embodiment, the user can easily set the line of sight.

[0033] (Variation 1) Modification 1 of embodiment 1 will be described. In modification 1, differences from embodiment 1 will be described. The same applies to the following modifications. In the first embodiment, an example of a synchronous cellular automaton in which cells in a space change simultaneously over time has been described. However, the control unit 210 of the first modification may be configured to arrange elements in consideration of aesthetic scenery by using an asynchronous cellular automaton in which cells change locally.

[0034] The first modification can also achieve the same effects as the first embodiment.

[0035] (Variation 2) A second modification of the first embodiment will be described. In the first embodiment, the explanation was given using the Moore neighborhood, which takes into account all eight cells adjacent to the center cell. However, the control unit 210 of the second modification may use the Neumann neighborhood, which takes into account only the four cells above, below, left, and right, to place elements while taking into account aesthetic scenery. By simplifying the model, the processing speed can be increased.

[0036] The second modification can also achieve the same effects as the first embodiment. Furthermore, it may be provided in the following aspects:

[0037] (1) An information processing system comprising: acquiring a captured image; dividing the captured image into a plurality of cells; setting elements for each of the plurality of cells; and optimizing the arrangement of the plurality of cells of the captured image in which the elements have been set according to constraint conditions using a cellular automaton model; the constraint conditions including conditions indicating constraints between elements regarding aesthetic landscape and line-of-sight conditions regarding the captured image.

[0038] (2) In the information processing system described above in (1), an element is set for each of the plurality of cells based on image information for each of the plurality of cells of the captured image.

[0039] (3) In the information processing system described in (1) or (2) above, a line of sight is set for the captured image, and the arrangement of multiple cells of the captured image in which elements are set in accordance with constraint conditions including conditions indicating constraints between elements related to the aesthetic landscape and the line of sight conditions that have been set is optimized using a cellular automaton model.

[0040] (4) In the information processing system described above in (3), a line of sight regarding the captured image is set based on a user operation via a screen.

[0041] (5) An information processing system according to any one of (1) to (4) above, which outputs a result of the optimized placement of the cells.

[0042] (6) The information processing system according to any one of (1) to (5) above, wherein the captured image is an image of a garden.

[0043] (7) In the information processing system described above in (6), the elements include man-made objects and non-man-made objects that make up the garden.

[0044] (8) In the information processing system described above in (7), the non-artificial objects include trees.

[0045] (9) An information processing system according to any one of (6) to (8) above, wherein the conditions indicating constraints between elements regarding the aesthetic landscape are conditions indicating constraints between elements regarding the aesthetic landscape that constitutes a garden.

[0046] (10) In the information processing system according to any one of (6) to (9) above, the line of sight condition is a condition as to from where and in what direction the garden is viewed.

[0047] (11) An information processing method executed by an information processing system, comprising: acquiring a captured image; dividing the captured image into a plurality of cells; setting elements for each of the plurality of cells; and optimizing the arrangement of the plurality of cells of the captured image in which the elements have been set according to constraint conditions using a cellular automaton model, wherein the constraint conditions include conditions indicating constraints between elements regarding aesthetic landscape and line-of-sight conditions regarding the captured image.

[0048] (12) A program for causing a computer to function as the information processing system according to any one of (1) to (10) above. Of course, this is not the case.

[0049] For example, the above-described embodiments and modifications may be implemented in any combination.

[0050] Finally, although various embodiments of the present invention have been described, these are presented as examples and are not intended to limit the scope of the invention. The new embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. The embodiments and their modifications are within the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0051] 100: Information processing device 110: Imaging device 150: Network 210: Control section 220: Storage section 230: Input section 240: Output section 250: Communications Department 600: Screen 650: Captured image 660: GUI parts 1000: Information processing systems

Claims

1. An information processing system, Acquire a captured image, Dividing the captured image into a plurality of cells; Setting an element for each of the plurality of cells; optimizing the arrangement of the plurality of cells of the captured image, the elements of which are set in accordance with the constraint conditions, using a model of a cellular automaton method; The constraint conditions include a condition indicating a constraint between elements related to an aesthetic landscape and a line of sight condition related to the captured image. Information processing system.

2. 2. The information processing system according to claim 1, setting an element for each of the plurality of cells based on image information of each of the plurality of cells of the captured image; Information processing system.

3. 2. The information processing system according to claim 1, A line of sight is set with respect to the captured image, optimizing, using a cellular automaton model, an arrangement of a plurality of cells of the captured image in which elements are set in accordance with constraint conditions including a condition indicating a constraint between elements related to the aesthetic landscape and the set line of sight condition; Information processing system.

4. 4. The information processing system according to claim 3, setting a line of sight with respect to the captured image based on a user operation via a screen; Information processing system.

5. 2. The information processing system according to claim 1, outputting the optimized placement of the cells; Information processing system.

6. 2. The information processing system according to claim 1, The captured image is a captured image of a garden. Information processing system.

7. 7. The information processing system according to claim 6, The elements include artificial and non-artificial objects that make up the garden. Information processing system.

8. 8. The information processing system according to claim 7, The non-artificial objects include trees. Information processing system.

9. 7. The information processing system according to claim 6, The condition indicating the constraints between the elements regarding the aesthetic landscape is a condition indicating the constraints between the elements regarding the aesthetic landscape that constitutes a garden. Information processing system.

10. 7. The information processing system according to claim 6, The line of sight conditions are conditions on from where and in what direction the garden is viewed, Information processing system.

11. An information processing method executed by an information processing system, comprising: Acquire a captured image, Dividing the captured image into a plurality of cells; Setting an element for each of the plurality of cells; optimizing the arrangement of the plurality of cells of the captured image, the elements of which are set in accordance with the constraint conditions, using a model of a cellular automaton method; The constraint conditions include a condition indicating a constraint between elements related to an aesthetic landscape and a line of sight condition related to the captured image. Information processing methods.

12. A program, Computer, A program for causing the information processing system according to any one of claims 1 to 10 to function as such.

Images