Map information generation apparatus, map information generation method, and program

Abstract

To generate map information including animal appearance forecast, for each characteristic of an imaging area, based on a recognized plant type.SOLUTION: An information processing apparatus 1 serving as a map information generation apparatus includes: an image acquisition unit 11 which acquires an image including trees; a plant image recognition unit 21 which recognizes the type of a plant captured in the image acquired by the image acquisition unit 11 and maturity of fruit of the plant; an animal type specifying unit 22 which specifies the type of an animal which is highly likely to appear around the captured plant, based on results recognized by the plant image recognition unit 21; a recognition position estimation unit 26 which estimates a position of the plant recognized by the plant image recognition unit 21; and a map information generation unit 27 which generates map information by associating the position of the plant estimated by the recognition position estimation unit 26 with the type of the animal specified by the animal type specifying unit 22.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a map information generating device, a map information generating method, and a program. [Background technology]

[0002] Technologies are being developed to prevent damage to crops caused by pests.

[0003] For example, there is a technology that supports pest control by notifying not only the type of pest that harms agricultural crops but also the degree of danger that indicates the need for measures such as placing traps against the pests (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-143215 Summary of the Invention [Problem to be solved by the invention]

[0005] The technology of Patent Document 1 mentioned above uses multiple detection sensors placed near the farmland where the crops are grown to estimate the growth status of the crops and the type of pests roaming the vicinity, and then uses the estimated growth status and type of pests to determine the risk of damage to the crops by pests near the location where the detection sensor devices are placed.

[0006] However, the technology described in Patent Document 1 above estimates the type of vermin wandering near the detection sensor and supports vermin control measures. In contrast, it is known that for monitoring situations over a wider area, images are captured and image recognition is performed. However, when there are many types of animals to deal with and it is not known when and where an animal will appear within the imaging range, it is not easy to perform accurate detection. Therefore, it is necessary to predict where and which animal will appear.

[0007] The present invention has been made in consideration of such problems, and provides a map information generation device, a map information generation method, and a program that enable the generation of map information including animal appearance predictions for each characteristic of a shooting area based on the recognized plant species. [Means for solving the problem]

[0008] One aspect of the map information generation device of the present invention is characterized by having an image acquisition unit that acquires images including trees, a plant image recognition unit that recognizes the type of plant captured in the image acquired by the image acquisition unit and the maturity of the plant's fruit, an animal species identification unit that identifies the type of animal that is likely to appear near the captured plant based on the recognition result of the plant image recognition unit, a recognized position estimation unit that estimates the position of the plant recognized by the plant image recognition unit, and a map information generation unit that generates map information that associates the position of the plant estimated by the recognized position estimation unit with the type of animal identified by the animal species identification unit.

[0009] One aspect of the map information generation method of the present invention is characterized by including an image acquisition step for acquiring an image including trees, a plant image recognition step for recognizing the type of plant captured in the image acquired by processing in the image acquisition step and the maturity of the plant's fruit, an animal species identification step for identifying the type of animal that is likely to appear near the captured plant based on the recognition result obtained by processing in the plant image recognition step, a recognized position estimation step for estimating the position of the plant recognized by processing in the plant image recognition step, and a map information generation step for generating map information that associates the position of the plant estimated by processing in the recognized position estimation step with the type of animal identified by processing in the animal species identification step.

[0010] One aspect of the program of the present invention causes a map information generation device to execute processing characterized by including an image acquisition step for acquiring an image including trees, a plant image recognition step for recognizing the type of plant captured in the image acquired by processing in the image acquisition step and the maturity of the plant's fruit, an animal species identification step for identifying the type of animal that is likely to appear near the captured plant based on the recognition result by processing in the plant image recognition step, a recognized position estimation step for estimating the position of the plant recognized by processing in the plant image recognition step, and a map information generation step for generating map information that associates the position of the plant estimated by processing in the recognized position estimation step with the type of animal identified by processing in the animal species identification step. [Effects of the Invention]

[0011] According to the present invention, map information can be generated that indicates the predicted appearance of animals for each characteristic of the photography area based on the recognized plant species. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a functional block diagram of an information processing device 1. [Figure 2] FIG. 2 is a diagram for explaining an example of a recognition result of a plant. [Figure 3] FIG. 3 is a diagram for explaining an example of setting an image area by the recognition area setting unit 24. In FIG. [Figure 4] FIG. 4 is a diagram for explaining an example of the generated output map information. [Figure 5] FIG. 5 is a flowchart illustrating the image recognition process. [Figure 6] FIG. 6 is a flowchart illustrating the map information generation process. [Figure 7] FIG. 7 is a flowchart illustrating the output process. DETAILED DESCRIPTION OF THE INVENTION

[0013] [One embodiment] A map information generating device according to an embodiment of the present invention will be described below.

[0014] FIG. 1 is a block diagram showing the functional configuration of an information processing device 1 as an embodiment of a map information generating device of the present invention.

[0015] The information processing device 1 includes an image acquisition unit 11, an operation input acquisition unit 12, a control unit 13, and a storage unit 14. The information processing device 1 has an image processing function for processing acquired images and recognizing the ripening state of plant fruits and the appearance of animals, and a function for generating map information using information obtained by the image processing function. Note that the information processing device 1 may be configured by multiple devices rather than a single device, and may also have other functions.

[0016] The image acquisition unit 11 acquires image data captured by an imaging device (not shown) and supplies it to the control unit 13. The imaging device is installed so that it can capture an image including trees at a predetermined location in a forest area from a predetermined angle, and so that it can capture an image of a size that allows at least an approximate number of fruits on the trees and their maturity to be estimated. The image acquisition unit 11 acquires image data from multiple imaging devices. The image acquisition unit 11 supplies the acquired image data to the control unit 13 together with position-related information that allows the recognition position estimation unit 26 (described later) to estimate the positions of plants and animals included in the image, such as the ID of the imaging device, latitude and longitude information of the location where the imaging device is installed, and imaging direction information.

[0017] The operation input acquisition unit 12 is configured by, for example, an input device such as a keyboard, a button, or a touch panel, or an interface that receives information corresponding to a user operation input in another device, acquires the user's operation input, and supplies it to the control unit 13. Note that the operation input acquisition unit 12 is not necessarily an essential component, and may be omitted if input of a user's command, which will be described later, is not required.

[0018] The control unit 13 is composed of a CPU (Central Processing Unit), a storage unit (e.g., ROM (Read Only Memory), RAM (Random Access Memory), non-volatile memory), and other elements including hardware. The control unit 13 controls the entire information processing device 1 by executing a control application program (not shown) stored in the storage unit 14.

[0019] The control unit 13 has functions as a plant image recognition unit 21 to an output control unit 29, which will be described later. That is, the control unit 13 has a function to process the image acquired by the image acquisition unit 11, detect information about the plant species and fruit maturity, and accurately detect animals from the image based on the detection results. Furthermore, the control unit 13 has a function to generate map information that reflects the detection results of plants and animals.

[0020] The storage unit 14 is configured to include, for example, a ROM, a RAM, a non-volatile memory, etc. The storage unit 14 stores information necessary for the control unit 13 to execute various functions described below. The storage unit 14 includes the functions of a plant dictionary storage unit 30 to a map information storage unit 34 described below.

[0021] (Control unit 13 and memory unit 14) The control unit 13 has functions as a plant image recognition unit 21, an animal species identification unit 22, a recognition dictionary selection unit 23, a recognition area setting unit 24, an animal image recognition unit 25, a recognition position estimation unit 26, a map information generation unit 27, an output information generation unit 28, and an output control unit 29. The storage unit 14 also has functions as a plant dictionary storage unit 30, a correspondence information storage unit 31, an animal dictionary storage unit 32, an identification result storage unit 33 (hereinafter referred to as the identification result storage unit 33), and a map information storage unit 34.

[0022] The plant dictionary storage unit 30 stores dictionary data necessary for recognizing the type of plant captured in the image data and the maturity of its fruit. Specifically, the plant dictionary storage unit 30 stores dictionary data consisting of information such as the tree shape of various plants, the color, shape, and size of leaves, the color, shape, and size of flowers, the color, shape, and size of fruits, and the pattern and color of trunks.

[0023] The plant image recognition unit 21 performs image analysis on the image data acquired by the image acquisition unit 11 based on the plant dictionary data stored in the plant dictionary storage unit 30, recognizes the species of the captured plant and the maturity of the fruit of that plant, estimates the number of ripe fruits, and supplies the supplied image data, position-related information, and recognition results to the animal species identification unit 22 and the identification result storage unit 33. The plant image recognition unit 21 may be configured to recognize the maturity of the fruit of each plant in multiple stages based on its color and size. The plant image recognition unit 21 may also be configured to estimate the number of fruits for each stage of fruit maturity for each plant. When estimating the number of ripe fruits, the plant image recognition unit 21 may estimate a rough number, or may determine whether or not there is a good harvest and use that as the estimation result.

[0024] An example of a plant recognition result will be described with reference to FIG.

[0025] Based on the plant dictionary data stored in the plant dictionary storage unit 30, the plant image recognition unit 21 recognizes from the supplied image data that XX fruits of plant A have ripened in areas 41 and 42, and that XX fruits of plant B have ripened in area 43. The plant image recognition unit 21 supplies the names of the plants recognized as having ripened fruits, the estimated number of ripened fruits, and information related to the captured area, together with the captured image data, to the animal species identification unit 22 and the identification result storage unit 33.

[0026] 1, the correspondence information storage unit 31 stores correspondence information indicating the type of animal that is likely to appear nearby, corresponding to the degree of ripeness and number of fruits for each plant. For example, the correspondence information stored in the correspondence information storage unit 31 associates the common nutcracker with Japanese stone pine and Japanese white pine, and associates the Japanese squirrel, Japanese field mouse, Asian black bear, etc. with the Japanese walnut.

[0027] For example, different correspondence information may be prepared depending on the region where the image data acquired by the image acquisition unit 11 is captured. Specifically, larch, Abies sachalinensis, and Japanese kokuwa are distributed mainly in Hokkaido, while Eustoma japonicum and wild cherry trees are distributed mainly south of Honshu, while Myrica rubra and Zelkova are distributed mainly south and west of the Kanto region, and camphor trees are distributed in central and southern Honshu, Shikoku, and Kyushu. Therefore, information on these plants may be available for the corresponding region. Even for plants found throughout Japan, such as oak, Japanese oak, castanopsis cuspidata, and Quercus crispula, the types of animals likely to appear when the fruit ripens can be associated with Hokkaido squirrels, chipmunks, mountain jays, Hokkaido speckled mice, and brown bears in Hokkaido, and with Honshu squirrels, Eurasian jays, Japanese field mice, and Asiatic black bears in Honshu. Furthermore, animals not found in other regions, such as Yakushima and the Amami Islands, may be used only when analyzing image data from that region. By doing so, accurate recognition becomes possible without increasing the processing load.

[0028] The animal species identification unit 22 identifies an animal that is likely to appear near the plant captured in the supplied image data, in other words, the species of animal that is likely to be captured in the supplied image data, based on the recognition results of the plant species and fruit maturity level supplied from the plant image recognition unit 21 and the correspondence information stored in the correspondence information storage unit 31. If the recognition results supplied from the plant image recognition unit 21 indicate only the plant species and that the fruit is not yet ripe, the animal species identification unit 22 determines that the animal species cannot be identified as the animal species identification result. The animal species identification unit 22 may also identify the species of animal that is likely to be captured in the image data by further considering the location and time when the supplied image data was captured. The animal species identification unit 22 supplies the animal species identification result, the recognition results of the plant species and fruit maturity level, and the supplied image data to the recognition dictionary selection unit 23, and supplies the animal species identification result to the identification result storage unit 33 for storage.

[0029] The animal dictionary storage unit 32 stores dictionary data for detecting various animals that is necessary for the animal image recognition unit 25 (described later) to analyze the supplied image data. The animal dictionary storage unit 32 also stores general-purpose dictionary data for detecting a wide range of animals, in addition to dictionary data for each specific animal for detecting specific animals with higher accuracy.

[0030] Based on the animal species identification result supplied from the animal species identification unit 22, the recognition dictionary selection unit 23 selects dictionary data for each specific animal that is used by the animal image recognition unit 25 (described later) to recognize the identified animal from the captured image data, reads it from the animal dictionary storage unit 32, and supplies it together with the captured image data to the animal image recognition unit 25. If the fruit of a specific plant is not recognized, the recognition dictionary selection unit 23 selects general-purpose dictionary data for detecting various animals.

[0031] When the recognition area setting unit 24 receives a user operation input from the operation input acquisition unit 12 instructing it to set an image area in which animal recognition processing is to be performed based on the plant recognition result, the recognition area setting unit 24 sets an image area in which animal recognition processing is to be performed based on the animal species identification result supplied from the animal species identification unit 22, and supplies the setting result to the animal image recognition unit 25.

[0032] An example of setting an image area by the recognition area setting unit 24 will be described with reference to FIG.

[0033] As explained using FIG. 2 , the plant image recognition unit 21 processes the supplied image data to recognize that XX fruits of plant A have ripened in regions 41 and 42 and that XX fruits of plant B have ripened in region 43. The animal species identification unit 22 obtains this recognition result and processes it to identify bird C and bird D as the animal species identified based on the ripening of the fruit of plant A, and identify mammal E, a mammal that primarily moves on the ground. Since the animal species identified in association with regions 41 and 42 are birds, the recognition area setting unit 24 sets region 51, which includes the sky above regions 41 and 42, as the recognition area for bird C and bird D, as shown in FIG. 3 . Since the animal species identified in association with region 43 is mammal E, a mammal that primarily moves on the ground, the recognition area setting unit 24 sets region 52, which includes the lower part of region 43, i.e., the ground, as the recognition area for bird C and bird D, as shown in FIG. 3 .

[0034] Returning to FIG. 1 , the animal image recognition unit 25 recognizes animals captured in the supplied image data using dictionary data selected by the recognition dictionary selection unit 23 within the recognition area if a recognition area is set by the recognition area setting unit 24. Otherwise, the recognition area is used for the entire area of ​​the supplied image data. If the fruit of a specific plant is not recognized, the animal image recognition unit 25 performs animal recognition processing using general-purpose dictionary data for detecting various animals. If the plant type and fruit ripeness are recognized, the animal image recognition unit 25 performs image recognition using dictionary data for each animal identified based on the recognition results, thereby enabling more accurate animal recognition with less processing. Furthermore, if a recognition area is previously set for each animal type identified based on the recognized plant type, the animal image recognition unit 25 can reduce the processing load for animal recognition without sacrificing accuracy. The animal image recognition unit 25 supplies the image recognition results to the identification result storage unit 33 for storage.

[0035] The identification result memory unit 33 stores the captured image data, position-related information corresponding to the image data, information on the type of plant recognized from the image data and the number of mature fruits thereof, the animal type identification result obtained corresponding to the plant, and the animal recognition result.

[0036] When the operation input acquisition unit 12 commands the generation of map information, the recognition position estimation unit 26 reads out the image data stored in the identification result memory unit 33, the corresponding position-related information, information on the type of plant and the number of mature fruits thereof, the identification result of the type of animal, and the recognition result of the animal, estimates the position of each type of plant recognized as having mature fruit and the imaging position of the recognized animal, and supplies the image data, the identification result of the type of animal, and the recognition results and position estimation results of the plants and animals to the map information generation unit 27.

[0037] The map information generation unit 27 uses the information supplied from the recognized position estimation unit 26 to generate map information in which the recognition results are associated on a map. The map information generation unit 27 first associates the location of each plant species recognized as having ripe fruit with the recognized image on the map, estimates a boundary line indicating the area in which the fruit is recognized as ripe, and associates it on the map. The map information generation unit 27 then associates the species identification result of the animal corresponding to each plant with each area. The map information generation unit 27 then associates the image capture position of the recognized animal with the recognized image on the map. The recognized images of the plants and animals may be cropped to a more easily recognizable size for association. The cropping size is set according to the distance between the plant and the camera, the angle of view, and the size of the recognized animal. Furthermore, the map information generation unit 27 may generate map information in which information such as areas requiring warning of the appearance of wild animals and spots for observing and photographing birds and small animals is further associated based on the animal species identification result or the animal recognition result. The map information generating unit 27 supplies the generated map information to the map information storage unit 34 for storage.

[0038] The map information storage unit 34 stores the supplied map information. The map information storage unit 34 may also store data tables and the like that allow easy search for animals, plants, regions, and the like so that the output information generation unit 28 can read out the necessary information through processing described below. The information stored in the map information storage unit 34 is not only output through processing described below, but can also be made public to warn of the appearance of wild animals, provided to research institutions, or managed as history and used as annual data to predict, for example, whether the following year's fruit harvest will be good or bad.

[0039] The output information generation unit 28 uses information stored in the map information storage unit 34 to generate output information based on a user command supplied from the operation input acquisition unit 12. For example, when a command to output map information showing the status of plants and animals in a specific area is received from the user, the output information generation unit 28 generates information in which the recognition results of plants and animals are associated with the map information of the specified area, and the content of the information can be confirmed by the user, and supplies the information to the output control unit 29. An example of this information will be described later with reference to FIG. 4. Furthermore, the output information generation unit 28 receives an input of a specific animal name or plant name, generates data for displaying image data of the recognition results in a list or in association with a map, and supplies the data to the output control unit 29.

[0040] The generated output map information will be described with reference to FIG.

[0041] In this example, the area where the fruit is recognized as ripe is shown by a solid line for plant A and a dotted line for plant B. Which plant is recognized in which area and the result of identifying the animal species corresponding to that plant may be recognized by icons or text data, or may be displayed as a pop-up by clicking within the area. Also, in this example, an icon indicating the animal species is displayed at the recognized animal position. When the user clicks on either icon, a cropped recognized image is displayed as shown in pop-up 61.

[0042] Even in areas where no animals are recognized, areas where fruit is recognized as ripe and the results of identifying the corresponding animal species can be displayed on a map, so even in areas where no animals have yet been recognized, the user can be shown the possibility that the animal will appear in that area in the future.

[0043] In addition, if the plant image recognition unit 21 recognizes the maturity of the fruits of each plant in multiple stages based on their color and size, and the number of fruits at each maturity level is estimated, the output information generation unit 28 may display the range of each maturity level so that it can be recognized.

[0044] The output control unit 29 executes a process of outputting the output information generated by the output information generation unit 28 to a display unit (not shown) for display, or transmitting the output information to another information processing device (not shown) via a network.

[0045] In this way, the information processing device 1 can accurately detect animals from the acquired image data. When performing image recognition of animals whose appearance and when within the imaging range are unknown, the information processing device 1 uses the recognition results of plants whose locations are fixed, so that the recognition process can be performed more accurately with a smaller amount of processing.

[0046] The information processing device 1 can also generate map information including predicted animal appearances for each characteristic of the photographed area based on the recognized plant species, and can present the status of local flora and fauna to the user in an easy-to-understand manner. This information can also be used to monitor the appearance of specific pests or for hobbies such as bird watching or observing small animals.

[0047] (Image recognition processing) Next, the image recognition process executed by the information processing device 1 will be described with reference to the flowchart of FIG.

[0048] In step S1 , the image acquisition unit 11 acquires image data and position-related information captured by an imaging device (not shown), and supplies them to the control unit 13 .

[0049] In step S2, the plant image recognition unit 21 of the control unit 13 performs image analysis of the image data acquired by the image acquisition unit 11 based on the plant dictionary data stored in the plant dictionary memory unit 30, and as explained using Figure 2, recognizes the type of plant being photographed and the maturity of the fruit of that plant, estimates the number of mature fruits, and supplies the supplied image data, position-related information, and recognition results to the animal species identification unit 22 and the identification result memory unit 33.

[0050] In step S3, the animal species identification unit 22 identifies an animal that is likely to appear near the plant captured in the supplied image data, in other words, the species of animal that is likely to be captured in the supplied image data, based on the recognition results of the plant species and fruit maturity level supplied from the plant image recognition unit 21 and the correspondence information stored in the correspondence information storage unit 31. The animal species identification unit 22 supplies the animal species identification results, the plant species and fruit maturity level recognition results, and the supplied image data to the recognition dictionary selection unit 23, and also supplies the animal species identification results to the identification result storage unit 33.

[0051] In step S4, based on the animal species identification result supplied from the animal species identification unit 22, the recognition dictionary selection unit 23 selects dictionary data for each specific animal that the animal image recognition unit 25 uses to recognize the identified animal from the captured image data, reads it from the animal dictionary storage unit 32, and supplies it together with the captured image data to the animal image recognition unit 25. If the fruit of a specific plant is not recognized, the recognition dictionary selection unit 23 selects general-purpose dictionary data for detecting various animals.

[0052] In step S5, the recognition area setting unit 24 determines whether or not the setting is to set an area for performing animal recognition.

[0053] If it is determined in step S5 that the setting is not to perform animal recognition by setting an area, then in step S6, the animal image recognition unit 25 performs animal recognition processing on the entire image using the selected dictionary.

[0054] If it is determined in step S5 that the setting is to set an area for performing animal recognition, then in step S7, the recognition area setting unit 24 sets a recognition area in which animal recognition processing is performed, based on the type identified by the animal type identification unit 22, as described using Figure 3, for example, and supplies the setting result to the animal image recognition unit 25.

[0055] In step S8, the animal image recognition unit 25 executes an animal recognition process for each recognition area using the selected dictionary.

[0056] After the processing of step S6 or step S8 is completed, in step S9, the animal image recognition unit 25 supplies and stores the image recognition result to the identification result storage unit 33. The identification result storage unit 33 stores the captured image data, positional relation information corresponding to the image data, and information on the type of plant recognized from the image data and the number of mature fruits thereof, together with the supplied animal recognition result, and then the processing ends.

[0057] This processing enables accurate recognition of animals from image data. If the fruit of a specific plant is not recognized, the animal image recognition unit 25 performs animal recognition processing using general-purpose dictionary data for detecting various animals. If the type of plant and the ripeness of the fruit are recognized, the animal image recognition unit 25 performs image recognition using dictionary data for each animal identified based on the recognition results, thereby enabling more accurate recognition of animals with less processing. Furthermore, if a recognition area is set in advance based on the animal type identified based on the recognized plant type, the animal image recognition unit 25 can reduce the amount of processing required for animal recognition without sacrificing accuracy.

[0058] (Map information generation process) Next, the map information generation process executed by the information processing device 1 will be described with reference to the flowchart of FIG.

[0059] In step S21, the recognition position estimation unit 26 acquires the information necessary for estimating position information and generating map information stored in the identification result storage unit 33, namely, image data, corresponding position-related information, information on the type of plant and the number of mature fruits thereof, the animal type identification result, and the animal recognition result.

[0060] In step S22, the recognition position estimation unit 26 estimates the position of each type of plant whose fruit has been recognized as ripe and the imaging position of the recognized animal, and supplies the image data, the animal type identification results, and the recognition results and position estimation results of the plants and animals to the map information generation unit 27.

[0061] In step S23, the map information generation unit 27 associates the location of each plant species recognized as having ripe fruit and the recognized image with map information, estimates a boundary line indicating the area recognized as having ripe fruit, and associates it with the map information.The map information generation unit 27 then associates, for each area, the result of identifying the species of animal corresponding to the plant.

[0062] In step S24, the map information generating unit 27 associates the recognized image capturing position of the animal and the recognized image with the map information.

[0063] In step S25, the map information generation unit 27 associates information based on the results of identifying the animal species, such as areas where warnings about the appearance of wild animals are required, and spots for observing and photographing birds and small animals, with the map information as needed. The map information generation unit 27 supplies the generated map information to the map information storage unit 34. The map information storage unit 34 stores the supplied map information, and the process ends.

[0064] This processing makes it possible to generate map information that includes animal appearance predictions for each characteristic of the photographed area based on the recognized plant species. The information stored in the map information storage unit 34 can be used for a wide range of purposes, such as making it public to warn of the appearance of wild animals, providing it to research institutions, or managing it as historical data and using it to predict whether the following year's fruit harvest will be good or bad.

[0065] (Output processing) Next, the output process executed by the information processing device 1 will be described with reference to the flowchart of FIG.

[0066] In step S31, the output information generating unit 28 determines, based on a user command supplied from the operation input acquiring unit 12, whether or not an output of map information has been commanded.

[0067] If it is determined in step S31 that a command to output map information has been issued, in step S32 the output information generation unit 28 reads out necessary information from the information stored in the map information storage unit 34 based on the user's command, generates output data such as that described with reference to Fig. 4, and supplies it to the output control unit 29. The output control unit 29 executes processing to output the map information generated by the output information generation unit 28 to a display unit (not shown) for display, or to transmit the map information to another information processing device (not shown) via a network.

[0068] In step S33, the output information generation unit 28 determines whether or not a command has been issued to output a recognition image of a plant or animal associated with the map based on a user command, such as a click operation at a specific position on the display screen, supplied from the operation input acquisition unit 12.

[0069] If it is determined in step S33 that a command has been issued to output a recognition image of a plant or animal associated with the map, then in step S34 the output information generation unit 28 generates a recognition image of the plant or animal, for example, as shown in pop-up 61 in Fig. 4, and supplies the image to the output control unit 29. The output control unit 29 executes processing to output the recognition image supplied by the output information generation unit 28 to a display unit (not shown) for display, or to transmit the image to another information processing device (not shown) via a network.

[0070] In step S31, if it is determined that the output of map information has not been instructed, if it is determined in step S33 that the output of a recognition image of a plant or animal associated with the map has not been instructed, or after the processing of step S34 is completed, in step S35, the output information generation unit 28 determines whether or not the output of a recognition image has been instructed based on a user instruction supplied from the operation input acquisition unit 12.

[0071] If it is determined in step S35 that a command to output a recognized image has been issued, then in step S36 the output information generation unit 28 generates data for displaying image data of the recognition results of the specific animal or plant specified by the user in a list or in association with a map, and supplies the data to the output control unit 29. The output control unit 29 executes processing to output and display the recognized image supplied by the output information generation unit 28 on a display unit (not shown), or to transmit the image data to another information processing device 1 (not shown) via a network. If it is determined in step S35 that a command to output a recognized image has not been issued, or after the processing of step S36 is completed, the processing ends.

[0072] By performing such processing, the information stored in the map information storage unit 34 can be output and presented in a format desired by the user.

[0073] [Supplementary explanation of the embodiment] The above-described embodiments each show a preferred specific example of the present invention. The numerical values, components, arrangement and connection order of the components, processing order in the flowcharts, etc. shown in the embodiments are merely examples and are not intended to limit the present invention. Furthermore, the drawings are not necessarily strict illustrations.

[0074] The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware, or into, for example, a general-purpose personal computer that can execute various functions by installing various programs.

[0075] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0076] [Note] The contents of the above-described embodiments can be understood, for example, as follows.

[0077] (1) Generation of map information The information processing device 1 as the map information generating device described above includes: An image acquisition unit 11 that acquires an image including trees (step S1); a plant image recognition unit 21 that recognizes the type of plant captured in the image acquired by the image acquisition unit 11 and the maturity of the fruit of the plant (step S2); an animal species identification unit 22 that identifies the species of animals that are likely to appear near the imaged plant based on the recognition result of the plant image recognition unit 21 (step S3); a recognized position estimation unit 26 that estimates the position of the plant recognized by the plant image recognition unit 21 (step S22); a map information generating unit 27 (step S23) that generates map information that associates the positions of the plants estimated by the recognized position estimating unit 26 with the types of animals identified by the animal type identifying unit 22; It has.

[0078] With this configuration, it is possible to generate map information including animal appearance predictions for each characteristic of the photographing area based on the recognized plant species.

[0079] (2) Mapping animal recognition results to map information Also, a recognition dictionary selection unit 23 that selects dictionary data for recognizing the animal identified by the animal type identification unit 22 (step S4); an animal image recognition unit 25 that recognizes animals captured in images acquired by the image acquisition unit 11 using the dictionary data selected by the recognition dictionary selection unit 23 (step S5); and The recognition position estimation unit 26 further estimates the position of the animal recognized by the animal image recognition unit 25 (step S22). The map information generating unit 27 generates map information that further associates the animal positions estimated by the recognized position estimating unit 26 (step S24).

[0080] With this configuration, the recognized position of the animal can be presented to the user.

[0081] (3) Mapping animal recognition images to map information Furthermore, the map information generating unit 27 generates map information in which the recognized image of the animal recognized by the animal image recognizing unit 25 is further associated with the position of the animal estimated by the recognized position estimating unit 26 (step S24).

[0082] With this configuration, it is possible to present a recognition image of the animal to the user. [Explanation of symbols]

[0083] 1...information processing device, 11...image acquisition unit, 12...operation input acquisition unit, 13...control unit, 14...storage unit, 21...plant image recognition unit, 22...animal species identification unit, 23...recognition dictionary selection unit, 24...recognition area setting unit, 25...animal image recognition unit, 26...recognition position estimation unit, 27...map information generation unit, 28...output information generation unit, 29...output control unit, 30...plant dictionary storage unit, 31...correspondence information storage unit, 32...animal dictionary storage unit, 33...recognition and species identification result storage unit, 34...map information storage unit

Claims

1. an image acquisition unit that acquires an image including trees; a plant image recognition unit that recognizes the type of plant captured in the image acquired by the image acquisition unit and the maturity of the fruit of the plant; an animal type identification unit that identifies the type of animal that is likely to appear near the plant being imaged based on the recognition result of the plant image recognition unit; a recognized position estimation unit that estimates the position of the plant recognized by the plant image recognition unit; a map information generating unit that generates map information that associates the position of the plant estimated by the recognized position estimating unit with the type of animal identified by the animal type identifying unit; A map information generating device comprising:

2. 2. The map information generating device according to claim 1, a recognition dictionary selection unit that selects dictionary data for recognizing the animal identified by the animal type identification unit; an animal image recognition unit that recognizes the animal captured in the image acquired by the image acquisition unit using the dictionary data selected by the recognition dictionary selection unit; and the recognition position estimation unit further estimates the position of the animal recognized by the animal image recognition unit; The map information generating unit generates map information that further associates the positions of the animals estimated by the recognized position estimating unit. A map information generating device characterized by:

3. 3. The map information generating device according to claim 2, The map information generation unit generates map information in which the recognized image of the animal recognized by the animal image recognition unit is further associated with the position of the animal estimated by the recognized position estimation unit. A map information generating device characterized by:

4. A map information generating method executed by a map information generating device, an image acquisition step of acquiring an image including a tree; a plant image recognition step of recognizing the type of plant captured in the image acquired by the processing of the image acquisition step and the maturity level of the fruit of the plant; an animal type identification step of identifying the type of animal that is likely to appear near the plant being imaged based on the recognition result of the plant image recognition step; a recognized position estimating step of estimating a position of the plant recognized by the processing of the plant image recognizing step; a map information generating step of generating map information that associates the position of the plant estimated by the processing of the recognized position estimating step with the type of animal identified by the processing of the animal type identifying step; A map information generating method comprising:

5. A program executed by a map information generating device, an image acquisition step of acquiring an image including a tree; a plant image recognition step of recognizing the type of plant captured in the image acquired by the processing of the image acquisition step and the maturity level of the fruit of the plant; an animal type identification step of identifying the type of animal that is likely to appear near the plant being imaged based on the recognition result of the plant image recognition step; a recognized position estimating step of estimating a position of the plant recognized by the processing of the plant image recognizing step; a map information generating step of generating map information that associates the position of the plant estimated by the processing of the recognized position estimating step with the type of animal identified by the processing of the animal type identifying step; A program for causing a map information generating device to execute a process comprising:

Images