Information processing system, method for controlling information processing system, and program

Abstract

To provide a mechanism to more accurately determine the growth state of agricultural and livestock products.SOLUTION: An information processing system comprises: acquisition means that acquires a picked-up image of a marker; and control means that controls to output a grid deformed on the basis of the shape of the image of the marker acquired by the acquisition means, while superimposing the grid on the image acquired by the acquisition means.SELECTED DRAWING: Figure 9

Description

Technical Field

[0001] The present invention relates to an information processing system, a control method for an information processing system, and a program, and more particularly to a technique for measuring the dimensions of agricultural and livestock products.

Background Art

[0002] Conventionally, the growth status of crops has been managed using a camera. Patent Document 1 discloses a technique for measuring the dimensions of a plant stem using a camera.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Disclosure of the Invention

Problems to be Solved by the Invention

[0004] Patent Document 1 describes a technique for measuring dimensions based on marks serving as dimension standards.

[0005] However, since accurate measurement cannot be performed unless the mark is photographed from the front, there remains a problem that even when photographed from various angles other than the front, it is desired to accurately measure the growth state of agricultural and livestock products.

[0006] Therefore, an object of the present invention is to provide a mechanism for more accurately determining the growth state of agricultural and livestock products.

Means for Solving the Problems

[0007] An information processing system comprising: acquisition means for acquiring an image in which a marker is imaged; and control means for controlling to superimpose and output a grid deformed based on the shape of the image of the marker acquired by the acquisition means on the image acquired by the acquisition means.

Advantages of the Invention

[0008] According to the present invention, a mechanism for more accurately determining the growth state of agricultural and livestock products can be provided.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The embodiments described below are examples of specific implementations of the present invention and are one of the specific embodiments of the configuration described in the claims.

[0011] FIG. 1 is a system configuration diagram showing an example of the configuration of the information processing system 100 of the present invention.

[0012] The left side of FIG. 1 shows the control unit, and shows a system that is communicably connected to a network camera 200 and a mobile terminal 201, which are image acquisition units, via a network such as a router and the Internet.

[0013] The server device 101 acquires data managed by the database server 102 from the database server, analyzes the image data included in the acquired data, and specifies the growth status of the agricultural crop, for example, whether fruits or flowers are formed, the ratio of the leaf area occupying the image, etc.

[0014] The database server 102 is a server that manages as a database for reading and writing data such as those in FIGS. 5, 6, 7, and 10, and transmits the data managed by the database server 102 to the server device 101 according to a request from the server device 101.

[0015] The right side of FIG. 1 shows the image acquisition unit, and shows a situation where ridges 300 such as strawberries growing in the field are photographed using the network camera 200 or the mobile terminal 201 fixedly installed on the ceiling of the greenhouse or the pillars installed in the field.

[0016] Note that this image acquisition unit exists for each field such as a field or a greenhouse, and there may be two or more image acquisition units for one farmer or agricultural management organization. For an enterprise operating agriculture, it may have image acquisition units of a plurality of fields (image acquisition units) nationwide, and a system in which they and the control unit are communicably connected via a network.

[0017] In this embodiment, strawberries are assumed as the agricultural crop for explanation, but it is not limited to strawberries, and other agricultural crops such as peppers, tomatoes, cucumbers, cabbages, lettuces, radishes, apples, bananas, kiwis, rice, and wheat may also be used.

[0018] In this embodiment, although agricultural crops are used, the present invention is not limited to agricultural crops, and livestock products such as cattle, pigs, and chickens may also be used. In that case, the yield prediction will be for the number of heads shipped, the amount of eggs shipped, etc.

[0019] Then, the image data captured by the network camera 200 is transmitted to the database server 102 via the router and managed by the database server 102 in association with the preset ID that captured the image data (an example of the management means in the present invention for managing the image data captured by the imaging means for imaging agricultural crops with the set preset in association with the preset at the time when the image data was captured).

[0020] Also, the image data captured by the mobile terminal 201 is managed by the database server 102 in association with the preset ID linked to the marker (e.g., QR code (registered trademark)) shown in the image (an example of the management means in the present invention for managing the image data captured by the imaging means for imaging agricultural crops with the set preset in association with the preset at the time when the image data was captured).

[0021] In this embodiment, the server device 101 and the database server 102 are separate devices. However, as another embodiment, the server device 101 and the database server 102 may be a single device.

[0022] Also, the network camera 200 and the mobile terminal 201 may be a digital camera, a wearable camera, an infrared camera, a camera attached to a vehicle, an aircraft, a drone, etc. Hereinafter, with reference to FIG. 2, the hardware configuration applicable to each information processing device constituting the information processing system 100 shown in FIG. 1 will be described.

[0023] FIG. 2 is a block diagram showing the hardware configuration applicable to each information processing device constituting the information processing system 100 shown in FIG. 1.

[0024] In FIG. 2, 201 is a CPU that comprehensively controls each device and controller connected to the system bus 204. Also, in the ROM 202 or the external memory 211, there are stored a BIOS (Basic Input / Output System) which is a control program of the CPU 201, an operating system program (hereinafter referred to as OS), and various programs described later that are necessary to realize the functions executed by each PC, etc.

[0025] 203 is a RAM that functions as the main memory, work area, etc. of the CPU 201. The CPU 201 loads programs, etc. necessary for the execution of processing from the ROM 202 or the external memory 211 into the RAM 203 and realizes various operations by executing the loaded programs.

[0026] 205 is an input controller that controls inputs from a keyboard (KB) 210, a pointing device such as a mouse (not shown), etc.

[0027] 206 is a video controller that controls the display on a display such as the display 210.

[0028] 207 is a memory controller that controls access to an external storage device (hard disk (HD)) that stores various data, a flexible disk (FD), or an external memory 211 such as a CompactFlash (registered trademark) memory connected via an adapter to a PCMCIA card slot. The access to the external memory 211 such as a CompactFlash (registered trademark) memory connected via an adapter to a PCMCIA card slot is controlled.

[0029] 208 is a communication I / F controller that controls the reception of image data from an external PC 213 via a network (TCP / IP). 209 is an image I / F controller that controls the reception of image data from a camera 214 via an image transfer cable (USB, Ethernet, Camera Link, etc.).

[0030] The various programs described below for implementing the present invention are recorded in the RAM 203 and executed by the CPU 201.

[0031] Also, the image data used during the execution of the above program is stored in the ROM 202, the external memory 211, the external PC 213, and the camera 214 according to the application, and is stored in the RAM 203 via various controllers during program execution.

[0032] FIG. 3 is an example of a block diagram showing the software configuration of an embodiment of the present invention. The information processing system 100 includes the following functional units.

[0033] The management unit 301 is a functional unit that manages the growth characteristic amounts for each variety of agricultural and livestock products.

[0034] The reception unit 302 is a functional unit that receives the designation of the variety of agricultural and livestock products.

[0035] The acquisition unit 303 is a functional unit that acquires an image of the agricultural and livestock products being imaged.

[0036] The prediction unit 304 is a functional unit that predicts the yield of the agricultural and livestock products based on a determination as to whether the characteristic amounts of the image of the agricultural and livestock products approximate the growth characteristic amounts corresponding to the variety received by the reception unit 302.

[0037] The prediction unit 304 is a functional unit that predicts the yield of the agricultural and livestock products based on a determination as to whether the characteristic amounts of the image of the agricultural and livestock products approximate the growth characteristic amounts corresponding to the variety received by the reception unit 302 when the reliability of the prediction using the learning model is less than a predetermined value.

[0038] The acquisition unit 303 is a functional unit that acquires an image of the marker being imaged.

[0039] The control unit 305 is a functional unit that controls to superimpose and output on the image acquired by the acquisition unit 303 a grid deformed based on the shape of the image of the marker acquired by the acquisition unit 303.

[0040] The control unit 305 is a functional unit that controls to superimpose and output on the image acquired by the acquisition unit 303 a grid with at least one of inclination or deformation applied thereto according to at least one of the orientation, inclination or deformation of the image of the marker acquired by the acquisition unit 303.

[0041] The calculation unit 306 is a functional unit that calculates the size or length of an object in the image.

[0042] The calculation unit 306 is a functional unit that calculates the size or length of an object in the image based on the grid.

[0043] The management unit 301 is a functional unit that manages a plurality of images in which agricultural and livestock products at the first location are imaged.

[0044] The acquisition unit 303 is a functional unit that acquires an image in which agricultural and livestock products at the second location are imaged.

[0045] The display unit 307 is a functional unit that acquires an image similar to the image acquired by the acquisition unit 303 from the images managed by the management unit 301 and displays it side by side with the image acquired by the acquisition unit 303.

[0046] The display unit 307 is a functional unit that identifies and displays the differences between an image similar to the image acquired by the acquisition unit 303 and the image acquired by the acquisition unit 303.

[0047] The display unit 307 is a functional unit that displays images similar to the image acquired by the acquisition unit 303 in an order based on the similarity degree.

[0048] The display unit 307 is a functional unit that displays, in the order of similarity, images similar to the image acquired by the acquisition unit 303 based on the similarity criteria for which selection from the user has been received. This concludes the description of FIG. 3.

[0049] The flowchart of FIG. 4 will be described.

[0050] The following processing is assumed to be performed by the CPU 201 of any one of the server device 101, the network camera 200, or the mobile terminal 201 that constitutes the information processing system 100.

[0051] In S401, the mobile terminal 201 receives a selection of the variety of agricultural products to be photographed from the user, and transmits the identification information of the selected variety to the server device 101. Here, it is assumed that the selection of "Variety A" has been received.

[0052] That is, this step shows an example of a process for receiving a designation of the variety of agricultural and livestock products.

[0053] In S402, the server device 101 acquires an image of the agricultural products photographed by the network camera 200 or the mobile terminal 201. Here, it is assumed that photos such as 810 in FIG. 8, 910 and 930 in FIG. 9 have been acquired.

[0054] That is, this step shows an example of a process for acquiring an image of the agricultural and livestock products that has been imaged.

[0055] That is, this step shows an example of a process for acquiring an image of the agricultural and livestock products at the second location that has been imaged.

[0056] That is, this step shows an example of a process for acquiring an image of the marker that has been imaged.

[0057] In S403, the server device 101 calculates the feature amounts of the acquired images.

[0058] In S404, the server device 101 acquires, from the database server 102, images of other farms that are similar to the calculated image feature amounts.

[0059] The database server 102 stores and manages images of agricultural and livestock products at other locations together with environmental information such as the date and time of shooting, variety, temperature, and humidity.

[0060] That is, the database server 102 shows an example of a means for managing a plurality of images of agricultural and livestock products imaged at the first location.

[0061] Here, similar images refer to images in which at least one of the variety, number of growth days, growth environment, number of leaves of agricultural crops, leaf length, leaf size, leaf density, leaf pattern, leaf shape, leaf color, stem length, stem thickness, stem density, stem pattern, stem shape, stem color, number of flowers, flower length, flower size, flower density, flower pattern, flower shape, flower color, flowering rate of flowers, number of fruits, fruit length, fruit size, fruit density, fruit pattern, fruit shape, fruit color, number of seeds, seed length, seed size, seed density, seed pattern, seed shape, and seed color of the agricultural and livestock products included in the image; the length, size, body temperature, pulse rate, heart rate, stress level of all or part of the body of the livestock product; and the soil color, size of soil particles, shape of soil particles, water color, water temperature, water components, air temperature, humidity, composition of the image, hue, contrast, brightness, and date and time of shooting are the same or similar.

[0062] In S405, the server device 101 transmits the image acquired in S404 to the mobile terminal 201, and the mobile terminal 201 displays the image acquired in S402 and the image acquired in S404 side by side.

[0063] That is, this step is a step showing an example of a process of acquiring an image similar to the image acquired from the managed image and displaying it side by side with the acquired image.

[0064] Specifically, as shown in FIG. 8, the image 810 acquired in S402 and the images 821, 822, and 823 acquired in S404 are displayed side by side.

[0065] Also, at that time, the differences between the image 810 and the images 821, 822, and 823 may be surrounded by round or square frames, blinked, or identified and displayed.

[0066] That is, this step is a step showing an example of a process of identifying and displaying the differences between an image similar to the acquired image and the acquired image.

[0067] The images (821, 822, 823) similar to the captured image (810) are arranged and displayed in the order of similarity. However, since there are various criteria for determining similarity as described above, it may be possible for the user to select such criteria (not shown). Specifically, the user can select criteria for arranging images such as "same variety", "color of the fruit", and "composition of the image".

[0068] That is, this step is a step showing an example of a process of displaying images similar to the acquired image in an order based on the similarity degree.

[0069] That is, this step is a step showing an example of a process of displaying images similar to the acquired image in an order of similarity degree based on the similarity criteria accepted from the user.

[0070] As described above, it becomes easier to compare the growth states of a plurality of similar farms, and it becomes possible to more accurately determine the growth state of agricultural and livestock products.

[0071] Also, as a result, the administrator of a farm where the growth has not progressed can improve the growth method, or the supervisor can give instructions to a farm where the growth has not progressed.

[0072] In addition, in this embodiment, the server device 101 is assumed to transmit the image acquired in S404 to the mobile terminal 201. However, the method is not limited to this. For example, the server device 101 may generate a page as shown in FIG. 8 and transmit the page information (e.g., URL, HTML, etc.) to the mobile terminal 201. Alternatively, the mobile terminal 201 may directly acquire images of other farms from the database server 102 instead of the server device 101.

[0073] The following processing is assumed to be performed by the mobile terminal 201, but it may also be performed by the server device 101 or by the cooperation of the mobile terminal 201 and the server device 101.

[0074] In S406, the mobile terminal 201 detects a marker (e.g., QR code (registered trademark)) (911 or 931 in FIG. 9) included in the image acquired in S402.

[0075] In S407, the mobile terminal 201 analyzes the size, inclination, and deformation of the marker detected in S406.

[0076] Specifically, the size is calculated by detecting the four corners of the marker, and the degree of inclination from the horizontal is calculated. Also, the degree of deformation of the marks at the four corners (here, double rectangles) is calculated.

[0077] For example, assuming that the marker in FIG. 9 is actually a square with a side length of 8 cm, it can be determined from the analysis that the marker 911 in FIG. 9 is inclined but maintains the shape of a square and has no deformation.

[0078] Also, the marker 931 in FIG. 9 is inclined approximately 10 degrees to the left. By comparing the sizes of the marks at the four corners, it can be seen that the front mark is larger and the back mark is smaller, indicating that the photo has depth. This can be determined from the analysis.

[0079] Regarding these analysis methods, the calculation formulas for determination as described above are known techniques, and there are various methods, so detailed descriptions are omitted.

[0080] In S408, based on the analysis in S407, the mobile terminal 201 superimposes and displays a grid on the image.

[0081] Specifically, when displaying a grid at 10 cm intervals, as shown by the dotted lines 920 and 940 in FIG. 9, the interval of the grid (1.25 times the size of the marker) is determined in proportion to the size of the marker (one side is 8 cm) by the analysis in S407, and the orientation of the grid is determined by aligning it with the inclination of the marker (parallel).

[0082] Also, based on the degree of deformation of the marker (the ratio and positional relationship of the sizes of the marks at the four corners), it is determined whether to display the grid in parallel or to express the depth using a perspective method in which the interval is narrowed as it goes deeper.

[0083] The calculation formula for the deformation processing of this grid is a known technique, and since there are various methods, a detailed description is omitted.

[0084] That is, this step shows an example of a process of controlling to superimpose and output a grid deformed based on the shape of the acquired marker image on the image acquired by the acquisition means.

[0085] That is, this step shows an example of a process of controlling to superimpose and output a grid with at least one of inclination or deformation on the image acquired by the acquisition means according to at least one of the orientation, inclination, or deformation of the acquired marker image.

[0086] As described above, since the grid is displayed in accordance with the inclination of the image, the growth state of the agricultural and livestock products can be determined more accurately.

[0087] Also, in S408, based on the displayed grid, the mobile terminal 201 displays the length of the stem (for example, the display of "16.7 cm" at 920 and the display of "40.8 cm" at 940 in FIG. 9) Specifically, the detected stem is linearly deformed and compared with the length of the grid to calculate and display the length of the stem. At this time, the target stem is identified and displayed with a thick line.

[0088] The calculation formula for obtaining the length of this stem is a known technique, and since there are various methods, a detailed explanation will be omitted.

[0089] That is, this step is a step showing an example of a process for calculating the size or length of an object in an image.

[0090] That is, this step is a step showing an example of a process for calculating the size or length of an object in an image based on a grid.

[0091] In this embodiment, the length of the longest stem among the stems in the image is displayed. However, it is not limited to this method, and the length and size may be displayed for all stems, leaves, buds, flowers, fruits, etc., or other methods such as displaying the length and size for the stems, leaves, buds, flowers, fruits for which an instruction is received from the user may also be used.

[0092] As described above, it becomes possible to more accurately determine the growth state of agricultural and livestock products.

[0093] In S409, the mobile terminal 201 identifies the stems, leaves, buds, flowers, and fruits of the agricultural products included in the image acquired in S402.

[0094] In S410, the mobile terminal 201 repeats the processes of S411 to S415 as many times as the number of stems, leaves, buds, flowers, and fruits identified in S402.

[0095] In S411, the mobile terminal 201 analyzes the growth characteristic quantities (size, length, color, density, shape, etc.) of the stems, leaves, buds, flowers, and fruits to be processed.

[0096] In S412, based on the growth feature quantities analyzed in S411, the mobile terminal 201 classifies the growth stage by using an AI model generated in advance.

[0097] For example, in the case of strawberry fruits, based on the color and size, it is classified into which growth stage among {green-ripe stage / white-ripe stage / red-ripe stage / harvest stage} it belongs to.

[0098] In S413, the mobile terminal 201 determines whether the reliability of the growth stage classified in S412 is equal to or higher than a predetermined value, that is, whether it is sufficiently reliable. If it is equal to or higher than the predetermined value, it returns to S410; if it is less than the predetermined value, it proceeds to S414.

[0099] In S414, the mobile terminal 201 determines the closest growth stage classification among the feature quantities of the variety selected in S401 in the variety master (Figure 10) stored in the database server 102 or the mobile terminal 201.

[0100] If the closest growth stage classification is the same as the growth stage by AI, it returns to S410; if it is different, it proceeds to S415.

[0101] Here, Figure 10 is a master of color and size for each variety, and statistical quantities of the feature quantities are registered in the master.

[0102] The statistical quantities are, in order from the left, "minimum value, average value, maximum value, standard deviation", and any value can be used in the determination of S414.

[0103] That is, Figure 10 is a diagram showing an example of a management means for managing the growth feature quantities of agricultural and livestock products for each variety.

[0104] Specifically, if the AI determines that the growth stage of the strawberry fruit of "variety A" included in the image is the "white-ripe stage", but the reliability is lower than the predetermined value, and as a result of determining which growth stage color of "variety A" the color of the fruit approximates using the master in Figure 10, it is determined that it most approximates the "red-ripe stage" rather than the "white-ripe stage", since it is different from the growth stage by AI, it proceeds to S415.

[0105] In addition, in this embodiment, the master of color and size for each variety is used. However, it is not limited to color and size, and it may be a master of length, density, shape, etc. other than color and size.

[0106] In S415, the mobile terminal 201 updates (overwrites) the growth stage classified by the AI in S412 with the growth stage determined in S414.

[0107] In S416, the mobile terminal 201 predicts the yield based on the growth stages of the stems, leaves, buds, flowers, and fruits classified in S412 or S415, and displays the predicted yield on the screen (707 in FIG. 7).

[0108] That is, this step shows an example of a process of predicting the yield of the agricultural and livestock product based on the determination of whether the feature amount of the image of the agricultural and livestock product approximates the growth feature amount corresponding to the received variety.

[0109] Thus, the description of FIG. 4 is completed.

[0110] As described above, when the reliability of the growth stage determination by AI is low, by using the master for each variety, it becomes possible to perform a more accurate yield prediction without generating a learning model for each variety.

[0111] Generating a learning model for each variety is very costly. Therefore, for farmers and agricultural corporations that grow various varieties, it becomes possible to perform a low-cost and highly accurate yield prediction.

[0112] In addition, in this embodiment, agricultural products such as strawberries are taken as an example. However, it is not limited to agricultural products, and livestock products such as cows, pigs, and chickens may also be used. In that case, the variety in S401 is the variety of the livestock product, the part specified in S409 is the whole body or a part of the body such as the legs, head, and face, the growth stage in S412 is divided into "birth stage, juvenile stage, growth stage, adult stage", etc., and the yield prediction in S416 is to predict the number of shipments or the number of eggs shipped.

[0113] Also, for reference, FIGS. 5 to 7 used in the yield prediction process will be described.

[0114] FIG. 5 is an example of a past yield performance table stored in the external memory 211 of the database server 102.

[0115] The past yield performance table is composed of a date 501 and a total field yield 502.

[0116] The total field yield 502 is estimated as the number of boxes filled with strawberries to be shipped. For example, assuming 1 box = 2 Kg, the number of boxes filled with strawberries shipped daily is estimated.

[0117] FIG. 6 is an example of an image analysis result table stored in the external memory 211 of the database server 102.

[0118] The image analysis result table is composed of a preset ID 601, a date 602, the number of highly mature fruits 603, and an estimated yield performance 604. The preset ID 601 indicates the location of the ridge 300 photographed by the network camera 200 or the mobile terminal 201. Note that the image data used in the analysis is also managed in association with the preset ID 601.

[0119] The number of highly mature fruits 603 represents the number of fruits that have matured to a harvestable state. The number of highly mature fruits 603 can be obtained by detecting highly mature fruits from the target image using a known technique such as "Faster R-CNN". The estimated yield performance represents an estimated value of the number of fruits harvested on a certain day.

[0120] The estimated yield performance 604 can be obtained by using the difference from the previous day of the number of highly mature fruits 603.

[0121] For example, when obtaining the first estimated yield achievement 604 of preset ID 601 on date 602 which is January 2nd, the difference between the value "20" of the high-maturity fruit count 603 on date 602 which is January 1st and the value "5" of the high-maturity fruit count 603 on date 602 which is January 2nd is taken, and it is estimated to be "15".

[0122] Figure 7 shows a method for calculating the error obtained by regression analysis.

[0123] The estimated yield achievement 702 is the value obtained by summing up the estimated yield achievements 604 for each date 602 in the combination of preset ID 601.

[0124] To obtain the correlation coefficients α and β, this estimated yield achievement 702 and the overall field yield 703 are plotted on the XY coordinates. The correlation coefficients α and β are obtained from the plotted XY coordinates and Y = αX + β is calculated.

[0125] Using this regression equation, the server device 101 calculates the regression estimated value 704 of the entire field from the estimated yield achievement. As described above, FIGS. 5 to 7 are used to predict the yield.

[0126] Regarding yield prediction, there are various other known techniques in addition to the above, and the present invention is not limited to the above yield prediction method.

[0127] Needless to say, the object of the present invention is also achieved by supplying a recording medium recording a program for realizing the functions of the above-described embodiments to a system or device, and the computer (or CPU or MPU) of the system or device reading and executing the program stored in the recording medium.

[0128] In this case, the program itself read from the recording medium realizes the novel functions of the present invention, and the recording medium storing the program constitutes the present invention.

[0129] As a recording medium for supplying a program, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a DVD-ROM, a magnetic tape, a non-volatile memory card, a ROM, an EEPROM, a silicon disk, etc. can be used.

[0130] Further, by executing the program read by the computer, not only the functions of the above-described embodiments are realized, but also based on the instructions of the program, the OS (operating system) etc. running on the computer performs part or all of the actual processing, and it goes without saying that the case where the functions of the above-described embodiments are realized by that processing is also included.

[0131] Furthermore, after the program read from the recording medium is written into the memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, based on the instructions of the program code, the CPU etc. provided in the function expansion board or function expansion unit performs part or all of the actual processing, and it goes without saying that the case where the functions of the above-described embodiments are realized by that processing is also included.

[0132] In addition, the present invention may be applied to a system composed of a plurality of devices or to an apparatus composed of a single device. Needless to say, the present invention is also applicable when it is achieved by supplying a program to a system or an apparatus. In this case, by reading the recording medium storing the program for achieving the present invention into the system or the apparatus, the system or the apparatus can enjoy the effects of the present invention.

[0133] The form of the above program may be in the form of object code, program code executed by an interpreter, script data supplied to an OS (operating system), etc.

[0134] Furthermore, by downloading and reading a program for achieving the present invention from a server, a database, etc. on a network by means of a communication program, the system or device can enjoy the effects of the present invention. Note that all configurations combining the above-described embodiments and their modifications are also included in the present invention.

Explanation of Reference Numerals

[0135] 100 Information Processing System 101 Server Device 102 Database Server 200 Network Camera 201 Mobile Terminal

Claims

1. An acquisition means for acquiring an image in which a marker is imaged; A control means for controlling to superimpose and output a grid deformed based on the shape of the image of the marker acquired by the acquisition means on the image acquired by the acquisition means An information processing system characterized by comprising.

2. The control means is Based on at least one of the orientation, inclination, or deformation of the image of the marker acquired by the acquisition means, controlling to superimpose and output a grid with at least one of inclination or deformation on the image acquired by the acquisition means The information processing system according to claim 1, characterized by this.

3. A calculation means for calculating the size or length of an object in the image The information processing system according to claim 1, further characterized by comprising.

4. The calculation means is Based on the grid, calculating the size or length of an object in the image The information processing system according to claim 3, characterized by this.

5. An acquisition step of acquiring an image in which a marker is imaged; A control step of controlling to superimpose and output a grid deformed based on the shape of the image of the marker acquired by the acquisition step on the image acquired by the acquisition step A control method for an information processing system characterized by comprising.

6. At least one computer, A program for functioning as each means of the information processing system described in any one of claims 1 to 4.

Images