Information processing device, control method, and program

Abstract

To provide a mechanism that estimates an event which occurs on the basis of a target recognized from an image.SOLUTION: An object detection determination notification server 110 calculates a total score for a combination of types of the object using a score that is added for each object being detected on the basis of a detection result of an object reflected on image data which is transmitted from a client PC 104 and which is obtained by being imaged by a network camera 102, and estimates that construction is performed in an area in which the object exists, in a case where the summed score exceeds a predetermined score.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] The present invention relates to a method for estimating whether a predetermined event has occurred based on an object recognized from an image.

Background Art

[0002] Recently, object detection devices equipped with computers use object detection models based on convolutional neural networks.

[0003] Learning data is created using image data in which the classes and positions of the objects to be detected are labeled, and the object detection model learns the positions and classes of the objects in the learning data, so that the object detection model can recognize and detect the objects in any captured image.

[0004] In order to improve the accuracy of object recognition, object recognition is performed using a model created according to the shooting situation such as the shooting position and time zone (for example, see Patent Document 1). In object detection by background difference, based on the color change tendency of the input image with respect to the background image, it is determined whether the object in the background image is detected due to mere lighting conditions or a newly appeared object is detected (for example, see Patent Document 2). It is considered to estimate the type of bird of the object included in the image based on the outline and flight manner of the bird (for example, see Patent Document 3).

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Patent Document 3

Disclosure of the Invention

Problems to be Solved by the Invention

[0006] However, while the inventions described in each patent document can improve the recognition accuracy for detecting a specific object, they do not go so far as to recognize events occurring in the area where the identified object exists.

[0007] For example, construction sites typically have construction-related objects (classes) such as construction signs, warning signs, traffic signs, excavators, zebra fences, plastic fences, arrows, or traffic cones (registered trademark). By recognizing these objects, it is possible to infer that the area where these objects are present is a construction site.

[0008] However, since similar-looking objects may also be photographed, a problem may arise in which the accuracy of recognizing events occurring in the area where the object is located decreases.

[0009] Therefore, the present invention aims to provide a mechanism for estimating whether a predetermined event has occurred based on an object recognized from an image. [Means for solving the problem]

[0010] To achieve the above objective, the present invention is characterized by comprising: an acquisition means for acquiring an object captured from image data; and an estimation means for estimating whether a predetermined event is occurring in the region where the object exists, based on a combination of objects acquired by the acquisition means. [Effects of the Invention]

[0011] According to the present invention, it is possible to estimate whether a predetermined event is occurring based on an object recognized from an image. [Brief explanation of the drawing]

[0012] [Figure 1] This is a configuration diagram showing an example of the general configuration of an information processing system. [Figure 2]It is a configuration diagram showing an example of the schematic configuration of the hardware of a client PC, an edge computer, an object detection determination notification server, and a monitoring PC. [Figure 3] It is a functional configuration diagram showing an example of the program configuration of an information processing system. [Figure 4] It is a flowchart showing the process of transmitting image data obtained by shooting with a network camera to a client PC. [Figure 5] It is a flowchart showing the process of combining the image data acquired by the client PC from the network camera and the position information obtained by the GPS function and transmitting them to the edge computer. [Figure 6] It is a flowchart showing the process of detecting an object from image data in an edge computer. [Figure 7] It is a flowchart showing the process of transmitting image data shot within a predetermined range in a client PC. [Figure 8] It is a flowchart showing the process of estimating whether it is a construction site in an object detection determination server. [Figure 9] It is a flowchart showing the process of outputting the determination result of object detection in an object detection determination notification server to a monitoring PC. [Figure 10] It is a schematic diagram showing an example of a learning class example. [Figure 11] It is a diagram showing an example of image data to be learned. [Figure 12] It is a diagram showing an example of a detection result. [Figure 13] It is a diagram showing an example of an annotation. [Figure 14] It is a diagram showing an example of scores for a class. [Figure 15] It is a configuration diagram showing the configuration of a database for storing detection results. [Figure 16] It is a configuration diagram showing the configuration on a memory for storing detection results. [Figure 17] It is a diagram showing an example of a map image of a construction site. [Figure 18] This is a diagram showing an example of the detected image data.

Embodiments for Carrying Out the Invention

[0013] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0014] FIG. 1 is a configuration diagram showing an example of the schematic configuration of an information processing system 100 in an embodiment of the present invention.

[0015] The construction monitoring patrol vehicle is equipped with a network camera 102, transmits and receives image data to and from external devices, and a client PC 104 equipped with a GPS / GNSS function (a device capable of receiving a GPS signal may be connected by USB), and an edge computer 106 equipped with a GPU for object detection from image data, and is configured to be connected via an in-vehicle local network 108.

[0016] An object detection determination notification server 110 arranged in a data center providing a cloud environment has a function of estimating that it is a construction site from an object detected from image data, and is connected to the client PC 104 and a monitoring PC 112 used by a monitor who monitors the construction site via the Internet 114.

[0017] FIG. 2 is a block diagram showing an example of the hardware configuration of each information processing device of the client PC 104, the edge computer 106, the object detection determination notification server 110, and the monitoring PC 112 in an embodiment of the present invention.

[0018] As shown in FIG. 2, the information processing device is connected with a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, an input controller 205, a video controller 206, a memory controller 207, and a communication I / F controller 208 via a system bus 204.

[0019] CPU201 provides comprehensive control over all devices and controllers connected to the system bus 204.

[0020] ROM202 or external memory211 holds the BIOS (Basic Input / Output System) and OS (Operating System), which are control programs executed by the CPU201, as well as computer-readable and executable programs and various necessary data (including data tables) for realizing this information processing method.

[0021] RAM203 functions as the main memory, work area, etc., of the CPU201. The CPU201 loads the necessary programs, etc., from ROM202 or external memory211 into RAM203, and then executes the loaded programs to perform various operations.

[0022] The input controller 205 controls input from input devices such as a keyboard 209 or a pointing device such as a mouse (not shown).

[0023] If the input device is a touch panel, users will be able to give various instructions by pressing (touching with their finger, etc.) icons, cursors, or buttons displayed on the touch panel.

[0024] Furthermore, the touch panel may be a multi-touch screen or other touch panel capable of detecting the positions of multiple fingers touching it.

[0025] The video controller 206 controls the display to external output devices such as the display 210. The display includes the display of a notebook computer integrated with the main unit.

[0026] Furthermore, the external output device is not limited to a display; for example, it may be a projector. Also, for the aforementioned touch-enabled device, an input device is provided.

[0027] The video controller 206 can control the video memory (VRAM) used for display control. It can utilize a portion of the RAM 203 as the video memory area, or it can provide a separate, dedicated video memory.

[0028] The memory controller 207 controls access to the external memory 211. The external memory can include an external storage device (hard disk), a flexible disk (FD), or a CompactFlash® memory connected to a PCMCIA card slot via an adapter, which stores boot programs, various applications, font data, user files, editing files, and other data.

[0029] The communication interface controller 208 connects to and communicates with external devices via a network and performs communication control processing over the network.

[0030] For example, it is possible to communicate using TCP / IP, telephone lines such as ISDN, and 3G mobile phone lines.

[0031] Furthermore, the CPU 201 enables display on the display 210 by, for example, performing the process of expanding (rasterizing) outline fonts into the display information area in RAM 203.

[0032] Furthermore, the CPU 201 enables user input via a mouse cursor (not shown) on the display 210.

[0033] Furthermore, the edge computer 106 could be configured with a GPU (Graphics Processing Unit) that excels at real-time image processing, rather than a CPU.

[0034] Next, with reference to Figure 3, an example of the functional configuration of various devices in the embodiment of the present invention will be described.

[0035] Figure 3 shows an example of the program configuration of an information processing system in an embodiment of the present invention. Details of each function will be explained later in conjunction with the flowchart.

[0036] The network camera 102 includes a camera module 300 and an image transmission / reception program 301. The camera module 300 processes the image data obtained by capturing images and outputs it to the image transmission / reception program 301. The image transmission / reception program 301 transmits the image data input from the camera module 300 to the client PC 104.

[0037] The client PC 104 is equipped with an image processing program 302 and an object detection transmission / reception program 303. The image processing program 302 acquires image data from the image transmission / reception program 301 and transmits it to the edge computer 106. The object detection transmission / reception program 303 receives information about the object extracted from the image data received from the edge computer 106, as well as location information where the image data was taken, and transmits it to the object detection determination notification server 110.

[0038] The edge computer 106 is equipped with an object detection program 304. It detects objects in image data transmitted from the client PC 104 using a model generated by training on a large amount of image data to be detected, and transmits the detection results to the object detection transmission / reception program 303.

[0039] The object detection determination notification server 110 includes an object detection determination program 305, a detection database 306, and a detection result display program 307. The object detection determination program 305 determines whether or not it is a construction site based on the object information and location information received by the object detection transmission / reception program 303, and stores the information related to the determination in the detection database 306.

[0040] The detection result display program 307 obtains information from the detection database 306 regarding whether or not it is a construction site and sends it to the monitoring PC 112.

[0041] The monitoring PC 112 is equipped with a display program 308, which, based on the information related to the determination received from the detection result display program 307, displays a map image with the construction site location pinned, and displays the construction site image via a link from the pinned location on the map image.

[0042] Next, the process for estimating a construction site, as performed by the information processing system 100 in the embodiment of the present invention, will be explained using the flowcharts shown in Figures 4 to 9.

[0043] The flowchart shown in Figure 4 illustrates the process of sending image data obtained by the network camera 102 to the client PC 104.

[0044] In step S100, the image transmission / reception program 301 receives an image data request (the request sent in step S200) from the image processing program 302. In step S102, the image transmission / reception program 301, having received the image data request in step S100, acquires the image data obtained by the camera module 300 through shooting and sends it to the client PC 104.

[0045] The flowchart shown in Figure 5 illustrates the process by which the client PC 104 transmits image data acquired from the network camera 102 and location information obtained using the GPS function to the edge computer 106.

[0046] This process is initiated when the system is accessed via a specified URL, etc., but the timing of its activation is not limited to this method; it may be activated at any time the user desires to use the system.

[0047] In step S200, the image processing program 302 sends an image data request to the network camera 102.

[0048] In step S202, the image processing program 302 receives the image data transmitted from the network camera 102 (the image data transmitted in step S302) as a response to the request in step S200.

[0049] In step S204, the image processing program 302 uses the GPS module and the GPS library to obtain the latest location information (latitude and longitude).

[0050] In step S206, the image processing program 302 transmits the image data received in step S202, the latest location information (latitude and longitude) and the current date and time (year, month, day, hour, minute, second, microseconds) acquired in step S204 to the edge computer 106.

[0051] In step S208, the image processing program 302 sleeps on the edge computer 106 according to the processing time required for object detection (e.g., if object detection takes 0.5 seconds, it sleeps for 0.5 seconds). After waking from sleep, processing resumes from step S200.

[0052] Next, we will explain the process of detecting objects from image data on the edge computer 106 using the flowchart shown in Figure 6.

[0053] In step S300, the object detection program 304 receives image data, the latest location information (latitude and longitude), and date and time (year, month, day, hour, minute, second, microsecond) transmitted from the client PC 104 (image data, the latest location information (latitude and longitude), and date and time (year, month, day, hour, minute, second, microsecond) transmitted in step S206).

[0054] In step S302, the object detection program 304 detects an object from the image data received in step S300.

[0055] As a method for detecting objects, as shown in the example training class in Figure 10, objects are detected from image data regarding the objects to be detected at the construction site.

[0056] The example class for learning consists of a No. to uniquely identify the class, a major category indicating the classification to which the object belongs, a subcategory indicating the object, and a class name indicating the name of the class.

[0057] This class refers to a model generated by mechanically learning from a large amount of image data related to each object, as shown in Figure 11.

[0058] These models are used to detect objects from image data, and as a result, detection results that demonstrate a confidence level of at least a predetermined value (e.g., 0.5 or higher) indicating that the object is identified are used.

[0059] The example of detection results shown in Figure 12 is an example where two cones, one excavator, and one warning sign were detected.

[0060] Position X and Position Y are pixel values ​​that point to the top-left origin of the detected class within the image data, width and height represent the length from the origin of the class in pixel values, and confidence indicates the likelihood that the object identified in the image data is indeed the object.

[0061] In step S304, if the object detection program 304 determines that there is one or more objects as a result of detection in step S302, it proceeds to step S306; otherwise, it proceeds to step S308.

[0062] In step S306, the object detection program 304 adds annotations to the image data indicating the detection results of the object detection model.

[0063] Annotation is the process of adding an annotation to image data by drawing a rectangle on the image based on the X, Y, width, and height of each row of the detection result, and then drawing the class name and confidence level on top of the rectangle.

[0064] As shown in Figure 13, an example of adding annotations to image data is shown, where a rectangle surrounding the detected excavator, a character identifying the class name (Sh), and a confidence level are annotated.

[0065] In step S308, the object detection program 304 sends the image data, the latest location information (latitude and longitude), date and time (year, month, day, hour, minute, second, microsecond), and the detection result to the client PC 104.

[0066] Next, we will explain the process of transmitting image data captured within a predetermined range on the client PC 104 using the flowchart shown in Figure 7.

[0067] In step S400, the object detection transmission / reception program 303 receives the image, the latest location information (latitude and longitude), date and time (year, month, day, hour, minute, second, microsecond), and the detection result from the client PC 104 (data transmitted in step S308).

[0068] In step S402, the object detection transmission / reception program 303 calculates the distance traveled from the latest position information (latitude and longitude) and reference position information (latitude and longitude) received in step S400.

[0069] The reference location information (latitude and longitude) is the location information (latitude and longitude) at the time of initial reception, and the location information (latitude and longitude) at the time the travel distance exceeds the threshold in step S404, which is stored in memory. The travel distance from the two location information points (latitude and longitude) is calculated using spherical trigonometry as shown below.

[0070] D=R*math.acos(math.cos(math.radians(lat1)*math.cos(math.radians(lat2)*math.cos(math.radians(lng2)-math.radians(lng1))+math.sin(math.radians(lat1)*math.sin(math.radians(lat2)) D: Distance between two points (m) R:Earth radius=6378137.0 lat1: Latitude of the first point lat2: Latitude of the second point lng1: Longitude of the first point lng2: Longitude of the point's location math.acos: Inverse cosine function in radians math.cos: Cosine function in radians math.sin: Sine function in radians math.radians:radian conversion function

[0071] In step S404, the object detection transmission / reception program 303 determines whether the distance traveled calculated in step S402 exceeds a threshold. If it determines that the distance traveled exceeds the threshold, it proceeds to step S406; otherwise, it proceeds to step S408.

[0072] The threshold should preferably be determined according to the size of the construction area typically used (for example, if the area of ​​one construction project is typically within 30 meters, the threshold should be set to 30 meters).

[0073] In step S406, the object detection transmission / reception program 303 transmits the latest location information (latitude and longitude) to the object detection determination notification server 110.

[0074] In step S408, the object detection transmission / reception program 303 determines whether there is one or more detection results. Note that there may be multiple object detection results for a single image data, so in this step, it is set to one or more records.

[0075] In step S410, the object detection transmission / reception program 303 transmits image data, the latest location information (latitude and longitude), date and time (year, month, day, hour, minute, second, microsecond), and the detection result to the object detection determination notification server 110.

[0076] Next, we will explain the process by which the object detection determination notification server 110 estimates whether or not it is a construction site, using the flowchart shown in Figure 8.

[0077] In step S500, the object detection program 305 receives image data transmitted from the client PC 104, along with the latest location information (latitude and longitude), date and time (year, month, day, hour, minute, second, microsecond), and the detection result (data transmitted in step S406 or step S410).

[0078] In step S502, the object detection program 305 determines whether the data received in step S500 consists only of the latest location information (latitude and longitude). If the received data consists only of the latest location information (latitude and longitude), the program proceeds to step S512.

[0079] In step S504, the object detection and judgment program 305 performs construction judgment processing calculations using image data with the following logic. The classification, class, and score are calculated based on the information shown in Figure 14.

[0080] (1) Add up the scores of all classifications / classes present in the detection results. (2) Even if there are multiple instances of the same class, the scores will remain the same as the scores for the class. (3) Notice, Caution, and Traffic Guide signs will not be counted towards the total points for the signs. (4) If Zebla_fence and Plastic_fence are mixed, the fence points will not be added.

[0081] In step S506, the object detection and determination program 305 performs construction determination processing based on the score calculated in step S502 using the following logic.

[0082] If the score counter reads 100 points or more, it is determined to be a construction site. If the score meter reads 50 points or higher, it is determined that there is a possibility of a construction site. If the score meter reads less than 50 points, it will be determined that it is not a construction site.

[0083] In this step, if it is determined that it is a construction site, the process proceeds to step S508; otherwise, the process proceeds to step S510.

[0084] In step S508, the object detection and determination program 305 saves the results of the construction determination process to the detection database 306.

[0085] Image data is added to a designated storage area in the detection database 306, and a URL for accessing the image data is obtained.

[0086] As shown in Figure 15, add the primary key date and time (year, month, day, hour, minute, second, microsecond), location information (latitude, longitude), score, total score (register the score), and a URL for accessing the image data.

[0087] In step S510, the object detection program 305 stores the image data, the latest location information (latitude and longitude), and the date and time (year, month, day, hour, minute, second, microsecond) in memory.

[0088] As shown in Figure 16, the primary key date and time (year, month, day, hour, minute, second, microsecond), location information (latitude, longitude), point count, and image data are saved.

[0089] In step S512, the object detection determination program 305 checks whether there are two or more data items in the memory that will be saved in step S510. If it determines that there are, it proceeds to step S514; otherwise, it proceeds to step S520.

[0090] In step S514, the object detection and determination program 305 performs construction determination processing calculations based on the detection results for multiple image data stored in memory using the following logic. The classification, class, and score are calculated based on the information shown in Figure 14.

[0091] (1) Add up the scores of all classifications and classes present in all detection results that span across image data in memory. (2) Even if there are multiple instances of the same class, the scores will remain the same as the scores for the class. (3) Notice, Caution, and Traffic Guide signs will not be counted towards the total points for the signs. (4) If Zebla_fence and Plastic_fence are mixed, the fence points will not be added.

[0092] In step S516, the object detection and determination program 305 performs construction determination processing based on the score calculated in step S514 using the following logic.

[0093] If the score counter reads 100 points or more, it is determined to be a construction site. If the score meter reads 50 points or higher, it is determined that there is a possibility of a construction site. If the score meter reads less than 50 points, it will be determined that it is not a construction site.

[0094] In this step, if it is determined that the location is a construction site, the process proceeds to step S518; otherwise, the process proceeds to step S520.

[0095] In step S518, the object detection and determination program 305 saves all construction determination results present in memory to the detection database 306.

[0096] Image data is added to a designated storage area in the detection database 306, and a URL for accessing the image data is obtained.

[0097] As shown in Figure 15, the primary key includes the date and time (year, month, day, hour, minute, second, microsecond), location information (latitude, longitude), the score, the total score calculated by aggregating scores across image data, and a URL for accessing the image data.

[0098] In step S520, the object detection and determination program 305 erases the data stored in memory.

[0099] Next, using the flowchart shown in Figure 9, we will explain the process by which the object detection determination result in the object detection determination notification server 110 is output to the monitoring PC 112.

[0100] In step 600, the display program 308 accepts input of an initial display URL to call the detection result display program 307 and sends a request to the object detection determination notification server 110 to display the monitoring screen.

[0101] In step S602, the detection result display program 307 receives a request from the monitoring PC 112 to display the monitoring screen.

[0102] In step S604, the detection result display program 307 sends the HTML and JavaScript (registered trademark) control programs for the initial screen of the monitoring screen to the monitoring PC 112.

[0103] In step S606, the display program 308 receives the HTML and JavaScript control program for the initial screen of the monitoring screen, which were sent from the object detection determination notification server 110 in step S604.

[0104] In step S608, the display program 308 renders the HTML for the initial screen of the monitoring screen and starts the JavaScript monitoring and control program.

[0105] In step S610, the display program 308 sends an inquiry request to the object detection determination notification server 110 to the detection result inquiry URL of the detection result display program 307.

[0106] In step 612, the detection result display program 307 receives the inquiry request sent from the monitoring PC 112 in step S610.

[0107] In step S614, the detection result display program 307 retrieves all records in chronological order of date and time (year, month, day, hour, minute, second, microsecond) for the detection result entities (see Figure 15) in the detection database 306, if there are records within the last hour of the current date and time (year, month, day, hour, minute, second, microsecond).

[0108] In step S616, the detection result display program 307 sends all records acquired in step S614 to the monitoring PC 112.

[0109] In step S618, the display program 308 receives a record relating to the detection result sent from the object detection determination notification server 110 in step S616.

[0110] In step S620, the display program 308 determines that construction is being carried out if at least one record was received in step S618, and proceeds to step S622. If no records are found, it determines that construction is not being carried out, and proceeds to step S624.

[0111] In step S622, the display program 308 displays a map image with construction sites pinned from the detection result record.

[0112] For example, as shown in Figure 17, the map image 400 of the construction site displays the image data for the record, sorted in ascending order by date and time as detected image 402, detected image 404, and detected image 406. The total score of detected image 402 is displayed as score 408, the total score of detected image 404 as score 410, and the total score of detected image 406 as score 412.

[0113] Then, the locations where each image data was captured are marked on the map with pins: shooting position 414 for the location where detected image 402 was taken, shooting position 416 for the location where detected image 404 was taken, and shooting position 418 for the location where detected image 406 was taken. The total scores for each shooting position are then displayed as scores 420, 420, and 424.

[0114] By clicking or interacting with the displayed pins, the image data captured at the corresponding shooting location will be displayed.

[0115] For example, in the map image 400 of the construction site shown in Figure 17, touching the shooting position 414 will cause the detected image 402, which is image data obtained by shooting at the shooting position 414, to be zoomed in and enlarged, as shown in Figure 18.

[0116] In step S624, the display program 308 goes to sleep in accordance with the timing of the process for detecting the desired construction site (for example, a few seconds), and after a predetermined time has elapsed, proceeds to step S205.

[0117] In this embodiment, events occurring at a construction site are estimated based on objects recognized from the image. However, the method is not limited to such objects; any combination of objects recognized from the image that allows for the estimation of events occurring in the direction the image was taken may be used as the target.

[0118] In the embodiments of the present invention, the process of estimating a construction site has been described as being performed by a network camera 102, a client PC 104, an edge computer 106, and an object detection determination notification server 110. However, it is also possible to configure the client PC 104, the edge computer 106, and the object detection determination notification server 110 in a single device.

[0119] Alternatively, the client PC 104 and the edge computer 106 may be configured as a single device, and the network camera 102 may also be equipped with these functions in that single device.

[0120] Furthermore, although the processing is performed on image data obtained by capturing images online with the network camera 102, it is also possible to acquire already captured (offline) image data stored on the client PC 104, edge computer 106, or object detection determination notification server 110 and perform the respective processing.

[0121] As described above, the embodiments of the present invention are described, but the program in the present invention is a program that a computer can execute the processing method of the flowcharts shown in Figures 4 to 9, and the storage medium of the present invention stores the program that a computer can execute the processing method of Figures 4 to 9.

[0122] The programs in this invention may also be programs for each processing method of the devices shown in Figures 4 to 9.

[0123] As described above, it goes without saying that the object of the present invention can also be achieved by supplying a recording medium containing a program that realizes the functions of the embodiments described above to a system or device, and by having the computer (or CPU or MPU) of that system or device read and execute the program stored on the recording medium.

[0124] In this case, the program read from the recording medium itself realizes the novel function of the present invention, and the recording medium on which that program is recorded constitutes the present invention.

[0125] For recording media used to supply programs, examples include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, magnetic tapes, non-volatile memory cards, ROMs, EEPROMs, silicon disks, and the like.

[0126] Furthermore, it goes without saying that the functions of the aforementioned embodiments are realized not only by the computer executing the program it has read, but also by the operating system (OS) running on the computer performing some or all of the actual processing based on the instructions of that program, thereby realizing the functions of the aforementioned embodiments.

[0127] Furthermore, it goes without saying that this also includes cases where, after a program read from a recording medium is written to the memory of a function expansion board inserted into a computer or a function expansion unit connected to a computer, the CPU or other components of the function expansion board or function expansion unit perform some or all of the actual processing based on the instructions of the program code, and the functions of the aforementioned embodiments are realized through that processing.

[0128] Furthermore, the present invention may be applied to a system consisting of multiple devices or to a device consisting of a single device.

[0129] Furthermore, it goes without saying that the present invention can also be applied when the invention is achieved by supplying a program to a system or device. In this case, by reading a recording medium containing a program for achieving the present invention into the system or device, the system or device can enjoy the effects of the present invention.

[0130] Furthermore, by downloading and reading the program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or device can enjoy the effects of the present invention. It should be noted that configurations combining the above-described embodiments and their variations are all included in the present invention. [Explanation of Symbols]

[0131] 100 Information Processing Systems 102 Network Cameras 104 Client PCs 106 Edge Computers 108 Local Network 110 Object Detection Judgment Notification Server 112 Monitoring PC 114 Internet

Claims

1. A means for identifying images relating to one or more objects from among the captured images, An output means that outputs information relating an image identified by the aforementioned identification means, information related to the image, and an evaluation value of the image based on the type of object detected from the image, Equipped with, A classification includes one or more types of objects. The aforementioned evaluation value is the sum of the scores related to the classification calculated using the scores corresponding to the types of objects detected from the image. If multiple objects belonging to the same classification are detected with the same score, the score for one of the detected objects with the same score will be used as the score related to that classification. An information processing device characterized by the following.

2. The information processing apparatus according to claim 1, characterized in that the information relating to the aforementioned image is information indicating the location where the image was taken.

3. The information processing apparatus according to claim 1 or 2, characterized in that the information relating to the aforementioned image is information indicating the date on which the image was taken.

4. The information relating to the aforementioned image is characterized by being information that shows the location where the image was taken on a map. The information processing apparatus according to any one of claims 1 to 3, characterized in that the output means outputs information indicating the location where the image was taken on a map, alongside the image.

5. The information processing apparatus according to any one of claims 1 to 4, characterized in that the output means displays an object detected from the image in an identifiable manner.

6. The information processing apparatus according to any one of claims 1 to 5, characterized in that the output means outputs the number of objects detected from the image.

7. The information processing apparatus according to any one of claims 1 to 6, characterized in that the aforementioned image is a plurality of images.

8. The information processing apparatus according to claim 7, characterized in that the aforementioned image is a plurality of images taken while changing the shooting position.

9. The information processing apparatus according to any one of claims 1 to 8, characterized in that the identifying means identifies a plurality of images relating to the same object.

10. The information processing device according to any one of claims 1 to 9, characterized in that the object is an object used in construction.

11. The aforementioned identification means is characterized by identifying images relating to the object using a trained model that has been trained to learn images including the object used in the construction work. The information processing device according to claim 10, characterized in that the evaluation value is calculated based on the type of object used in the construction work identified by the specific means.

12. A means for identifying images relating to one or more objects from among the captured images, An output means that outputs information relating an image identified by the aforementioned identification means, information related to the image, and an evaluation value of the image based on the type of object detected from the image, Equipped with, A classification includes one or more types of objects. The aforementioned evaluation value is the sum of the scores related to the classification calculated using the scores corresponding to the types of objects detected from the image. If multiple objects belonging to the same classification are detected with the same score, the score for one of the detected objects with the same score will be used as the score related to that classification. An information processing system characterized by the following.

13. The identification means of the information processing device includes an identification step of identifying an image relating to one or more objects from among the captured images, An output means of an information processing device outputs information that associates an image identified by the specified step, information related to the image, and an evaluation value of the image based on the type of object detected from the image. Equipped with, A classification includes one or more types of objects. The aforementioned evaluation value is the sum of the scores related to the classification calculated using the scores corresponding to the types of objects detected from the image. If multiple objects belonging to the same classification are detected with the same score, the score for one of the detected objects with the same score will be used as the score related to that classification. A control method for an information processing device characterized by the following.

14. The identification means of the information processing device includes an identification step of identifying an image relating to one or more objects from among the captured images, An output means of an information processing device outputs information that associates an image identified by the specified step, information related to the image, and an evaluation value of the image based on the type of object detected from the image. Equipped with, A classification includes one or more types of objects. The aforementioned evaluation value is the sum of the scores related to the classification calculated using the scores corresponding to the types of objects detected from the image. If multiple objects belonging to the same classification are detected with the same score, the score for one of the detected objects with the same score will be used as the score related to that classification. A control method for an information processing system characterized by the following.

15. A program for causing a computer to function as one of the means described in any one of claims 1 to 11.

Images