Information processing system, information processing method, and program

Abstract

To provide an information processing system, an information processing method, and a program for analyzing a moving image.SOLUTION: In an information processing system 1000, a server device 100 sets a plurality of detection areas in a moving image photographed by a camera 120 installed at a prescribed position, and selects a first detection area and a second detection area from among the plurality of detection areas. It counts the number of objects moved from the first detection area to the second detection area. It displays the number of objects in association with information showing the first detection area and the second detection area in a client device 110.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] Patent Document 1 discloses an apparatus that sets a passing line in a captured image, measures the number of vehicles that cross the passing line, estimates the vehicle types of the vehicles that cross the passing line, and measures the traffic volume by vehicle type.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The apparatus described in Patent Document 1 has a problem in that it is impossible to grasp from which detection area to which detection area an object included in a video has moved with respect to a plurality of detection areas included in the video.

Means for Solving the Problems

[0005] According to one aspect of the present invention, there is provided an information processing system that analyzes a video. This information processing system sets a plurality of detection areas in the video. A first detection area and a second detection area are selected from among the plurality of detection areas. The number of objects that have moved from the first detection area to the second detection area is counted. The number of objects is displayed in association with information indicating the first detection area and the second detection area.

Brief Description of the Drawings

[0006]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

MODE FOR CARRYING OUT THE INVENTION

[0007] Hereinafter, embodiments will be described with reference to the drawings. Various characteristic matters shown in the following embodiments (including the following modification examples. The same applies hereinafter) can be combined with each other.

[0008] In the specification, the "unit" may include, for example, hardware resources implemented by a circuit in a broad sense and information processing of software that can be specifically realized by these hardware resources. Also, in Embodiment 1, various types of information are handled. Regardless of whether these information are represented by the high and low of signal values as a set of binary bits composed of 0 or 1, by physical numerical values of signal values, or by quantum superposition, communication and calculation can be executed on a circuit in a broad sense.

[0009] Also, the circuit in a broad sense is a circuit realized by appropriately combining at least a circuit, circuitry, a processor, a memory, etc. That is, it includes an application specific integrated circuit (ASIC), programmable logic devices (for example, a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0010] Also, the program for realizing the software appearing in the embodiment may be implemented in a manner downloadable from a server, the program may be executed on a cloud computer, or may be stored in a non-volatile or volatile non-temporary storage medium and distributed.

[0011] <Embodiment 1> 1. System Configuration FIG. 1 is a diagram showing an example of the system configuration of the information processing system 1000. As shown in FIG. 1, the information processing system 1000 includes, as a system configuration, a server device 100, a client device 110, and a camera 120. The server device 100, the client device 110, and the camera 120 are communicably connected via a network 150. Note that the server device 100 may be implemented by a cloud system. The camera 120 is an example of an imaging device.

[0012] The client device 110 accesses the server device 100, causes a screen or the like described later to be displayed on a display unit, and counts, for example, objects included in a video according to an operation of an operator of the client device 110 (hereinafter simply referred to as an operator). The camera 120 is, for example, a network camera, shoots a road or the like, and transmits shooting data (video) to the server device 100. The server device 100 stores the shooting data transmitted from the camera 120 in a predetermined storage area, or causes a screen to be displayed on the client device 110 in response to a request from the client device 110. More specifically, the server device 100 executes the processes of the embodiments shown below. In FIG. 1, only one camera 120 is shown for simplicity, but the information processing system 1000 may include a plurality of cameras 120. However, hereinafter, for simplicity of explanation, one camera 120 will be described as an example. Similarly, in FIG. 1, only one client device 110 is shown for simplicity, but the information processing system 1000 may include a plurality of client devices 110. However, hereinafter, for simplicity of explanation, one client device 110 will be described as an example.

[0013] Also, in FIG. 1, a PC (Personal Computer) is shown as an example of the client device 110, but it is not limited to a PC, and may be a tablet computer or the like. The client device 110 may be any device as long as it can display a screen described later and receive an input via the screen. However, hereinafter, for simplicity of explanation, the client device 110 will be described as an example of a PC.

[0014] Here, the information processing system described in the claims may be composed of a plurality of devices or may be composed of a single device. When the information processing system described in the claims is composed of a single device, an example of such a device is the server device 100. When the information processing system described in the claims is composed of a plurality of devices, an example of the plurality of devices is the server device 100 and the client device 110, or a cloud system that provides the functions of the server device 100. The cloud system is composed of, for example, a plurality of server devices.

[0015] 2. Hardware Configuration (Hardware Configuration of Server Device 100) FIG. 2 is a diagram showing an example of the hardware configuration of the server device 100. The server device 100 includes, as its hardware configuration, a control unit 210, a storage unit 220, and a communication unit 230.

[0016] The control unit 210 is a CPU (Central Processing Unit) or the like and controls the entire server device 100. The storage unit 220 is any one of an HDD (Hard Disk Drive), a ROM (Read Only Memory), a RAM (Random Access Memory), an SSD (Solid State Drive), or any combination thereof, and stores programs and data (for example, photographed data, learned models) used when the control unit 210 executes processing based on the programs. By the control unit 210 executing processing based on the programs stored in the storage unit 220, the processing of the server device 100 and the functions of the server device 100 shown in the sequence diagram shown in FIG. 4 described later are realized. The storage unit 220 is an example of a storage medium. In the first embodiment, the data used when the control unit 210 executes processing based on the programs is described as being stored in the storage unit 220, but it may be stored in the storage unit of another device communicable with the server device 100 or the like. That is, the data may be stored in the storage unit of any device as long as the control unit 210 can refer to it. The communication unit 230 is a NIC (Network Interface Card) or the like, connects the server device 100 to the network 150, and controls communication with other devices.

[0017] (Hardware Configuration of Client Device 110) FIG. 3 is a diagram showing an example of the hardware configuration of the client device 110. The client device 110 includes, as a hardware configuration, a control unit 310, a storage unit 320, an input unit 330, an output unit 340, and a communication unit 350.

[0018] The control unit 310 is a CPU or the like, and controls the entire client device 110. The storage unit 320 is any one of an HDD, a ROM, a RAM, an SSD, or any combination thereof, and stores programs and data used when the control unit 310 executes processing based on the programs. By the control unit 310 executing processing based on the programs stored in the storage unit 320, the functions of the client device 110 and the like are realized. The input unit 330 is a keyboard, a mouse, etc., and inputs operation information and the like based on the operations of the operator. The output unit 340 is a display or the like, and outputs the information input by the input unit 330, or outputs the result of the processing by the control unit 310 or the like. The communication unit 350 connects the client device 110 to the network 150 and controls communication with other devices.

[0019] 3. Information Processing Hereinafter, the information processing according to Embodiment 1 will be described. (Overview of Processing) The control unit 210 of the server device 100 detects one or more objects included in the video. The control unit 210 accepts the selection of a plurality of detection areas related to the video. The control unit 210 analyzes the movement of one or more objects in a plurality of detection areas based on the video. The control unit 210 controls so that the result of the analysis is displayed. The result of the analysis includes information on which detection area among the plurality of detection areas the one or more objects have moved from and to which detection area. According to such a configuration, it is possible to grasp which detection area among the plurality of detection areas included in the video the object included in the video has moved from and to which detection area. Note that an example of an object is a vehicle. A vehicle refers to something that can move with a person on it. Vehicles include, for example, vehicles such as trucks, buses, cars, and motorcycles. In Embodiment 1, the vehicle will be described as an example of an object.

[0020] (Details of Processing) FIG. 4 is a sequence diagram showing an example of information processing in the information processing system 1000. In sequence SQ401, the camera 120 installed at a predetermined position photographs a road or the like included in the photographing range. In sequence SQ402, the camera 120 transmits the photographed image data to the server device 100. Camera 120 always repeats the processing of sequence SQ401 and sequence SQ402. Note that camera 120 may repeat the processing of sequence SQ401 and sequence SQ402 at the time indicated by the set time information based on the set time information. In that case, the time information is, for example, 6:00 to 18:00, 8:00 to 12:00, etc. The time information can also be set from the client device 110 via the server device 100.

[0021] When receiving the captured data from camera 120, in sequence SQ403, the control unit 210 of the server device 100 causes the received captured data to be stored in the storage unit 220 or the like.

[0022] In sequence SQ404, the control unit 210 transmits the traffic volume survey screen to the client device 110 based on the display request of the traffic volume survey screen from the client device 110. When receiving the traffic volume survey screen from the server device 100, in sequence SQ405, the control unit 310 causes the received traffic volume survey screen to be displayed on the output unit 340 or the like.

[0023] In Embodiment 1, it is described that the server device 100 generates a screen and transmits it to the client device 110. Then, the client device 110 displays the received screen. However, as another example, the server device 100 may transmit the data necessary for generating the screen to the client device 110. Then, the client device 110 may generate and display a screen based on the received data. In the case of such processing as well, the process of transmitting the data necessary for the screen to the client device 110 is an example of the process of controlling the display of the screen.

[0024] FIG. 5 is a diagram showing an example of a traffic volume survey screen 500 displayed on the client device 110. As shown in FIG. 5, the traffic volume survey screen 500 includes a camera selection area 510, a video selection area 520, a detection area selection area 530, an analysis area 540, and a result display area 550. The camera selection area 510 is an area for selecting the camera 120. The video selection area 520 is an area for selecting a video. The detection area selection area 530 is an area for selecting a detection area. The analysis area 540 is an area for inputting an analysis instruction or the like. The result display area 550 is an area where the result of the analysis or the like is displayed. Details of each area will be described with reference to FIGS. 6 to 16 and the like described later. Note that the display, transition, etc. of the screens described later are performed by the control unit 210.

[0025] FIG. 6 is a diagram for explaining the details of the camera selection area 510. When the button 610 is selected in the camera selection area 510, the control unit 210 displays a list of cameras included in the information processing system 1000 so that it can be selected. The operator can select a desired camera from the displayed list of cameras. In the example of FIG. 6, camera A is selected. When the button 620 is selected, the control unit 210 controls to display the pull-down menu 630. The operator can select a desired process from the list of processes related to the cameras included in the pull-down menu 630. The list of processes includes adding a camera, changing the name of a camera, deleting a camera, updating the camera list, and the like. When "changing the name of a camera" is selected from the pull-down menu 630, the control unit 210 displays a pop-up window 640. The operator can input a new camera name in the pop-up window 640.

[0026] When a camera is selected in the camera selection area 510, in sequence SQ406, identification information for identifying the selected camera is transmitted from the client device 110 to the server device 100. Upon receiving the identification information of the camera, the control unit 210 displays, in a video selection area 520, the captured data captured by the camera identified by the identification information of the camera so as to be selectable. These processes accept the selection of the imaging device via the screen. And it is an example of a process of making the video captured by the selected imaging device selectable via the screen.

[0027] FIG. 7 is a diagram for explaining details of the video selection area 520. In the video information list 710, a list of shooting data of the camera selected in the camera selection area 510 is displayed. Since the shooting data of the camera is stored in the storage unit 220 of the server device 100 or the like, the control unit 210 controls to display a list of the shooting data of the selected camera in the video information list 710. These shooting data can be created, for example, by using a movie clip function of a viewer or the like to cut out a video of a partial time period from the video shot by the camera. In the example of FIG. 7, only one piece of shooting data is displayed in the video information list 710, but if there are a plurality of shooting data, they will of course be displayed. The operator may select one piece of shooting data from the video information list 710, or may select shooting data by uploading shooting data or the like stored in the client device 110 to the server device 100 as described later. When the upload button 720 is selected, the control unit 210 controls to display the pull-down menu 730. When "Insert a video file from the PC" is selected from the pull-down menu 730, the control unit 210 displays the pop-up window 740. For example, when it is detected that the shooting data stored in the storage unit 320 of the client device 110 or the like is selected and dragged and dropped into a predetermined area of the pop-up window 740, the control unit 210 uploads the shooting data to the server device 100. However, hereinafter, for simplicity of explanation, it will be described that the operator selects one piece of shooting data from the video information list 710, but a plurality of installation data may be selected.

[0028] When one piece of shooting data is selected from the video information list 710, in sequence SQ407, identification information for identifying the selected video (shooting data) is transmitted from the client device 110 to the server device 100. When receiving the identification information of the shooting data, in sequence SQ408, the control unit 210 detects a vehicle from the shooting data identified by the identification information and tracks the movement of the detected vehicle. For example, the control unit 210 inputs the shooting data into a learned model. The learned model is a learned model that is learned with the shooting data as input data, the detection result of the vehicle included in the shooting data, and the tracking data indicating the movement of the detected vehicle as output data. The control unit 210 acquires the detection result and the tracking data output from the learned model. The process of sequence SQ407 is an example of a process in which the control unit 210 accepts the selection of a video via a screen. Also, the process of sequence SQ408 is an example of a process of detecting one or more objects included in the selected video. In the present embodiment, the object is mainly described by taking a vehicle as an example, but it may be other objects such as a person or an animal, etc. In addition, as another example of detecting a vehicle, the control unit 210 may use an image analysis technique to extract a vehicle from frame images that are temporally continuous, and output the result of tracking the extracted vehicle.

[0029] FIG. 8 is a diagram for explaining the details of the detection area selection region 530. When the button 810 is selected, the control unit 210 controls to display the pull-down menu 820. The operator selects a desired menu from the menus displayed in the pull-down menu 820. For example, when "edit preset" is selected, the control unit 210 controls to display the preset detection area superimposed on the still image acquired by the camera 120 in the display area 830. The preset detection area is a detection area set in advance for each camera.

[0030] FIG. 9 is a diagram for explaining details of the display area 830. In the example of FIG. 9, it is assumed that a detection area 910, a detection area 920, and a detection area 930 are preset as presets. The operator can operate the input unit 330 or the like to set a new detection area (detection area 940 in the example of FIG. 9) in the display area 830. Each of the plurality of detection areas is a polygonal area. The creation of the polygon is performed by an operation such as clicking on the position corresponding to the vertex with a mouse and double-clicking at the last position. In this embodiment, instead of the detection area (two-dimensional area), a detection line (one-dimensional line segment) may be adopted and the same control may be performed. When adopting the detection line, two directions perpendicular to it may be automatically set as, for example, IN / OUT or the user may arbitrarily set them, and the objects passing through each direction may be separately counted. In that case, similar to the IN / OUT of the detection area described later, the control unit 210 may perform control related to the IN / OUT of this detection line.

[0031] When a new detection area is input in the display area 830 and the button 840 is selected, in sequence SQ409, the control unit 210 executes a process of overwriting and saving the newly input detection area in the selected preset. This process is an example of a process for receiving selection of a plurality of detection areas related to a video via the screen. When the button 840 is selected, the control unit 210 controls to display the pop-up window 850. In the pop-up window 850, the detection area input by the operator is displayed together with the preset detection areas. The operator can change the preset name or the like in the pop-up window 850. When the button 860 is selected, the control unit 210 stores the data in the storage unit 220 or the like as a new preset with the newly input detection area overwritten in the selected preset. Note that a plurality of presets may coexist, and after displaying them in a list, a desired one may be selectable from them.

[0032] Note that although the description has been given using the example of adding a new detection area to a preset, instead of editing the preset, the operator may set a plurality of new detection areas for the still image displayed in the display area 830 and operate to register them as, for example, a new preset. When receiving these operations, the control unit 210 registers the newly set plurality of detection areas in the storage unit 220 or the like as a preset.

[0033] As another example, the control unit 210 may cause the video acquired by the camera 120 to be played back in the display area 830 of the detection area selection area 530. Then, the control unit 210 may receive the selection of a plurality of detection areas on the played-back video. These processes are an example of a process of playing back a video on a screen and receiving the selection of a plurality of detection areas on the played-back video. With such a configuration, the operator can select, as a detection area, an area that the operator wants to confirm while observing the movement of the actual vehicle. Note that candidates corresponding to the detection area may be displayed by image analysis or the like, and a desired detection area may be selectable from among them. The candidates may be, for example, portions of an image corresponding to a crosswalk or a stop line.

[0034] FIG. 10 is a diagram for explaining the details of the analysis area 540. The analysis area 540 includes a button 1010 and a button 1020. The button 1010 is a button selected when instructing the start of counting vehicles or the like included in the captured data. The button 1020 is a button selected when instructing the start of generating an analysis video based on the captured data.

[0035] When detecting that the button 1010 has been selected, in the sequence SQ410, the control unit 210 analyzes the movement of the vehicle in the plurality of detection areas based on the captured data, the result of the detection in the sequence SQ408, and the plurality of selected detection areas. The movement related to the detection area includes, for example, movements such as which detection area the vehicle has moved from and to among the plurality of detection areas. For example, the control unit 210 inputs the captured data, the result of vehicle detection, and the selected plurality of detection areas into the learned model. The learned model is a learned model that has been learned with the captured data, the result of vehicle detection, and the selected plurality of detection areas as input data, and the analysis result regarding the movement of the vehicle in the plurality of detection areas as output data. The control unit 210 acquires the analysis result output from the learned model. Here, as the analysis result, information on which detection area the vehicle has moved from to which detection area, the identification result of the vehicle type, the identification result of the vehicle type included in the captured data at predetermined time intervals, the date and time information of the area where the detected vehicle entered, the date and time information of the area where the vehicle exited, information indicating the size of the detected vehicle (for example, information on the size of the bounding box described later), etc. are included. In addition, as another example of the analysis, the control unit 210 may obtain the above-described analysis result by using an image analysis technique.

[0036] When it is detected that the button 1020 in FIG. 10 has been selected, the control unit 210 generates an analysis video based on the captured data and controls so that the analysis video is displayed in a predetermined area of the traffic volume survey screen 500. FIG. 11 is a diagram showing an example of the analysis video. In the analysis video, the vehicle detected in the sequence SQ408 and the minimum bounding rectangle indicating that the vehicle has been detected (for example, the bounding box 1120 in FIG. 11) are included. Further, in the analysis video, an object (the object 1130 indicated by a circle in the example of FIG. 11) indicating the center point of the detected vehicle is included. When the center point of this vehicle enters the detection area, the control unit 210 analyzes that the vehicle has entered the detection area, and when the center point of this vehicle exits the detection area, the control unit 210 analyzes that the vehicle has exited the detection area.

[0037] FIG. 12 is a diagram showing an example of the analysis area 540. Information indicating the progress of processing is displayed in the analysis area 540 under the control of the control unit 210. Information 1210 is information indicating the upload status of the captured data. Information 1210 changes from 0% to 100%. Information 1220 is information indicating the progress of the vehicle detection process (the process of sequence SQ408) in the video shown by the captured data. Information 1220 changes from 0% to 100%. Information 1230 is information indicating the progress of the vehicle counting process (a part of the process of sequence SQ410) in the video shown by the captured data. Information 1230 changes from 0% to 100%. Information 1240 is information indicating the creation status of the analyzed video. Information 1240 changes from 0% to 100%.

[0038] In sequence SQ411 of FIG. 4, the control unit 210 transmits the analysis result to the client device 110. When receiving the analysis result from the server device 100, in sequence SQ412, the control unit 310 causes the output unit 340 or the like to display a traffic volume survey screen in which the analysis result is reflected in the result display area 550.

[0039] FIG. 13 is a diagram (part 1) showing the analysis result displayed in the result display area 550. In FIG. 13, a summary of the analysis result is displayed. In the result display area 550 of FIG. 13, for each piece of information on which detection area a vehicle has moved from (From) to which detection area (To), the identification result of the vehicle type and the like are displayed. One of the features of Embodiment 1 is that when a plurality of detection areas are set here, the process of selecting two of them and setting them as From / To is automatically performed for all combination patterns. Note that the combination of From / To may be arbitrarily set by the user. Also, for each combination of From / To, a route name may be automatically set or arbitrarily set by the user. For example, the first row of FIG. 13 shows that there are 4 trucks, 0 buses, 12 cars, and 2 motorcycles, for a total of 18 vehicles that have moved from detection area 0 to detection area 1. By checking such analysis results, the operator can immediately grasp which detection area the vehicle included in the video has moved from and to among the plurality of detection areas included in the video, the type of the moved vehicle, the total, etc. In FIG. 13, the number of objects that have moved from the detection area (From) to the detection area (To) is displayed in association with the information indicating the detection area (From) and the detection area (To).

[0040] FIG. 14 is a diagram (part 2) showing the analysis results displayed in the result display area 550. In FIG. 14, the analysis results for each time are displayed. Each column shows the results of the vehicle types for each of the paths in FIG. 13. For example, the column "0-1_TRUCK" is the count of the number of vehicles of the vehicle type TRUCK that have passed through the path From: 0 / To: 1. In the result display area 550 of FIG. 14, the analysis results for each time are displayed. In the example of FIG. 14, the analysis results for each minute are displayed, but the operator can specify via the screen for which interval (e.g., every 2 minutes, every 5 minutes, etc.) the analysis results are to be displayed.

[0041] FIG. 15 is a diagram (part 3) showing the analysis results displayed in the result display area 550. In FIG. 15, the detection logs of the vehicles detected from the captured data are displayed. The detection logs include a time stamp (date and time information when the event occurred), an ID for identifying the detected vehicle, the type of the vehicle (such as the vehicle type of truck, bus, car, motorcycle, etc.), information on whether it entered the area or exited the area (In, Out), the area name for identifying the detection area, etc. Note that an event is, for example, entering the detection area or exiting the detection area.

[0042] Here, the control unit 210 may calculate the time the vehicle stayed in the detection area based on the timestamps and the information on whether the vehicle entered or exited the area included in the detection log, and include it in the detection log. By doing so, the analysis result will include information indicating how long one or more vehicles stayed in a plurality of detection areas. The control unit 210 may extract vehicles that stayed in the detection area for a time equal to or longer than a threshold value. Then, when there are vehicles that stayed in the detection area for a time equal to or longer than the threshold value, the control unit 210 may highlight the information of these vehicles.

[0043] According to the process of the first embodiment, it is possible to grasp from which detection area to which detection area the object included in the video has moved with respect to the plurality of detection areas included in the video. Also, by executing the process of detecting the object included in the video before the detection area is selected, it is possible to shorten the time from when the detection area is selected until the analysis process is executed and the analysis result is output. That is,

[0044] (Modification Example 1) A modification example 1 of the first embodiment will be described. In modification example 1, other examples of a part of the process of the first embodiment will be described. That is, modification example 1 is included in the first embodiment and is not a different embodiment from the first embodiment.

[0045] The control unit 210 of Modification 1 (hereinafter simply referred to as the control unit 210) controls to display a video such as that shown in the video 1600 in the detection area selection area 530. The video displayed in the detection area selection area 530 is the video selected in the video selection area 520. The control unit 210 controls to superimpose and display on the video the trajectory information indicating the movement of one or more vehicles detected in the process of the sequence SQ408. FIG. 16 is a diagram showing an example of a video on which the trajectory information is displayed. In the example of FIG. 16, four pieces of trajectory information (trajectories 1 to 4) are displayed. Here, each trajectory corresponds to the movement of a vehicle from a certain lane / lane to a different lane / lane, for example, at an intersection. From the viewpoint of usability, those that can be regarded as the same trajectory are preferably displayed as one trajectory together. At this time, the more the number of those regarded as the same trajectory, for example, the wider the width of the trajectory or the more prominent the color of the trajectory, the more the usability is improved. Note that the control unit 210 may control to display the trajectory information so as to wrap around below the vehicle at the location where the vehicle and the trajectory information intersect. Then, when the user selects a trajectory from among the candidates of trajectories 1 to 4 (multiple selections are possible), it is set as a criterion for analysis, and for each trajectory, the number of vehicles that can be regarded as having moved along the trajectory (having made the movement corresponding to the trajectory) is counted and output. When a certain trajectory is selected, both the forward and reverse directions of travel on that trajectory are automatically set, the number of vehicles is automatically divided and counted for each, and they are displayed so as to be distinguishable from each other. Or, after selecting a trajectory, the direction of travel may be further selectable, or the candidates of the trajectories may be divided and displayed for each direction of travel and then made selectable, and the trajectory and the direction of travel may be set individually. As a form of outputting the counted number of vehicles, for example, in FIG. 13, the image is such that From / To is replaced with the trajectory name or the direction of travel. Or, From / To and the trajectory name or the direction of travel may be written together. At this time, From / To may be automatically set, for example, by video analysis or the like, to an area near the stop line or near the crosswalk of the lane / lane that intersects corresponding to the selected trajectory or the direction of travel. Note that in Embodiment 1, by using the opposite method to this, From / To and the trajectory name or the direction of travel may be written together in FIG. 13.

[0046] Further, the control unit 210 may automatically select a plurality of detection areas based on the trajectory information. By such processing, for example, the control unit 210 can automatically set detection areas at the entrance and exit of a location where there is relatively a large amount of trajectory information of the vehicle.

[0047] Also, the control unit 210 may control to superimpose and display information indicating candidates for a plurality of detection areas on the video based on the trajectory information. By such processing, the user can select a plurality of detection areas while confirming the trajectory information of the vehicle and the information of the plurality of detection areas suggested as candidates.

[0048] Also, the control unit 210 may display a plurality of trajectories along which a plurality of objects have moved in the video, select a first trajectory to be used as a detection criterion from among the plurality of trajectories, count the number of objects that have moved along the first trajectory, and display the number of objects in association with information indicating the first trajectory. The control unit 210 receives an instruction from the user for selecting the first trajectory.

[0049] Also, the control unit 210 may display a plurality of trajectories along which a plurality of objects have moved in the video, select a first trajectory to be used as a detection criterion from among the plurality of trajectories, count the number of objects that have moved along the first trajectory, and display the number of objects in association with information indicating the first trajectory. The control unit 210 receives an instruction from the user for selecting the first trajectory.

[0050] Furthermore, it may be provided in each of the aspects described below.

[0051] (1) An information processing system for analyzing a video, which sets a plurality of detection areas in the video, selects a first detection area and a second detection area from among the plurality of detection areas, counts the number of objects that have moved from the first detection area to the second detection area, and displays the number of objects in association with information indicating the first detection area and the second detection area.

[0052] (2) The information processing system according to (1) above, which receives an instruction from a user for setting the detection area.

[0053] (3) The information processing system according to (1) above, which automatically selects the first detection area and the second detection area.

[0054] (4) The information processing system according to (1) above, which receives an instruction from a user for selecting the first detection area and the second detection area.

[0055] (5) The information processing system according to (2) above, which receives the instruction while playing the video.

[0056] (6) The information processing system according to (5) above, which detects an object included in the video and displays information indicating that the object has been detected together with the object when playing the video.

[0057] (7) The information processing system according to (1) above, which receives an instruction from a user for selecting the video.

[0058] (8) The information processing system according to (7) above, which receives an instruction from a user for selecting a photographing device and enables the selection of the video from among a plurality of videos photographed by the photographing device.

[0059] (9) In the information processing system according to (1) above, an information processing system that displays information indicating the staying time of the object within the detection area.

[0060] (10) In the information processing system according to (9) above, an information processing system that highlights the information when the staying time is equal to or greater than a threshold value.

[0061] (11) In the information processing system according to any one of (1) to (10) above, an information processing system, wherein the object is a vehicle.

[0062] (12) An information processing system for analyzing a video, the system setting a plurality of detection lines in the video, selecting a first detection line and a second detection line from among the plurality of detection lines, counting the number of objects that have moved from the first detection line to the second detection line, and displaying the number of objects in association with information indicating the first detection line and the second detection line.

[0063] (13) An information processing system for analyzing a video, the system displaying a plurality of trajectories along which a plurality of objects have moved in the video, selecting a first trajectory to be used as a detection criterion from among the plurality of trajectories, counting the number of objects that have moved along the first trajectory, and displaying the number of objects in association with information indicating the first trajectory.

[0064] (14) An information processing method executed by an information processing system for analyzing a video, the method setting a plurality of detection areas in the video, selecting a first detection area and a second detection area from among the plurality of detection areas, counting the number of objects that have moved from the first detection area to the second detection area, and displaying the number of objects in association with information indicating the first detection area and the second detection area.

[0065] (15) A program for causing a computer to function as the information processing system according to any one of (1) to (13) above. Of course, this is not all-inclusive.

[0066] For example, part or all of the processing of the server device 100 described above may be executed by the camera 120. The above-described embodiments and modifications may be arbitrarily combined and implemented.

[0067] Finally, although various embodiments of the present invention have been described, these are presented as examples and are not intended to limit the scope of the invention. The novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. The embodiments and their modifications are included in the scope and gist of the invention and are included in the invention described in the claims and the equivalent scope thereof.

Explanation of Reference Numerals

[0068] 100: Server device 110: Client device 120: Camera 150: Network 210: Control unit 220: Storage unit 230: Communication unit 310: Control unit 320: Storage unit 330: Input unit 340: Output unit 350: Communication unit 500: Traffic survey screen 510: Camera selection area 520: Video selection area 530: Detection area selection area 540: Analysis area 550: Result display area 610: Button 620: Button 630: Pull-down menu 640: Pop-up window 710: Video information list 720: Upload button 730: Pull-down menu 740: Pop-up window 810: Button 820: Pull-down menu 830: Display area 840: Button 850: Pop-up window 860: Button 910: Detection area 920: Detection area 930: Detection area 940: Detection area 1000: Information processing system 1010: Button 1020: Button 1120: Bounding box 1130: Object 1210: Information 1220: Information 1230: Information 1240: Information

Claims

1. An information processing system for analyzing a video, comprising: setting a plurality of detection areas in the video; selecting a first detection area and a second detection area from the plurality of detection areas; counting the number of objects that have moved from the first detection area to the second detection area; displaying the number of objects in association with information indicating the first detection area and the second detection area. An information processing system.

2. The information processing system according to claim 1, further comprising: receiving an instruction from a user for setting the detection area. An information processing system.

3. The information processing system according to claim 1, further comprising: automatically selecting the first detection area and the second detection area. An information processing system.

4. The information processing system according to claim 1, further comprising: receiving an instruction from a user for selecting the first detection area and the second detection area. An information processing system.

5. The information processing system according to claim 2, further comprising: receiving the instruction while playing the video. An information processing system.

6. The information processing system according to claim 5, further comprising: detecting an object included in the video; displaying information indicating that the object has been detected together with the object when playing the video. An information processing system.

7. The information processing system according to claim 1, further comprising: receiving an instruction from a user for selecting the video. An information processing system.

8. The information processing system according to claim 7, further comprising: receiving an instruction from a user for selecting a photographing device; enabling selection of the video from a plurality of videos photographed by the photographing device. An information processing system.

9. The information processing system according to claim 1, further comprising: displaying information indicating a stay time of the object within the detection area. An information processing system.

10. The information processing system according to claim 9, further comprising: highlighting the information when the stay time is equal to or greater than a threshold value. An information processing system.

11. The information processing system according to claim 1, wherein the object is a vehicle. An information processing system.

12. An information processing system for analyzing a video, comprising: setting a plurality of detection lines in the video; selecting a first detection line and a second detection line from the plurality of detection lines; Count the number of objects that have moved from the first detection line to the second detection line, associate the number of the objects with information indicating the first detection line and the second detection line, and display the result, An information processing system.

13. An information processing system for analyzing a video, display a plurality of trajectories along which a plurality of objects have moved in the video, select a first trajectory to be used as a detection criterion from among the plurality of trajectories, count the number of objects that have moved along the first trajectory, associate the number of the objects with information indicating the first trajectory, and display the result, An information processing system.

14. An information processing method executed by an information processing system for analyzing a video, set a plurality of detection areas in the video, select a first detection area and a second detection area from among the plurality of detection areas, count the number of objects that have moved from the first detection area to the second detection area, associate the number of the objects with information indicating the first detection area and the second detection area, and display the result, An information processing method.

15. A program, for causing a computer to function as the information processing system according to any one of claims 1 to 13. ​

Images