Information processing method and information processing system

Abstract

For a mobile object to which a regulation is applied, it is possible to determine whether the mobile object is not behaving in a predetermined manner based on the regulation, depending on whether the regulation is appropriate based on the direction of travel. [Solution] An information processing method in which a computer pre-sets a detection range corresponding to the restricted position of the moving body TG shown in an image generated by capturing an image of the movement path of the moving body TG using an imaging unit 21, and a reference direction associated with the detection range, identifies the position of the moving body TG shown in the image, identifies the behavior of the moving body TG shown in the video based on a video of consecutive images in chronological order, identifies the direction of travel of the moving body TG shown in the video based on the video, and if the identified direction of travel of the moving body TG matches the reference direction, and if at least a part of the moving body TG has not exhibited a specified behavior within the detection range since entering the detection range, determines that a violation event has occurred.

Description

[Technical Field]

[0001] The present invention relates to an information processing method and an information processing system. [Background technology]

[0002] BACKGROUND ART Conventionally, there is known a technique for determining whether or not a vehicle has stopped appropriately at an intersection based on an image generated by an imaging unit capturing an image of the intersection (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2004-348469 Summary of the Invention [Problem to be solved by the invention]

[0004] Meanwhile, vehicles used within the premises of a factory, warehouse, or the like where cargo is transported, or people moving within the premises, may be subject to restrictions such as stop signs or slow-down orders based on signs posted along the routes within the premises. However, unlike general roads, routes within the premises may be designed for two-way movement. Therefore, a certain restriction may be applied to one direction of travel but not to the other. In such cases, the method disclosed in Patent Document 1 may not be able to determine whether or not a restriction should be applied depending on the direction of travel. [Means for solving the problem]

[0005] An information processing method for achieving the above-mentioned object is characterized in that a computer pre-sets a detection range corresponding to the restricted position of the moving body shown in an image generated by capturing an image of the moving body's moving path using an imaging unit installed at a predetermined position capable of capturing the image of the moving body's moving path, and a reference direction corresponding to the detection range, identifies the position of the moving body shown in the image, identifies the behavior of the moving body shown in the video based on a video in which the images are consecutive in time series, identifies the direction of travel of the moving body shown in the video based on the video, and if the identified direction of travel of the moving body matches the reference direction, and if at least a part of the moving body has not performed a predetermined behavior within the detection range since entering the detection range, determines that a violation event has occurred.

[0006] With this configuration, it is possible to determine whether a moving body to which a regulation applies is not behaving in a predetermined manner based on the regulation, depending on whether the regulation is appropriate based on the direction of travel.

[0007] In the information processing method for achieving the above object, the predetermined behavior may include the moving object stopping temporarily or moving slowly. According to this configuration, it is possible to determine whether or not a predetermined behavior corresponding to stopping or slowing down has been performed.

[0008] In an information processing method that achieves the above-mentioned object, if the time during which the speed of the moving body within the detection range remains below a speed threshold is less than a predetermined time, it is preferable to determine that the moving body is not stopping or slowing down.

[0009] In the information processing method for achieving the above object, it is preferable that, when the speed of the moving object within the detection range does not become equal to or less than a speed threshold, it is determined that the moving object is not stopping or moving slowly.

[0010] With this configuration, it is possible to determine whether a moving object is not behaving in a predetermined manner based on regulations, based on simple indicators such as speed and the time the moving object has been within the detection range. In an information processing method that achieves the above-mentioned objective, when it is determined that the violation event has occurred, a violation video may be generated by extracting a portion of the video that includes the image of the scene where the violation event occurred.

[0011] According to this configuration, a violation video showing a scene in which a regulation is violated can be generated. In the information processing method for achieving the above object, when it is determined that the violation event has occurred, determination result information indicating a determination result corresponding to the violation event may be generated.

[0012] According to this configuration, by referring to the determination result information, detailed information about the violation event can be grasped for each event. In an information processing method that achieves the above-mentioned objective, the mobile body related to the violation event may be identified based on identification information corresponding to the mobile body, and the judgment result information may be generated by further including mobile body identification information that can identify the identified mobile body.

[0013] According to this configuration, by referring to the determination result information, it is possible to refer to information about a moving object that has violated a regulation. In an information processing method that achieves the above-mentioned object, the judgment result information may include imaging unit identification information that can identify the imaging unit, and a confirmation screen may be generated that displays the occurrence status of the violation event for each imaging unit on the map image based on the judgment result information and a map image that shows the predetermined position where the imaging unit corresponding to the imaging unit identification information is installed and the movement route.

[0014] According to this configuration, the location where a violation of a regulation occurred on the travel route can be presented in an intuitive and easy-to-understand manner. In the information processing method for achieving the above object, when it is determined that the violation event has occurred, the mobile body may be notified that the violation event has occurred.

[0015] According to this configuration, it is possible to warn a moving object that is present within or near the detection range by notifying the moving object that a violation event has occurred. In the information processing method for achieving the above object, it may be determined that a compliance event has occurred when the moving object is performing a predetermined behavior within the detection range.

[0016] With this configuration, it is possible to determine whether a mobile object to which a regulation is applied has behaved in a predetermined manner in accordance with the regulation, depending on whether the regulation is appropriate based on the direction of travel. In an information processing method that achieves the above-mentioned object, when it is determined that the compliance event has occurred, a compliance video may be generated by extracting a portion of the video that includes the image of the scene where the compliance event occurred.

[0017] With this configuration, it is possible to determine whether a mobile object to which a regulation applies complies with the regulation, and to generate a compliance video showing a scene in which the regulation is complied with. In the information processing method for achieving the above object, judgment result information indicating a judgment result corresponding to the compliance event may be generated.

[0018] According to this configuration, by referring to the determination result information, detailed information about the compliance event shown in the compliance video can be grasped for each event. In an information processing method that achieves the above-mentioned object, the preset image used to specify the detection range includes a frame image that is one size larger than the image and the image placed within the frame image, and the detection range may be determined based on a user's operation on an operation area set within the frame image.

[0019] With this configuration, the detection range can be selected up to the edge of the image. In an information processing method for achieving the above object, a plurality of detection ranges and a reference direction corresponding to each of the plurality of detection ranges may be determined on the movement route, and it may be determined whether the violation event has occurred for each of the plurality of detection ranges.

[0020] According to this configuration, it is possible to make determinations regarding a plurality of detection ranges based on an image generated by one imaging unit. In the information processing method for achieving the above object, the at least part of the moving object may be a front end of the moving object in the traveling direction on the image.

[0021] In an information processing method that achieves the above-mentioned object, when the images that make up the video are input, the position of the moving body is identified based on the information output by inputting each of the images that make up the video into a trained model that outputs the position of the moving body using a bounding box, and at least a portion of the moving body may be the lower front end in the direction of travel of the bounding box that represents the moving body on the image.

[0022] With this configuration, the position of a moving object can be identified using a simple method that uses a trained model. In an information processing method that achieves the above-mentioned objective, the reference direction is a direction connecting the detection range and an entry range into which the moving body enters after passing through the detection range, and when the position of the identified moving body indicates that it has moved from the detection range to the entry range, it may be determined that the traveling direction and the reference direction match.

[0023] According to this configuration, the reference direction can be determined without relying on user operation, and a moving object whose traveling direction matches the reference direction can be identified as a target to which restrictions are to be applied.

[0024] In the information processing method for achieving the above object, the degree of the violation event may further be determined based on a movement pattern of the mobile object within the detection range. According to this configuration, by referring to the determination result information, it is possible to further refer to the degree of violation of the regulation by the mobile object that has violated the regulation.

[0025] An information processing system that achieves the above-mentioned objective is an information processing system comprising: an imaging unit provided at a predetermined position capable of imaging the movement path of a moving body; one or more management devices that store a video of a chronological series of images generated by the imaging unit imaging the movement path in a server device on the cloud; and an information processing device that performs various processes based on the video stored in the server device, wherein the information processing device comprises: a determination unit that determines a detection range corresponding to the restricted position of the moving body shown in the image generated by the imaging unit imaging the movement path and a reference direction corresponding to the detection range; a position identification unit that identifies the position of the moving body shown in the image; a behavior identification unit that identifies the behavior of the moving body shown in the video based on the video; a direction identification unit that identifies the direction of travel of the moving body shown in the video based on the video; and a judgment unit that determines that a violation event has occurred when the identified direction of travel of the moving body matches the reference direction and if at least a part of the moving body has not performed a predetermined behavior within the detection range since entering the detection range.

[0026] According to this configuration, the same effects as those of the above-described information processing method can be obtained. An information processing system that achieves the above-mentioned objective is an information processing system comprising: an imaging unit provided at a predetermined position capable of imaging the movement path of a moving body; one or more management devices that store in the information processing device a video of a chronological series of images generated by the imaging unit imaging the movement path; and an information processing device that performs various processes based on the video, wherein the information processing device comprises: a determination unit that determines a detection range corresponding to the restricted position of the moving body shown in the image generated by the imaging unit imaging the movement path and a reference direction corresponding to the detection range; a position identification unit that identifies the position of the moving body shown in the image; a behavior identification unit that identifies the behavior of the moving body shown in the video based on the video; a direction identification unit that identifies the direction of travel of the moving body shown in the video based on the video; and a determination unit that determines that a violation event has occurred when the identified direction of travel of the moving body matches the reference direction and if at least a part of the moving body has not performed a predetermined behavior within the detection range since entering the detection range.

[0027] According to this configuration, the same effects as those of the above-described information processing method can be obtained. [Effects of the Invention]

[0028] According to the present invention, for a mobile object to which a regulation is applied, it is possible to determine whether the mobile object is not behaving in a predetermined manner based on the regulation, depending on whether the regulation is appropriate based on the direction of travel. [Brief explanation of the drawings]

[0029] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an information processing system. [Figure 2] FIG. 2 is a diagram illustrating an example of the configuration of an information processing device. [Figure 3] FIG. 3 is a diagram used to explain the preset image. [Figure 4] FIG. 4 is a diagram used to explain the processing of the position specifying unit. [Figure 5]FIG. 5 is a diagram showing an example of the contents of the determination result information. [Figure 6] FIG. 6 is a diagram showing an example of the confirmation screen. [Figure 7] FIG. 7 is a flowchart illustrating an example of processing by the information processing device. [Figure 8] FIG. 8 is a flowchart illustrating an example of processing by the information processing device. [Figure 9] FIG. 9 is a flowchart illustrating an example of processing by the information processing device. [Figure 10] FIG. 10 is a diagram used to explain the approach range. [Figure 11] FIG. 11 is a diagram used to explain another example of the detection range. DETAILED DESCRIPTION OF THE INVENTION

[0030] <Embodiment> Hereinafter, a specific embodiment of the information processing system 1 will be described with reference to the drawings. [Overall configuration] 1, the information processing system 1 includes an information processing device 10, a management device 20, and a server device 30. The information processing device 10, the management device 20, and the server device 30 are connected to each other via a network NW so that they can send and receive information. The network NW includes, for example, the Internet, a wide area network (WAN), a local area network (LAN), a provider terminal, a wireless communication network, a wireless base station, a dedicated line, and the like. Note that the network NW may include a local network in part.

[0031] The management device 20 includes a plurality of imaging units 21. The management device 20 and the imaging units 21 are connected so as to be able to send and receive information. Hereinafter, it is assumed that the management device 20 and the imaging units 21 are directly connected. The imaging units 21 are provided at predetermined positions within the premises of a factory, warehouse, or the like where cargo is transported, where they can capture images of the travel path of a moving object TG, such as a moving forklift TG1 or a person TG2. Specifically, the imaging units 21 are provided at predetermined positions within the travel path where they can capture images of stop positions LN defined on the travel path. Furthermore, the plurality of imaging units 21 are provided at different positions, and each captures an image of a different stop position LN.

[0032] The information processing system 1 executes the processes from step ST1 to step ST6. Each step ST will be explained below. In procedure ST1, the management device 20 transmits the video MV generated by the imaging unit 21 to the server device 30 via the network NW. The video MV is a video in which images IMa are generated in chronological order by the imaging unit 21 capturing images of the travel path. The video MV is generated, for example, every predetermined time, and is transmitted to the server device 30 by the management device 20 as soon as it is generated. The predetermined time is, for example, several minutes to several tens of minutes. The video MV also includes information indicating the date and time when the video MV was captured and generated, and imaging unit identification information that can identify the imaging unit 21 that captured and generated the video MV.

[0033] The server device 30 is realized by a cloud server including a non-transitory storage medium such as a hard disk drive (HDD), flash memory, electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), or random access memory (RAM). The server device 30 stores the moving image MV received from the management device 20 via the network NW in a storage unit.

[0034] In step ST2, the information processing device 10 reads the moving image MV from the storage unit of the server device 30 via the network NW. In step ST3, the information processing device 10 presents a preset image IM1 that prompts the user of the management device 20 to determine a detection range AR and a reference direction DR based on the moving image MV acquired in step ST2 and the image IMa that constitutes the moving image MV. As shown in FIGS. 3 and 4, the detection range AR is a range corresponding to a stop position LN defined on the moving path, and is a range that is set to determine whether the moving object TG has stopped appropriately before the stop position LN. The reference direction DR is a direction associated with the detection range AR, and indicates the direction of travel on the moving path that is subject to restriction, i.e., stopping at the stop position LN. The stop position LN is an example of a restriction position.

[0035] As shown in FIG. 1 , in step ST4, the information processing device 10 acquires information indicating the detection range AR and information indicating the reference direction DR from the management device 20. In step ST5, the information processing device 10 determines, based on the acquired information, whether the moving object TG has stopped appropriately within the detection range AR defined on the moving path. If the information processing device 10 determines that the moving object TG has not stopped appropriately, it determines that a violation event EV1 has occurred and generates a violation video MV1 showing the scene where the violation event EV1 occurred. If the information processing device 10 determines that the moving object TG has stopped appropriately, it determines that a compliance event EV2 has occurred and generates a compliance video MV2 showing the scene where the compliance event EV2 occurred.

[0036] In step ST6, the information processing device 10 provides a screen prompting the user to check information related to the violation event EV1 and the compliance event EV2. Details of the various configurations and processes used by the information processing system 1 to realize the processes of steps ST1 to ST6 will be described below.

[0037] [Configuration of information processing device 10] 2, the information processing device 10 includes, for example, a communication unit 11, a control unit 100, and a storage unit 150. The communication unit 11 includes, for example, a communication circuit or a communication module, and transmits and receives various data to and from the management device 20, the server device 30, and other devices via the network NW. The communication unit 11 performs communication via wired or wireless communication. In the case of wireless communication, the communication mode of the communication unit 11 may be short-range wireless communication such as Wi-Fi (registered trademark) or Bluetooth (registered trademark).

[0038] The control unit 100 is realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device (not shown) that includes a non-transitory storage medium such as an HDD or flash memory provided in the storage unit 150.

[0039] The storage unit 150 may be realized by the various storage devices described above, or an EEPROM, a ROM, a RAM, etc. The storage unit 150 stores, for example, a trained model 151, a violation event video folder 152, a compliance event video folder 153, and judgment result information 154. These will be described in detail later.

[0040] The control unit 100 includes, for example, a determination unit 101, a type identification unit 102, a position identification unit 103, a behavior identification unit 104, a direction identification unit 105, a judgment unit 106, a video generation unit 107, a judgment result generation unit 108, and a presentation unit 109.

[0041] The determination unit 101 determines the detection range AR and the reference direction DR based on information indicating the detection range AR corresponding to the stop position LN captured by the imaging unit 21 and information indicating the reference direction DR. The type identification unit 102 identifies the type of moving object TG shown in the moving image MV. The position identification unit 103 identifies the position of the moving object TG in the image IMa. The behavior identification unit 104 identifies the behavior of the moving object TG shown in the moving image MV based on the moving image MV. The direction identification unit 105 identifies the traveling direction of the moving object TG based on the moving image MV. The determination unit 106 determines whether the moving object TG is appropriately stopped within the detection range AR defined on the movement path.

[0042] The video generation unit 107 generates a violation video MV1 including a scene in which the moving object TG does not stop properly within the detection range AR. The video generation unit 107 also generates a compliance video MV2 including a scene in which the moving object TG stops properly within the detection range AR. The judgment result generation unit 108 generates judgment result information 154 including a judgment result corresponding to the violation video MV1 and a judgment result corresponding to the compliance video MV2. The presentation unit 109 generates an image used to present the judgment result information 154 to the user, and presents the image to the user.

[0043] [Regarding the decision unit 101] The detailed processing of each unit will be described below. The determination unit 101 identifies a moving-object-free image IMa1 that does not include a moving object TG and includes a stopping position LN among the images IMa constituting the moving image MV read from the server device 30. This identification process may be performed by a user who has viewed the moving image MV, or may be performed by inputting the moving image MV to a trained model that, when input, outputs a moving-object-free image IMa1 that does not include a moving object TG in the moving image MV. Next, the determination unit 101 generates a preset image IM1 to be used in procedure ST3. The preset image IM1 is an image used as a screen that prompts the user of the management device 20 to determine the detection range AR and the reference direction DR. The determination unit 101 presents the generated preset image IM1 to the user of the management device 20. Specifically, the determination unit 101 makes the generated preset image IM1 available for reference by the management device 20 via the network NW.

[0044] 3, the preset image IM1 includes an area T1 indicating a moving object-free image IMa1. The user of the management device 20 uses the preset image IM1 displayed on the management device 20 to specify a detection range AR and a reference direction DR.

[0045] Specifically, the user of the management device 20 inputs a plurality of vertices P representing a detection range AR corresponding to the stop position LN shown in the moving object-free image IMa1 in the region T1. In this example, the detection range AR is represented by a rectangle having four vertices P, vertices P1 to P4. The user of the management device 20 also inputs a reference direction DR indicating the travel direction to which the stop position LN shown in the moving object-free image IMa1 applies. Specifically, the user of the management device 20 inputs a start point SP and an end point FP of the reference direction DR. In this example, the travel direction from the rear side to the front side of the stop position LN is represented as the reference direction DR. An operation region is provided in the region T1, extending to the periphery of the moving object-free image IMa1, and the vertices P, start point SP, and end point FP can be selected up to the edge of the moving object-free image IMa1. Specifically, a frame image IMw that is slightly larger than the moving object-free image IMa1 is displayed in the region T1, and the inside of the frame image IMw within the region T1 is set as the operation region.

[0046] Furthermore, the user of the management device 20 specifies a moving object TG to be determined by the information processing device 10. The option C1 for selecting a moving object TG indicates "all," which means that any moving object TG is to be determined, the option C2 indicates "forklift," the option C3 indicates "electric car," the option C4 indicates "person," the option C5 indicates "truck," the option C6 indicates "bus," and the option C7 indicates "car." The options for the moving object TG are merely examples and are not limited thereto. Furthermore, the options for the moving object TG may indicate some or all of the moving objects TG moving along the travel route. The process of specifying the moving object TG to be determined may be performed by using an image other than the preset image IM1. For example, a pop-up image displayed together with the preset image IM1 may have check boxes for selecting seven options C1 to C7 as options for the moving object TG to be determined. In this case, the user of the management device 20 specifies the moving object TG to be determined by operating the check boxes in the pop-up image.

[0047] In step ST4, the determination unit 101 acquires information indicating the coordinates of each vertex P of the detection range AR in the moving object-free image IMa1. The coordinates are, for example, in units of pixels, and are coordinates where the coordinates of the bottom left corner of the moving object-free image IMa1 are (0,0). The determination unit 101 determines, as the detection range AR, a pixel region within the range specified by the vertices P in the image IMa constituting the moving image MV. The determination unit 101 also acquires information indicating the coordinates of the start point SP and the end point FP of the reference direction DR in the moving object-free image IMa1. The determination unit 101 determines, as the reference direction DR, the direction specified by the start point SP and the end point FP in the image IMa constituting the moving image MV. Specifically, the determination unit 101 determines, as the reference direction DR, the angle of the line segment connecting the start point SP and the end point FP in the image IMa, which is the screen coordinate system of the imaging unit 21.

[0048] The determination unit 101 performs the same process as described above for each of the multiple imaging units 21 included in the management device 20, and determines a detection range AR and a reference direction DR corresponding to the stop position LN captured by the imaging unit 21. The determination unit 101 associates information indicating the pixel area of ​​the detection range AR and information indicating the angle of the reference direction DR with identification information of the imaging unit 21 corresponding to this information, and stores the information in the storage unit 150. Furthermore, in step ST4, the determination unit 101 acquires information indicating the moving object TG to be determined by the information processing device 10.

[0049] [Regarding the type identification unit 102] The type identification unit 102 identifies the type of moving object TG shown in the video MV. This identification process may be performed by a user who has viewed the video, or by inputting the video into a trained model that outputs the type of moving object TG when the video is input.

[0050] [Regarding the position identification unit 103] The position identification unit 103 identifies the position of the moving object TG shown in the image IMa constituting the video MV. The position identification unit 103 inputs the image IMa to the trained model 151. The trained model 151 is, for example, a trained model that, when the image IMa is input, outputs information indicating the position of the moving object TG on the image IMa. The position identification unit 103 inputs each of the images IMa constituting the video MV to the trained model 151, and obtains, for each image IMa, information indicating the position of the moving object TG shown in the image IMa.

[0051] FIG. 4 shows image IMa2, one of images IMa constituting the video MV, in which a person TG2 is moving from the rear to the front of the stop position LN. For ease of explanation, image IMa2 shows a bounding box BX indicating the position of person TG2 identified by the position identification unit 103. As shown in FIG. 4, when image IMa is input, for example, the trained model 151 identifies the area on image IMa in which the moving object TG exists by the bounding box BX. Therefore, the trained model 151 outputs the coordinates of the bounding box BX as the partial area in which the moving object TG is identified, as information indicating the position of the moving object TG. The coordinates of the bounding box BX are indicated, for example, by the coordinates of two diagonal points.

[0052] [About the behavior identification unit 104] The behavior identification unit 104 identifies the behavior of the moving object TG shown in the video MV based on the video MV. The behavior identification unit 104 identifies the behavior of the moving object TG based on, for example, a change over time in the position of the moving object TG identified by the position identification unit 103. Hereinafter, the behavior of the moving object TG is assumed to include the speed, maximum speed, minimum speed, acceleration, and the duration of time during which the speed of the moving object TG remains equal to or less than a speed threshold. Note that the behavior of the moving object TG is merely an example and is not limited to this, and may include some of these indicators or may include other indicators.

[0053] The behavior identification unit 104 identifies the speed of the moving object TG based on the video MV. First, the behavior identification unit 104 identifies the time required for at least a portion of the moving object TG to enter and exit the detection range AR based on the position of the moving object TG identified by the position identification unit 103. Hereinafter, "at least a portion of the moving object TG" refers to the position of the front end lower portion JP in the traveling direction of the bounding box BX representing the moving object TG on the image IMa. Therefore, if the front end lower portion JP is present within the detection range AR in the image IMa, the behavior identification unit 104 determines that the moving object TG has entered the detection range AR. Thereafter, if the front end lower portion JP is no longer present within the detection range AR in the image IMa, the behavior identification unit 104 determines that the moving object TG has exited the detection range AR. More specifically, the behavior identification unit 104, for example, identifies the time that has elapsed from when an image IMa is captured in which it is determined that the front lower end JP of the bounding box BX has entered the detection range AR as the moving object TG approaches the stopping position LN, to when an image IMa is captured in which it is determined that the front lower end JP of the bounding box BX has left the detection range AR, as the period that the moving object TG has been within the detection range AR since it entered the detection range AR.

[0054] Furthermore, the behavior identification unit 104 performs general image processing on the moving image MV for the identified period, thereby identifying the speed, maximum speed, minimum speed, acceleration, etc. of the moving object TG within the detection range AR.

[0055] Furthermore, the behavior identification unit 104 identifies, based on the video MV, the time during which the speed of the moving object TG in the detection range AR continues to be equal to or less than the speed threshold. The speed threshold is a speed corresponding to the regulation at the stop position LN. As described above, in this embodiment, the moving object TG is required to stop at the stop position LN. Therefore, in this embodiment, the speed threshold is assumed to be 0 [km / h]. That is, the behavior identification unit 104 identifies, based on the video MV, the time during which the moving object TG continues to be stopped in the detection range AR. The behavior identification unit 104 identifies the period during which the moving object TG is present within the detection range AR, for example, by processing similar to the processing described above. The behavior identification unit 104 performs general image processing on the video MV for the identified period to identify the speed of the moving object TG and identify the time during which the speed within the detection range AR continues to be equal to or less than the speed threshold.

[0056] [Regarding the direction identification unit 105] The direction specifying unit 105 specifies the traveling direction of the moving object TG shown in the moving image MV based on the moving image MV. The direction specifying unit 105 specifies the traveling direction of the moving object TG based on, for example, a change over time in the position of the moving object TG specified by the position specifying unit 103.

[0057] [Regarding the determination unit 106] The determination unit 106 determines whether the moving object TG is performing a predetermined behavior within the detection range AR. As described above, in this embodiment, the moving object TG is required to stop at the stop position LN. Therefore, the determination unit 106 determines whether the moving object TG is appropriately stopped within the detection range AR. Specifically, when the traveling direction of the moving object TG identified by the direction identification unit 105 at the time when the moving object TG enters the detection range AR matches the reference direction DR associated with the detection range AR, the determination unit 106 determines that the moving object TG is the moving object TG to be determined.

[0058] The reference direction DR is indicated by a line segment connecting the start point SP and the end point FP (i.e., the reference direction DR), and has a predetermined angular range. For example, the reference direction DR is a bisector of an angle within the predetermined angular range. Therefore, when the moving body TG enters the detection range AR, if the traveling direction of the moving body TG identified by the direction identification unit 105 is within a predetermined angular range centered on the reference direction DR, the determination unit 106 determines that the traveling direction of the moving body TG identified by the direction identification unit 105 matches the reference direction DR, and considers the moving body TG to be the moving body TG to be determined.

[0059] Then, based on the identification result of the behavior identification unit 104, if the time during which the speed of the moving object TG to be judged within the detection range AR continues to be equal to or less than the speed threshold is less than a predetermined time, the determination unit 106 determines that the moving object TG has behaved in a predetermined manner, that is, has not stopped temporarily, and that a violation event EV1 has occurred. Also, based on the identification result of the behavior identification unit 104, if the time during which the speed of the moving object TG to be judged within the detection range AR continues to be equal to or less than the speed threshold is equal to or more than a predetermined time, the determination unit 106 determines that the moving object TG has behaved in a predetermined manner, that is, has stopped temporarily, and that a compliance event EV2 has occurred.

[0060] Note that the determination unit 106 may determine that the moving object TG has not performed a predetermined behavior of stopping temporarily and that a violation event EV1 has occurred when the speed of the object to be determined, i.e., the moving object TG, within the detection range AR does not become equal to or less than the speed threshold based on the identification result of the behavior identification unit 104. Similarly, the determination unit 106 may determine that the moving object TG has performed a predetermined behavior of stopping temporarily and that a compliance event EV2 has occurred when the speed of the object to be determined, i.e., the moving object TG, within the detection range AR becomes equal to or less than the speed threshold based on the identification result of the behavior identification unit 104.

[0061] [About the video generation unit 107] When the determination unit 106 determines that a violation event EV1 has occurred, the video generation unit 107 cuts out a portion of the video MV including an image IMa of the scene where the violation event EV1 has occurred, and generates the cutout portion as a violation video MV1. When the determination unit 106 determines that a compliance event EV2 has occurred, the video generation unit 107 cuts out a portion of the video MV including an image IMa of the scene where the compliance event EV2 has occurred, and generates the cutout portion as a compliance video MV2. Here, the image IMa of the scene where the violation event EV1 has occurred is, for example, the image IMa of the image IMa constituting the video MV at the timing when the determination unit 106 determines that the violation event EV1 has occurred. In other words, it is the image IMa of the timing when the moving object TG does not properly pause in the detection range AR and exits the detection range AR. Furthermore, the image IMa of the screen where the compliance event EV2 has occurred is, for example, the image IMa of the image IMa constituting the video MV at the timing when the determination unit 106 determines that the compliance event EV2 has occurred. That is, this is an image IMa at a timing when the moving object TG is within the detection range AR as a result of the moving object TG appropriately making a temporary stop within the detection range AR.

[0062] Furthermore, the video generation unit 107 identifies the position of the moving object TG shown in the violation video MV1 based on the coordinates of the bounding box BX acquired by the position identification unit 103 for each image IMa constituting the violation video MV1. Then, the video generation unit 107 generates, as the violation video MV1, a video in which a line indicating the detection range AR and a bounding box BX indicating the identified position of the moving object TG are superimposed on each image IMa constituting the violation video MV1. That is, the violation video MV1 shows the moving object TG and the bounding box BX indicating the position of the moving object TG when the violation event EV1 occurred. The video generation unit 107 also performs the same process on the compliance video MV2. That is, the compliance video MV2 shows a line indicating the detection range AR and the moving object TG and the bounding box BX indicating the position of the moving object TG when the compliance event EV2 occurred.

[0063] Hereinafter, the video generation unit 107 will generate as a violation video MV1 a portion of the video MV from when the moving object TG, determined to have undergone a violation event EV1, enters the detection range AR until it exits the detection range AR. Also, the video generation unit 107 will generate as a compliance video MV2 a portion of the video MV from when the moving object TG, determined to have undergone a compliance event EV2, enters the detection range AR until it exits the detection range AR.

[0064] The video generation unit 107 stores the generated violation video MV1 in a violation event video folder 152 of the storage unit 150, while storing the generated compliance video MV2 in a compliance event video folder 153 of the storage unit 150. The violation event video folder 152 is an example of a first storage location, and the compliance event video folder 153 is an example of a second storage location.

[0065] [Regarding the judgment result generating unit 108 and the judgment result information 154] The judgment result generating unit 108 generates / updates judgment result information 154 indicating the judgment result corresponding to the violation video MV1 and the judgment result corresponding to the compliance video MV2.

[0066] 5, the determination result information 154 includes one or more records in which information indicating, for example, the date and time when the violation event EV1 or the compliance event EV2 occurred, the playback time of the violation video MV1 or the compliance video MV2 corresponding to that time, the event number, image capture unit identification information that can identify the image capture unit 21, the name of the moving object TG, moving object identification information that can identify the moving object TG, the determination type, the determination result, the determination reason, the entry time of the moving object TG into the detection range AR, the exit time of the moving object TG from the detection range AR, the average speed of the moving object TG, the maximum speed of the moving object TG, the minimum speed of the moving object TG, the duration of the speed being equal to or less than the speed threshold, and the speed threshold is associated with one another. The determination result information 154 is realized, for example, by a data file in CSV (Comma Separated Values) format.

[0067] Each time the judgment unit 106 judges that a violation event EV1 has occurred or that a compliance event EV2 has occurred, a record is generated and the generated record is added to the judgment result information 154, thereby generating / updating the judgment result information 154. In other words, each time the video generation unit 107 generates a violation video MV1 or a compliance video MV2, a record is generated and the judgment result information 154 is generated / updated.

[0068] The determination result generation unit 108 identifies the date and time when the violation event EV1 or the compliance event EV2 occurred for the moving image MV that is the subject of determination by the determination unit 106, based on information indicating the date and time included in the moving image MV. Specifically, the determination result generation unit 108 identifies the date and time a predetermined time before the moving object TG, for which the determination unit 106 has determined that the violation event EV1 or the compliance event EV2 has occurred, entered the detection range AR. The predetermined time is, for example, several seconds (specifically, approximately 3 seconds). The determination result generation unit 108 also identifies the playback duration of the moving image MV on the date and time when the violation event EV1 or the compliance event EV2 occurred.

[0069] Furthermore, the judgment result generating unit 108 identifies the event number assigned to the violation event EV1 or the compliance event EV2. The event number is, for example, a serial number that is common to the violation event EV1 and the compliance event EV2. Therefore, the judgment result generating unit 108 identifies the number next to the event number of the violation event EV1 or compliance event EV2 that occurred immediately before as the event number of the violation event EV1 or compliance event EV2 that occurred next. Note that the event number may be a serial number for each violation event EV1 and each compliance event EV2.

[0070] The determination result generation unit 108 identifies the imaging unit identification information included in the moving image MV for the moving image MV that is the subject of determination by the determination unit 106. The determination result generation unit 108 identifies the name of the moving object TG based on the identification result of the type identification unit 102. Based on the identification result of the type identification unit 102, the determination result generation unit 108 identifies one of a forklift, an electric car, a person, a truck, a bus, and a car as the name of the moving object TG.

[0071] The determination result generating unit 108 identifies moving object identification information based on the corresponding violation video MV1 or compliance video MV2. Here, each moving object TG is assigned with moving object identification information. Specifically, a vehicle such as a forklift TG1 is assigned moving object identification information indicated by a number, a barcode, or the like at a position on its top surface that can be imaged by the imaging unit 21. Furthermore, the person TG2 is assigned moving object identification information indicated by a number, a barcode, or the like at a position on a helmet or clothing worn by the person TG2 that can be imaged by the imaging unit 21. The determination result generating unit 108 identifies the moving object identification information of the moving object TG to be determined by performing general image processing on the violation video MV1 or the compliance video MV2. Note that the determination result generating unit 108 may also identify the moving object identification information of the occupant of the moving object TG or the person TG2 by performing general image recognition, such as face authentication, on an image IMa captured of the moving object TG.

[0072] The judgment result generation unit 108 identifies the judgment type based on the processing performed by the judgment unit 106. In this example, the judgment unit 106 makes a judgment regarding a temporary stop, and therefore the judgment type is all "temporary stop." The judgment result generation unit 108 identifies the judgment result based on the processing performed by the judgment unit 106. In this example, if the judgment unit 106 determines that a violation event EV1 has occurred, the judgment result is "NG," and if the judgment unit 106 determines that a compliance event EV2 has occurred, the judgment result is "OK." The judgment result generation unit 108 identifies the judgment reason based on the processing performed by the judgment unit 106. In this example, if the judgment unit 106 determines that a violation event EV1 has occurred, the judgment reason is "no stop," and if the judgment unit 106 determines that a compliance event EV2 has occurred, the judgment result is "stopped."

[0073] The determination result generation unit 108 determines the entry time based on the processing performed by the behavior identification unit 104. As described above, the behavior identification unit 104 determines, as the time that has elapsed since the image IMa in which it is determined that the front end lower JP of the bounding box BX has entered the detection range AR as the moving object TG approaches the stop position LN, is captured until the image IMa in which it is determined that the front end lower JP of the bounding box BX has exited the detection range AR, as the time that the front end lower JP has been present within the detection range AR since it entered the detection range AR. Therefore, the determination result generation unit 108 determines, as the entry time, the time at which the front end lower JP of the bounding box BX of the moving object TG to be determined entered the detection range AR, and determines, as the exit time, the time at which the front end lower JP of the bounding box BX of the moving object TG to be determined exits the detection range AR. The determination result generating unit 108 may specify, as the entry time, the playback time of the moving image MV at the entry time or the frame number of an image IMa that constitutes the moving image MV and indicates the timing of the entry time. The determination result generating unit 108 may specify, as the exit time, the playback time of the moving image MV at the exit time or the frame number of an image IMa that constitutes the moving image MV and indicates the timing of the exit time.

[0074] The determination result generation unit 108 identifies the average speed of the moving object TG based on the processing performed by the behavior identification unit 104. The determination result generation unit 108 also identifies the maximum speed of the moving object TG based on the processing performed by the behavior identification unit 104. The determination result generation unit 108 also identifies the minimum speed of the moving object TG based on the processing performed by the behavior identification unit 104. The determination result generation unit 108 also identifies the time during which the speed of the moving object TG within the detection range AR continues to be equal to or less than the speed threshold value based on the processing performed by the behavior identification unit 104.

[0075] The judgment result generating unit 108 generates / updates the judgment result information 154 by generating a record that associates the various pieces of information identified by the above-mentioned processing with each other for each corresponding violation video MV1 and compliance video MV2.

[0076] Furthermore, the video generation unit 107 may determine the title of the violation video MV1 corresponding to the violation event EV1 and the title of the compliance video MV2 corresponding to the compliance event EV2 based on the judgment result information 154 generated / updated by the judgment result generation unit 108. The video generation unit 107 generates the violation video MV1 and the compliance video MV2 by giving them a title such as "(event number)_(judgment name)_(moving object name)_(judgment result).mp4", for example.

[0077] [About presentation unit 109] The presentation unit 109 generates a confirmation screen IMb so as to present the determination result information 154 to the user in an easy-to-understand manner. The confirmation screen IMb is a screen that shows the occurrence status of the violation event EV1 for each imaging unit 21. The presentation unit 109 presents the generated confirmation screen IMb to the user of the management device 20. Specifically, the presentation unit 109 makes the generated confirmation screen IMb available for reference by the management device 20 via the network NW.

[0078] As shown in FIG. 6, the confirmation screen IMb includes a designation image IMc, a map image IMd, and a slider bar image IMe. The designation image IMc includes, for example, pull-down lists PD1 to PD4. The pull-down list PD1 is a pull-down list used to designate the start of a period of time desired to be confirmed among the records included in the determination result information 154. The pull-down list PD2 is a pull-down list used to designate the end of a period of time desired to be confirmed among the records included in the determination result information 154. The pull-down list PD3 is a pull-down list used to select a determination type of time desired to be confirmed among the records included in the determination result information 154. The pull-down list PD4 is a pull-down list used to select an imaging unit 21 of time desired to be confirmed among the records included in the determination result information 154.

[0079] The map image IMd is, for example, an image that schematically shows a premises or the like in which the imaging unit 21 is installed. Furthermore, icons ICN are shown at positions on the map image IMd that correspond to positions where the imaging unit 21 is installed on the travel route of the schematically shown premises or the like. In this example, seven imaging units 21, namely imaging units 21a to 21g, are installed in the premises or the like. Therefore, seven icons ICNa to ICNg are shown at corresponding positions on the map image IMd. The map image IMd is generated in advance for each premises to be determined and stored in the storage unit 150. Furthermore, of the icons ICN that represent the imaging units 21, a symbol image SY is shown superimposed on a specific icon ICN. The symbol image SY is, for example, an image that represents the occurrence status of a violation event EV1. Details of the specific icon ICN on which the symbol image SY is superimposed will be described later.

[0080] The slider bar image IMe includes, for example, a slider bar SB, a start point slider CS1, and an end point slider CS2. The slider bar SB represents the entire period specified by the pull-down lists PD1 and PD2. In this example, since a single day is specified by the pull-down lists PD1 and PD2, the slider bar SB represents the period from midnight to midnight. On the other hand, if multiple days are specified by the pull-down lists PD1 and PD2, the slider bar SB represents the multiple days. If several hours are specified by the pull-down lists PD1 and PD2, the slider bar SB represents the several hours. The start point slider CS1 and the end point slider CS2 are used to specify a specific period to be checked within the period indicated by the slider bar SB. In the initial state, the start point slider CS1 and the end point slider CS2 are located at both ends of the slider bar SB so as to specify the entire period of the pull-down lists PD1 and PD2.

[0081] The user of the management device 20 operates the pull-down lists PD1 to PD4 and the start point slider CS1 and end point slider CS2 of the slider bar SB based on the confirmation screen IMb presented by the presentation unit 109. The presentation unit 109 acquires information indicating the operation by the user of the management device 20.

[0082] The presentation unit 109 identifies a record to be displayed as the map image IMd from among the records included in the determination result information 154, based on information specified using the pull-down list PD and slider bar SB on the confirmation screen IMb. First, the presentation unit 109 searches the determination result information 154 from among the records included in the determination result information 154, using a date within the period specified by the pull-down lists PD1 to PD2 as a search key. The presentation unit 109 identifies and narrows down records associated with dates matching the specified dates. Next, the presentation unit 109 searches for records in the determination result information 154, using a date and time within the period specified by the start point slider CS1 and the end point slider CS2 as a search key. The presentation unit 109 identifies and narrows down records matching the date and time. Furthermore, the presentation unit 109 searches for the identified record of the determination result information 154, using a record of the determination type specified by the pull-down list PD3 and the target imaging unit 21 specified by the pull-down list PD4 as a search key. The presentation unit 109 identifies and narrows down the records that match the specified determination type and match the imaging unit identification information of the specified imaging unit 21.

[0083] The presentation unit 109, while referring to the identified record, identifies information indicating the image capture unit identification information and the judgment result from among the information included in the record. When the identified information indicating the judgment result indicates "NG," the presentation unit 109 generates a map image IMd and updates the confirmation screen IMb so as to display a symbol image SY superimposed on the icon ICN corresponding to the image capture unit identification information associated with the "NG" judgment result. Furthermore, the symbol image SY includes a number image indicating the number of identified records for which the information indicating the judgment result in the image capture unit 21 corresponding to the superimposed icon ICN was "NG." As shown in FIG. 6, in this example, the symbol image SY superimposed on the icon ICNa indicates that seven "NG" judgments were made during a specified period at the stop position LN captured by the image capture unit 21a corresponding to the icon ICNa.

[0084] The presenting unit 109 also changes the display mode of the symbol image SY based on the number of occurrences of the violation event EV1 indicated in the judgment result information 154. For example, the presenting unit 109 may enlarge, compared to its normal size, the symbol image SY superimposed on the icon ICN of the imaging unit 21 capturing an image of a regulated location where a large number of violation events EV1 occurred during a period designated by the pull-down lists PD1 to PD2. As shown in FIG. 6, in this example, the symbol image SY superimposed on the icon ICNb indicates that one "NG" judgment was made during the designated period at the stop position LN captured by the imaging unit 21b corresponding to the icon ICNb. Therefore, the number of "NG" judgments made by the imaging unit 21b is smaller than the number of "NG" judgments made by the imaging unit 21a. Accordingly, the presenting unit 109 generates a map image IMd and updates the confirmation screen IMb so that the symbol image SY superimposed on the icon ICNa is displayed larger than the symbol image SY superimposed on the icon ICNb.

[0085] [Operation of information processing device 10] Hereinafter, the details of the processing of the information processing device 10 in steps ST3 and ST4 will be described with reference to Fig. 7. The processing of the flowchart shown in Fig. 7 is executed as an initial setting, for example, when the information processing system 1 is introduced, when the management device 20 is installed on the premises or the like, when the angle of view of the imaging unit 21 is changed, when a new imaging unit 21 is installed, before the determination process is performed in the information processing device 10, etc.

[0086] First, the determination unit 101 presents a preset image IM1 to the user of the management device 20 (step S100). Next, the determination unit 101 determines whether the user of the management device 20 has performed an operation to specify a detection range AR and an operation to specify a reference direction DR (step S102). The determination unit 101 waits until the user of the management device 20 has performed an operation to specify a detection range AR and an operation to specify a reference direction DR (step S102; NO). When the determination unit 101 determines that the user of the management device 20 has performed an operation to specify a detection range AR and an operation to specify a reference direction DR (step S102; YES), the determination unit 101 determines the detection range AR based on the operation to specify the detection range AR, and determines the reference direction DR based on the operation to specify the reference direction DR (step S104). Specifically, the determination unit 101 determines, as the detection range AR, a pixel region within a range specified by vertices P of the image IMa constituting the moving image MV. The determination unit 101 also acquires information indicating the coordinates of the start point SP and the end point FP of the reference direction DR in the moving object-free image IMa1. The determination unit 101 determines the direction specified by the start point SP and the end point FP in the image IMa constituting the moving image MV as the reference direction DR.

[0087] The process of the information processing device 10 in step ST5 will be described in detail below with reference to Fig. 8. The process of the flowchart shown in Fig. 8 is executed for a moving image MV every time the management device 20 transmits the moving image MV to the server device 30, for example.

[0088] First, the type identification unit 102 reads and acquires a moving image MV from the storage unit of the server device 30 via the network NW (step S200). Next, the type identification unit 102 identifies the type of moving object TG shown in the moving image MV (step S202). Below, a case will be described where the moving image MV shows the movement of one moving object TG.

[0089] Next, for the moving image MV acquired in step S200, the position identifying unit 103 identifies the position of the moving object TG shown in the image IMa for each of the images IMa constituting the moving image MV (step S204). Specifically, the position identifying unit 103 inputs the image IMa into the trained model 151. The position identifying unit 103 inputs each of the images IMa constituting the moving image MV into the trained model 151, and acquires, for each image IMa, information indicating the position of the moving object TG shown in the image IMa.

[0090] Next, the behavior identification unit 104 identifies the behavior of the moving object TG shown in the moving image MV based on the moving image MV (step S206). Specifically, the behavior identification unit 104 identifies the speed, maximum speed, minimum speed, acceleration, and duration of time during which the speed of the moving object TG remains equal to or less than the speed threshold value based on the moving image MV.

[0091] Next, the direction identifying unit 105 identifies the traveling direction of the moving object TG shown in the moving image MV based on the moving image MV acquired in step S200 (step S208). Specifically, the direction identifying unit 105 identifies the traveling direction of the moving object TG based on the change over time in the position of the moving object TG identified by the position identifying unit 103.

[0092] Next, the determination unit 106 determines whether or not the traveling direction identified by the direction identification unit 105 matches the reference direction DR determined by the determination unit 101 (step S210). Specifically, if the traveling direction of the moving object TG identified by the direction identification unit 105 when the moving object TG enters the detection range AR is within a predetermined angle range centered on the reference direction DR, the determination unit 106 determines that the traveling direction of the moving object TG identified by the direction identification unit 105 matches the reference direction DR of the stop position LN shown in the moving image MV. If the determination unit 106 determines that the traveling direction does not match the reference direction DR (step S210; NO), the determination unit 106 determines that the moving object TG shown in the moving image MV is a moving object TG to which stopping at the stop position LN does not apply, and ends the series of processes without executing subsequent processes.

[0093] When the determination unit 106 determines that the traveling direction of the moving object TG identified by the direction identification unit 105 matches the reference direction DR (step S210; YES), the determination unit 106 determines whether a violation event has occurred (step S212). Specifically, the determination unit 106 determines whether the moving object TG is performing a predetermined behavior within the detection range AR. When the moving object TG is not performing a predetermined behavior within the detection range AR, the determination unit 106 determines that a violation event EV1 has occurred. More specifically, when the time during which the speed of the moving object TG within the detection range AR remains equal to or less than a speed threshold is less than a predetermined time based on the identification result of the behavior identification unit 104, the determination unit 106 determines that the moving object TG is not performing a predetermined behavior within the detection range AR and that a violation event EV1 has occurred. The "at least a part of the moving object TG" refers to the position of the front lower end JP in the traveling direction of the bounding box BX representing the moving object TG on the image IMa. On the other hand, when the direction of travel of the moving body TG identified by the direction identification unit 105 matches the reference direction DR of the stopping position LN shown in the video MV, and based on the identification result of the behavior identification unit 104, the time during which the speed of the moving body TG within the detection range AR remains below the speed threshold is longer than a predetermined time, the judgment unit 106 determines that the moving body TG is performing a predetermined behavior within the detection range AR, and therefore a violation event EV1 has not occurred (i.e., a compliance event EV2 has occurred).

[0094] When the determination unit 106 determines that a violation event EV1 has occurred (step S212; YES), the video generation unit 107 cuts out a portion of the video MV that includes an image IMa of the scene where the violation event EV1 has occurred, to generate a violation video MV1 (step S214). Specifically, the video generation unit 107 cuts out a portion of the video MV from when the moving object TG, in which it has been determined that the violation event EV1 has occurred, enters the detection range AR to when it exits the detection range AR. Next, the video generation unit 107 generates a violation video MV1 by superimposing a bounding box BX indicating the position of the moving object TG identified by the trained model 151 and a line indicating the detection range AR on each image IMa of the cut-out video. Next, the video generation unit 107 stores the generated violation video MV1 in the violation event video folder 152 of the storage unit 150 (step S216).

[0095] When the determination unit 106 determines that the compliance event EV2 has occurred (step S212; NO), the video generation unit 107 generates a compliance video MV2 by cutting out a portion of the video MV that includes an image IMa of the screen where the compliance event EV2 has occurred (step S218). Specifically, the video generation unit 107 cuts out a portion of the video MV from the time when the moving object TG, for which it has been determined that the compliance event EV2 has occurred, enters the detection range AR until it exits the detection range AR. Next, the video generation unit 107 generates the compliance video MV2 by superimposing a bounding box BX indicating the position of the moving object TG identified by the trained model 151 and a line indicating the detection range AR on each image IMa of the cut-out video. Next, the video generation unit 107 stores the generated compliance video MV2 in the compliance event video folder 153 of the storage unit 150 (step S220).

[0096] Next, the judgment result generating unit 108 generates / updates the judgment result information 154 indicating the judgment result corresponding to the violation video MV1 and the judgment result corresponding to the compliance video MV2 (step S222).

[0097] The process of the information processing device 10 in procedure ST6 will be described in detail below with reference to Fig. 9. The process of the flowchart shown in Fig. 9 is executed, for example, at the timing when the user of the management device 20 wishes to check the determination result information 154.

[0098] First, the presentation unit 109 generates a confirmation screen IMb and presents it to the user of the management device 20 (step S300). Next, the presentation unit 109 determines whether the user of the management device 20 has specified a period of time to be confirmed using the pull-down lists PD1 to PD2, the start point slider CS1, and the end point slider CS2 on the confirmation screen IMb (step S302). If the pull-down lists PD1 to PD2 and the slider bar SB have not been operated, the presentation unit 109 determines that no period has been specified (step S302; NO), and proceeds to step S306. If the pull-down lists PD1 to PD2, the start point slider CS1, and the end point slider CS2 have been operated to specify a period (step S302; YES), the presentation unit 109 acquires information indicating the period specified by the operation and identifies a record of the determination result information 154 that matches the period (step S304). The presentation unit 109 searches the determination result information 154 using the dates and times of the period specified by the pull-down lists PD1 to PD2 and the start point slider CS1 and end point slider CS2 as search keys. The presentation unit 109 identifies and narrows down records that match the dates of the period.

[0099] Next, the presentation unit 109 determines whether the user of the management device 20 has specified a judgment type to be confirmed using the pull-down list PD3 on the confirmation screen IMb (step S306). If the pull-down list PD3 has not been operated, the presentation unit 109 determines that no judgment type has been specified (step S306; NO), and proceeds to step S310. If the pull-down list PD3 has been operated to specify a judgment type (step S306; YES), the presentation unit 109 acquires information indicating the judgment type specified by the operation and identifies a record of the judgment result information 154 (step S308). The presentation unit 109 uses the judgment type specified by the pull-down list PD3 as a search key to search for the record of the judgment result information 154 specified in step S304, or searches the judgment result information 154 if no record has been specified yet. The presentation unit 109 identifies and narrows down the records associated with the judgment type.

[0100] Next, the presentation unit 109 determines whether a target imaging unit desired to be confirmed has been designated by the user of the management device 20 using the pull-down list PD4 on the confirmation screen IMb (step S310). If the pull-down list PD4 has not been operated, the presentation unit 109 determines that a target imaging unit has not been designated (step S310; NO), and proceeds to step S314. If the pull-down list PD4 has been operated to designate a target imaging unit (step S310; YES), the presentation unit 109 acquires information indicating the target imaging unit designated by the operation and identifies a record of the determination result information 154 (step S312). The presentation unit 109 searches for the record of the determination result information 154 identified in step S304 or step S308, using the target imaging unit designated by the pull-down list PD4 as a search key, or searches for the determination result information 154 if no record has yet been identified. The presentation unit 109 identifies and narrows down the records associated with the target imaging unit.

[0101] Next, if a period is acquired in step S302, the presentation unit 109 generates a map image IMd representing records for that period, or if a period is not acquired, generates records for the entire period (step S314). The presentation unit 109 updates the confirmation screen IMb including the map image IMd generated in step S314, and presents it to the user of the management device 20 (step S316), and then ends the series of processes.

[0102] [Effects of the embodiment] According to the above embodiment, the following effects can be obtained. (1) When the identified traveling direction of the moving object TG matches the reference direction DR and the moving object TG does not behave as specified within the detection range AR, the determination unit 106 determines that a violation event EV1 has occurred. This configuration can prevent the moving object TG from erroneously determining that a violation event has occurred when the moving object TG passes through a stop position LN beyond an intersection that does not constitute a traffic restriction for the moving object TG without making a temporary stop. That is, in cases where there is no need to apply traffic restrictions to the moving object TG, as described above, the traveling direction of the moving object TG does not match the reference direction DR, and the moving object TG can be excluded from the determination target. As a result, it can prevent the erroneous determination that an event has occurred. Furthermore, when the identified traveling direction of the moving object TG matches the reference direction DR, the determination unit 106 determines that a violation event EV1 has occurred if the time during which the speed of the moving object TG within the detection range AR remains equal to or less than the speed threshold is less than a predetermined time.

[0103] With this configuration, the determination unit 106 can determine that the moving body TG, to which a temporary stop is applied at the stop position LN, is not appropriately performing a temporary stop as a predetermined behavior within the detection range AR.

[0104] (2) When it is determined that a violation event EV1 has occurred, the video generation unit 107 cuts out a part of the video MV that includes an image IMa of the scene where the violation event EV1 has occurred, and generates a violation video MV1. With this configuration, the video generation unit 107 can generate a violation video MV1 that shows a scene where a regulation has been violated.

[0105] (3) When the travel direction of the identified moving object TG matches the reference direction DR and the moving object TG is performing a predetermined behavior within the detection range AR, the determination unit 106 determines that the compliance event EV2 has occurred. Specifically, when the travel direction of the identified moving object TG matches the reference direction DR and the speed of the moving object TG within the detection range AR continues to be equal to or less than the speed threshold for a predetermined time or more, the determination unit 106 determines that the compliance event EV2 has occurred. With this configuration, the determination unit 106 can determine that the moving object TG, to which the regulation applies, has appropriately performed a temporary stop as a predetermined behavior within the detection range AR.

[0106] (4) When it is determined that the compliance event EV2 has occurred, the video generator 107 cuts out a portion of the video MV that includes the image IMa of the scene where the violation event EV1 has occurred, and generates the compliance video MV2. With this configuration, the information processing device 10 can determine whether or not a regulation has been complied with for a moving object TG to which a regulation applies, and generate the compliance video MV2 that shows the scene where the regulation has been complied with.

[0107] (5) When an image IMa constituting a moving image MV is input, the position identification unit 103 inputs the image IMa into the trained model 151, which outputs the position of the moving object TG using a bounding box BX, and identifies the position of the moving object TG based on the information output by inputting the image IMa. In this case, at least a portion of the moving object TG is the lower front end JP in the traveling direction of the bounding box BX that represents the moving object TG on the image IMa.

[0108] According to this configuration, the position specifying unit 103 can specify the position of the moving object TG by a simple method using the trained model 151. (6) The preset image IM1 used to specify the detection range AR includes a frame image IMw that is slightly larger than the image IMa, and a moving-object-free image IMa1 that is placed within the frame image IMw. The determination unit 101 determines the detection range AR based on an operation performed by the user of the management device 20 on an operation area set within the frame image IMw.

[0109] According to this configuration, it is possible to select the vertex P, the start point SP, and the end point FP up to the edge position of the moving object-free image IMa1. (7) The judgment result generating unit 108 generates judgment result information 154 indicating the judgment result corresponding to the violation event EV1. The judgment result generating unit 108 also generates judgment result information 154 indicating the judgment result corresponding to the compliance event EV2. With this configuration, by referencing the judgment result information 154, detailed information about the violation event EV1 can be ascertained for each event. By referencing the judgment result information 154, detailed information about the compliance event EV2 can be ascertained for each event.

[0110] (8) The video generation unit 107 stores the violation event EV1 in the violation event video folder 152 and the compliance event EV2 in the compliance event video folder 153 of the memory unit 150 that stores the judgment result information 154. With this configuration, the violation video MV1 and the compliance video MV2 can be accessed more easily than when the violation video MV1 and the compliance video MV2 are mixed in the same storage location.

[0111] (9) The judgment result generation unit 108 identifies the moving object TG related to the violation event EV1 based on the moving object identification information attached to the moving object TG, and generates the judgment result information 154 by further including the identified moving object identification information. Also, the judgment result generation unit 108 identifies the moving object TG related to the compliance event EV2 based on the moving object identification information attached to the moving object TG, and generates the judgment result information 154 by further including the identified moving object identification information.

[0112] According to this configuration, the determination result generating unit 108 can refer to information about a mobile body TG that has violated regulations by referring to the determination result information 154. Furthermore, the determination result generating unit 108 can refer to information about a mobile body TG that has complied with regulations by referring to the determination result information 154.

[0113] (10) The determination result generating unit 108 generates determination result information 154 including imaging unit identification information that can identify the imaging unit 21. The presenting unit 109 generates a confirmation screen IMb based on a map image IMd that indicates a predetermined position where the imaging unit 21 corresponding to the imaging unit identification information is provided and the travel route, and the determination result information 154. With this configuration, the presenting unit 109 can intuitively and easily present the position where a violation of a regulation occurred or the position where compliance with a regulation occurred on the travel route.

[0114] The above-described embodiments may be modified as follows: The above-described embodiments and the following modifications may be combined with each other within the scope of technical compatibility. The information processing system 1 may include a plurality of management devices 20. In this case, the information processing device 10 executes the above-described process for each management device 20. Furthermore, the management device 20 may be connected to one imaging unit 21.

[0115] In the case where moving images MV are generated at intervals of several minutes to several tens of minutes as described above, when the moving image generating unit 107 generates a violation moving image MV1 or a compliance moving image MV2 from a part of the moving image MV, a violation event EV1 or a compliance event EV2 may occur in a gap in the moving image MV. In this case, the moving image generating unit 107 may generate the violation moving image MV1 and the compliance moving image MV2 using the moving image MV used for the judgment and a moving image MV that is chronologically consecutive to the moving image MV.

[0116] The information included in the determination result information 154 is an example and is not limited to this. The determination result information 154 may include information other than that described above, or may include some of the information described above.

[0117] A barcode is one example of a method for realizing the moving object identification information, and is not limited to this. The moving object identification information may be realized, for example, by RFID (Radio Frequency Identification) or a beacon. In this case, a receiving device for RFID or a beacon is provided near the imaging unit 21, and the receiving device acquires the moving object identification information by receiving the RFID or beacon attached to the moving object TG traveling near the stop position LN. The management device 20 transmits information in which the moving object identification information and the date and time when the moving object identification information was read to the server device 30. The determination result generation unit 108 identifies the moving object identification information of the moving object TG corresponding to the violation video MV1 or the moving object TG corresponding to the compliance video MV2 based on the information.

[0118] In the above description, the determination unit 106 performs the determination process based on the traveling direction and a predetermined angle range centered on the reference direction DR. However, this is not limiting. The determination unit 106 may determine whether a violation event EV1 or a compliance event EV2 has occurred only when the traveling direction of the moving object TG identified by the direction identification unit 105 matches the reference direction DR. Furthermore, the user of the management device 20 may specify the predetermined angle range.

[0119] Although the case where the management device 20 transmits the video MV to the server device 30 has been described, this is not limiting. The management device 20 may transmit the video MV to the information processing device 10. Furthermore, if the information processing device 10 has a storage unit that the server device 30 has, the information processing system 1 does not need to have the server device 30. The information processing device 10 stores the received video MV in the storage unit 150 or another storage unit that has the same function as the storage unit that the server device 30 has. Each unit included in the control unit 100 performs various processes based on the video MV received by the information processing device 10.

[0120] The video MV may be constantly transmitted by the management device 20 to the server device 30 and / or the information processing device 10. The processing of the flowchart in Fig. 8 may be executed at a frequency other than every time the management device 20 transmits a moving image MV to the server device 30 or the information processing device 10. The processing of the flowchart in Fig. 8 may be executed, for example, at predetermined time intervals (for example, once a day). In this case, the processing of the flowchart shown in Fig. 8 is repeatedly executed for each moving image MV newly acquired from the server device 30.

[0121] When a moving image MV includes a plurality of moving objects TG moving, the information processing device 10 executes the processes from step S202 to step S220 of the flowchart shown in FIG. 8 for each of the moving objects TG.

[0122] The pull-down list PD displayed on the confirmation screen IMb is an example and is not limited to this. For example, the pull-down list PD may be provided in a number corresponding to the number of selectable pieces of information included in the determination result information 154.

[0123] The confirmation screen IMb does not have to include the pull-down lists PD1 to PD2 and the slider bar SB. In this case, the presentation unit 109 omits the process of step S302 from the process of the flowchart shown in FIG.

[0124] Of the processes shown in FIG. 9, the processes of steps S302 to S304, steps S306 to S308, and steps S310 to S312 may be executed in any order. The reference direction DR may be determined by a method other than the operation of the user of the management device 20 inputting the start point SP and the end point FP. As shown in FIG. 10 , the moving object TG stops within the detection range AR, then exceeds the detection range AR, passes through the stopping position LN, and proceeds into the entry range ARa. The determination unit 101 identifies the entry range ARa by, for example, performing general image processing on the moving object-free image IMa1. Specifically, the imaging unit 21 captures an image of an intersection or the like on the moving object's route within the premises, and the determination unit 101 identifies the shape of the intersection shown in the moving object-free image IMa1 to identify the central area of ​​the intersection as the entry range ARa. In this case, the determination unit 101 identifies the direction connecting the detection range AR and the entry range ARa as the reference direction DR, regardless of the user's operation of the management device 20. The direction connecting the detection range AR and the entry range ARa is, for example, the direction connecting the center of the detection range AR and the center of the entry range ARa. In addition, when the determination unit 106 indicates that the position of the moving body TG identified by the position identification unit 103 has moved from the detection range AR to the entry range ARa, it determines that the direction of travel of the moving body TG matches the reference direction DR.

[0125] According to this configuration, the reference direction DR can be determined without relying on the operation of the user of the management device 20, and a moving object TG whose traveling direction matches the reference direction can be identified as a target to which restrictions are to be applied.

[0126] Instead of using the preset image IM1, the determination unit 101 may determine the detection range AR by performing general image processing on the moving-object-free image IMa1. Specifically, the imaging unit 21 captures an image of the movement path and the stop positions LN within the premises, and the determination unit 101 determines a predetermined area as the detection range AR by identifying the shape of the movement path shown in the moving-object-free image IMa1 and the position and direction of the stop positions LN attached to the movement path. In this case, the determination unit 101 determines the detection range AR without relying on the operation of the user of the management device 20.

[0127] Similarly to the detection range AR, the approach range ARa may be determined by the determination unit 101 in response to input of a plurality of vertices P representing the approach range ARa in the preset image IM1.

[0128] The detection range AR may be represented in a form other than a quadrangle having four vertices. As shown in FIG. 11, the moving object-free image IMa1 may show a portion of the entire detection range AR missing. In this case, the detection range AR cannot be represented by a quadrangle. Therefore, the detection range AR may be represented by a triangle having three vertices P or a polygon having four or more vertices P. The determination unit 101 determines, as the detection range AR, a pixel region within a range specified by the vertices P of the image IMa constituting the moving image MV. Similarly, the entry range ARa may be represented in a form other than a quadrangle having four vertices. That is, the entry range ARa may be represented by a triangle having three vertices P or a polygon having four or more vertices P.

[0129] If the moving object-free image IMa1 includes multiple stop positions LN, the detection range AR and the reference direction DR corresponding to the detection range AR may be specified for the multiple stop positions LN using the preset image IM1. The determination unit 101 determines the detection range AR and the reference direction DR based on the multiple detection ranges AR specified using the preset image IM1 and the information on the reference directions DR corresponding to each of the detection ranges AR. The determination unit 106 performs a determination process for each detection range AR. With this configuration, the determination unit 106 can make determinations regarding multiple stop positions LN based on the image IMa generated by one imaging unit 21.

[0130] The position identification unit 103 may identify the position of the moving object TG by a method other than the bounding box BX. For example, when the image IMa is input, the position identification unit 103 may identify the position of the moving object TG using a trained model that segments the entire area on the image IMa of the moving object TG. In this case, the position identification unit 103 inputs each of the images IMa that constitute the video MV into the trained model, and acquires information indicating the area of ​​the moving object TG shown in the image IMa for each image IMa. In this case, "at least a part of the moving object TG" refers to the position of the front end of the area indicating the moving object TG on the image IMa in the traveling direction.

[0131] Although the above description has been given of a case where the stop restriction imposed on the travel route is a stop position LN, this is not limited to this. A stop restriction may be indicated by a mode other than a stop position LN. Examples of other modes include a crosswalk imposed on the travel route, a no-entry area established on the travel route according to unique rules, and other areas that require a stop before entering. In this case, the determination unit 101 prompts the user of the management device 20 to set a detection range AR for these restrictions using a preset image IM1. For example, in the case of a crosswalk, by setting a detection range AR and a reference direction DR on both sides of the crosswalk before and after crossing, the determination unit 106 can prevent erroneous determination that a violation event has occurred even if the vehicle does not temporarily stop within the detection range AR at the end of the crossing.

[0132] The restriction imposed on the travel route may be a restriction other than a stop. Specifically, the restriction imposed on the travel route may be a slow-down restriction. In this case, the determination unit 101 prompts the user of the management device 20 to set a detection range AR to which the slow-down restriction is applied, using the preset image IM1. When the traveling direction of the moving object TG identified by the direction identification unit 105 matches the reference direction DR of the stop position LN shown in the video MV, and based on the identification result of the behavior identification unit 104, if the time period during which the speed of the moving object TG within the detection range AR remains equal to or less than the speed threshold is less than a predetermined time, the determination unit 106 determines that the moving object TG is not performing a predetermined behavior, i.e., slow-down, within the detection range AR and that a violation event EV1 has occurred. The speed threshold is a value indicating a number [km / h] when slow-down is required at a restriction location. Similarly, when the traveling direction of the moving object TG identified by the direction identification unit 105 matches the reference direction DR of the stop position LN shown in the video MV, and based on the identification result of the behavior identification unit 104, the time period during which the speed of the moving object TG within the detection range AR remains equal to or less than the speed threshold is equal to or longer than a predetermined time, the determination unit 106 determines that the moving object TG is exhibiting a predetermined behavior, i.e., slowing down, within the detection range AR, and therefore determines that a compliance event EV2 has occurred. In this case, the determination result generation unit 108 identifies the determination type as "slowing down" as information to be included in the determination result information 154. With this configuration, the determination unit 106 can determine that a moving object TG that is not slowing down is not exhibiting a predetermined behavior or has exhibited a predetermined behavior within the detection range AR. In this case, the location where slowing down is applied is an example of a restriction location.

[0133] The determination unit 106 may determine that the moving object TG has stopped temporarily or slowed down when the time during which the speed of the moving object TG within the detection range AR remains equal to or less than the speed threshold is a predetermined time or longer. In other words, the determination unit 106 may determine that the moving object TG has not stopped temporarily or slowed down when the time during which the speed of the moving object TG within the detection range AR remains equal to or less than the speed threshold is shorter than a predetermined time.

[0134] Furthermore, if the speed of the moving object TG when entering the detection range AR or the average speed of the moving object TG within the detection range AR is not equal to or less than the speed threshold, the determination unit 106 may determine that the moving object TG did not stop or slow down appropriately and determine that a violation event EV1 has occurred. Similarly, if the speed of the moving object TG when entering the detection range AR or the average speed of the moving object TG within the detection range AR is equal to or less than the speed threshold, the determination unit 106 determines that the moving object TG did stop or slow down appropriately and determine that a compliance event EV2 has occurred.

[0135] The determination unit 106 may further determine the degree of violation of the regulation in the violation event EV1 based on the movement of the moving object TG within the detection range AR. For example, the determination unit 106 may determine the degree of violation of the regulation as one of two levels, such as “bad” or “very bad,” depending on the degree of violation. In this case, “bad” indicates that the regulation is violated but the degree of violation is minor, while “very bad” indicates that the regulation is violated and the degree of violation is severe. For example, if the regulation is a stop sign, the determination unit 106 determines the degree of violation to be “very bad” the shorter the time the moving object TG's speed within the detection range AR is at 0 km / h. For example, if the regulation is a slow speed, the determination unit 106 determines the degree of violation to be “very bad” the faster the moving object TG's speed within the detection range AR is above a predetermined speed or the greater the acceleration of the moving object TG. The determination result generation unit 108 generates the determination result information 154 by further including information indicating the degree of violation in the determination result information 154. According to this configuration, the determination result generating unit 108 can refer to the determination result information 154 to further refer to the degree of violation of the mobile unit TG that violated the regulation.

[0136] The determination unit 106 may further determine the degree of compliance with the regulation in the compliance event EV2 based on the movement of the moving object TG within the detection range AR. For example, the determination unit 106 may determine the degree of compliance with the regulation using two levels, such as "good" and "excellent," depending on the degree of compliance. In this case, "excellent" indicates that the regulation is complied with but to a low degree, while "good" indicates that the regulation is complied with and to a high degree. For example, if the regulation is a stop sign, the determination unit 106 determines that the degree of compliance is "excellent" the longer the time the speed of the moving object TG within the detection range AR is at 0 km / h. For example, if the regulation is a slow-down sign, the determination unit 106 determines that the degree of compliance is "excellent" the slower the speed of the moving object TG within the detection range AR is than a predetermined speed or the smaller the acceleration of the moving object TG. The determination result generation unit 108 generates the determination result information 154 by further including information indicating the degree of compliance in the determination result information 154. According to this configuration, the determination result generating unit 108 can refer to the determination result information 154 to further refer to the degree of compliance of the mobile object TG that complies with the regulations.

[0137] The degree of violation and the degree of compliance may be expressed not only in two stages but also in three stages, or in four or more stages. Instead of (or in addition to) a configuration in which the size of the symbol image SY is changed depending on the number of times the violation event EV1 has occurred, the presentation unit 109 may flash the symbol image SY superimposed on the icon ICN of the imaging unit 21 that has captured an image of a regulated location where a large number of violation events EV1 have occurred.

[0138] The presentation unit 109 may change the display mode of the symbol image SY based on the number of times the compliance event EV2 has occurred instead of (or in addition to) the number of times the violation event EV1 has occurred. For example, the presentation unit 109 may enlarge, compared to a normal size, the symbol image SY to be superimposed on the icon ICN of the imaging unit 21 that has captured an image of a regulated location where a large number of compliance events EV2 have occurred during a period specified by the pull-down lists PD1 to PD2. Furthermore, the presentation unit 109 may blink the symbol image SY to be superimposed on the icon ICN of the imaging unit 21 that has captured an image of a regulated location where a large number of compliance events EV2 have occurred.

[0139] According to this configuration, the presenting unit 109 can present information indicating the number of violations or the degree of violation to the user viewing the confirmation screen IMb in an intuitive and easy-to-understand manner. The presentation unit 109 may change the display mode of the symbol image SY based on the degree of violation indicated in the determination result information 154. For example, on the confirmation screen IMb representing a record of the violation event EV1, the presentation unit 109 may make the symbol image SY superimposed on the icon ICN of the image capture unit 21 corresponding to the record larger than the normal size as the information indicating the degree of violation of the record is "extremely bad" or the proportion of "extremely bad" is higher. Furthermore, on the confirmation screen IMb representing a record of the violation event EV1, the presentation unit 109 may blink the symbol image SY superimposed on the icon ICN of the image capture unit 21 corresponding to the record as the information indicating the degree of violation of the record is "extremely bad" or the proportion of "extremely bad" is higher.

[0140] Although the above description has been given of a case where the control unit 100 includes the video generating unit 107, this is not limiting. The control unit 100 does not have to include the video generating unit 107. In this case, the information processing device 10 does not generate the violation video MV1 and the compliance video MV2 based on the determination result of the determination unit 106. Furthermore, steps S212 to S220 of the process shown in FIG. 8 are omitted.

[0141] Although the above description has been given of the case where the information processing device 10 generates the determination result information 154 based on the determination result of the determination unit 106, this is not limiting. The information processing device 10 does not need to generate the determination result information 154. In this case, the control unit 100 does not need to include the determination result generation unit 108, and the process of step S222 in the process shown in FIG. 8 may be omitted.

[0142] Although the above description has been given of the case where the control unit 100 includes the presentation unit 109, this is not limiting. The control unit 100 does not have to include the presentation unit 109. In this case, the information processing device 10 does not provide the user of the management device 20 with a confirmation screen IMb based on the determination result information 154. The control unit 100 does not execute the process shown in FIG. 9.

[0143] When the information processing device 10 determines that a violation event EV1 has occurred, it may notify the moving object TG that the violation event EV1 has occurred. In this case, a notification unit is provided near the detection range AR. The notification unit may be, for example, a speaker that notifies by sound or a rotating light that notifies by light. The control unit 100 also includes a notification control unit. When the determination unit 106 determines that a violation event EV1 has occurred, the notification control unit identifies the detection range AR in which the moving object TG to be determined is located. Specifically, the notification control unit identifies the imaging unit 21 that generated the video MV that served as the source data for the determination process of the determination unit 106, and identifies the detection range AR in which the identified imaging unit 21 captures the image. The notification control unit notifies the moving object TG that the violation event EV1 has occurred via the notification unit corresponding to the identified detection range AR. With this configuration, the notification control unit notifies the moving object TG that is within or near the detection range AR that the violation event EV1 has occurred, thereby alerting the occupant of the moving object TG if the moving object TG is a vehicle, or a person TG2.

[0144] The notification unit may be provided in the moving object TG instead of (or in addition to) being provided near the detection range AR. Specifically, if the moving object TG is a vehicle, the notification unit may be provided in the driver's seat, etc. If the moving object TG is a person TG2, the notification unit may be implemented in a manner that the person TG2 can wear. In this case, when the determination unit 106 determines that a violation event EV1 has occurred, the notification control unit identifies the moving object identification information of the moving object TG to be determined. The process of identifying the moving object identification information is similar to the process executed by the determination result generation unit 108 described above, and therefore, description thereof will be omitted. Based on the identified moving object identification information, the notification control unit notifies the moving object TG associated with the identified moving object identification information that the violation event EV1 has occurred. With this configuration, the notification control unit notifies the moving object TG that caused the violation event EV1 that the violation event EV1 has occurred, thereby alerting the occupant of the moving object TG if the moving object TG is a vehicle, or the person TG2.

[0145] When notification is performed by the notification control unit, the video MV is constantly transmitted by the management device 20 to the server device 30 or the information processing device 10. The processing of the flowchart shown in FIG. 8 is executed immediately when the server device 30 or the information processing device 10 receives the video MV. Furthermore, the processing by the notification control unit is executed between step S210 and step S212 of the processing shown in FIG. 8. With this configuration, the notification control unit can issue a notification at a timing when the moving object TG that caused the violation event EV1 is closer to the detection range AR. When the video MV is constantly transmitted to the information processing device 10, the information processing device 10 is provided with a storage unit.

[0146] Instead of generating the violation video MV1 and the compliance video MV2, the determination result generating unit 108 may generate link information that associates the date and time when the violation event EV1 occurred with the playback time of the video MV at that date and time, and include the link information in the determination result information 154. With this configuration, the user of the management device 20 can easily check the video at the time when the violation event EV1 occurred by checking the determination result information 154.

[0147] The management device 20 may extract a portion of the video MV that shows the moving object TG and transmit it to the server device 30 or the information processing device 10. According to this configuration, when the position identification unit 103 identifies the position of the moving object TG from the video MV, the number of images IMa that make up the video MV is reduced, thereby reducing the processing load.

[0148] The management device 20 may execute part of the processing executed by the information processing device 10. For example, the management device 20 may execute the processing executed by the determination unit 101 among the processing executed by the information processing device 10. In this case, the management device 20 transmits information indicating the detection range AR and information indicating the reference direction DR to the information processing device 10. The determination unit 106 performs determination processing based on the information indicating the detection range AR and the information indicating the reference direction DR received from the management device 20.

[0149] The information processing device 10 may perform at least one of a process related to a violation of a regulation and a process related to compliance with a regulation. The technical ideas that can be understood from the above-described embodiment and modified examples will be described.

[0150] [Mode 1] This information processing method involves a computer presetting a detection range corresponding to a restricted position of a moving body shown in an image generated by capturing an image of the moving body's path of movement using an imaging unit installed at a predetermined position capable of capturing the image of the moving body, and a reference direction associated with the detection range; identifying the position of the moving body shown in the image; identifying the behavior of the moving body shown in the video based on a video of consecutive images in chronological order; identifying the direction of travel of the moving body shown in the video based on the video; and, if the identified direction of travel of the moving body matches the reference direction, determining that a violation event has occurred if at least a part of the moving body has not exhibited a predetermined behavior within the detection range since entering the detection range.

[0151] [Aspect 2] The information processing method according to aspect 1, wherein the predetermined behavior includes the moving body stopping temporarily or moving slowly. [Aspect 3] An information processing method described in [Aspect 2], in which if the time during which the speed of the moving body within the detection range remains below a speed threshold is less than a predetermined time, it is determined that the moving body is not stopping or slowing down.

[0152] [Aspect 4] An information processing method according to [Aspect 2], in which if the speed of the moving body within the detection range does not fall below a speed threshold, it is determined that the moving body is not stopping or slowing down.

[0153] [Aspect 5] An information processing method described in [Aspect 1], in which, when it is determined that the violation event has occurred, a violation video is generated by extracting a portion of the video that includes the image of the scene where the violation event occurred.

[0154] [Aspect 6] The information processing method according to [Aspect 5], wherein, when it is determined that the violation event has occurred, judgment result information indicating the judgment result corresponding to the violation event is generated. [Aspect 7] An information processing method described in [Aspect 6], which identifies the mobile body related to the violation event based on identification information corresponding to the mobile body, and generates the judgment result information by further including mobile body identification information that can identify the identified mobile body.

[0155] [Aspect 8] An information processing method described in [Aspect 6] or [Aspect 7], wherein the judgment result information includes imaging unit identification information that can identify the imaging unit, and a confirmation screen is generated that displays the occurrence status of the violation event for each imaging unit on the map image based on the judgment result information and a map image showing the specified position where the imaging unit corresponding to the imaging unit identification information is installed and the movement route.

[0156] [Aspect 9] An information processing method described in any one of [Aspect 1] to [Aspect 8], in which, when it is determined that the violation event has occurred, the mobile body is notified that the violation event has occurred.

[0157] [Aspect 10] An information processing method described in any one of [Aspect 1] to [Aspect 9], in which a compliance event is determined to have occurred if the moving object is performing a predetermined behavior within the detection range.

[0158] [Aspect 11] An information processing method described in [Aspect 10], in which, when it is determined that the compliance event has occurred, a compliance video is generated by extracting a portion of the video that includes the image of the scene where the compliance event occurred.

[0159] [Aspect 12] An information processing method according to [Aspect 11], which generates judgment result information indicating a judgment result corresponding to the compliance event. [Aspect 13] The preset image used to specify the detection range includes a frame image that is one size larger than the image, and the image arranged within the frame image; An information processing method according to any one of [Aspect 1] to [Aspect 12], in which the detection range is determined based on a user's operation on an operation area set within the frame image.

[0160] [Aspect 14] An information processing method described in any one of [Aspect 1] to [Aspect 13], which determines multiple detection ranges on the movement path and reference directions corresponding to each of the multiple detection ranges, and determines whether a violation event has occurred for each of the multiple detection ranges.

[0161] [Aspect 15] An information processing method described in any one of [Aspect 1] to [Aspect 14], wherein at least a portion of the moving body is the front end of the moving body in the direction of travel on the image.

[0162] [Aspect 16] When the images constituting the video are input, a trained model outputs the position of the moving object using a bounding box. The trained model inputs each of the images constituting the video, and identifies the position of the moving object based on the information output by the trained model; An information processing method described in any one of [Aspect 1] to [Aspect 15], wherein at least a portion of the moving object is the lower front end of the bounding box representing the moving object on the image in the direction of travel.

[0163] [Aspect 17] An information processing method described in any one of [Aspect 1] to [Aspect 16], in which the reference direction is the direction connecting the detection range and the entry range into which the moving body enters after passing through the detection range, and when the position of the identified moving body indicates that it has moved from the detection range to the entry range, it is determined that the direction of travel and the reference direction match.

[0164] [Aspect 18] An information processing method described in any one of [Aspect 1] to [Aspect 17], further determining the severity of the violation event based on the manner of movement of the moving object within the detection range.

[0165] [Aspect 19] An information processing system includes an imaging unit installed at a predetermined position capable of imaging the movement path of a moving body, one or more management devices that store a video of a chronological series of images generated by the imaging unit imaging the movement path on a server device on the cloud, and an information processing device that performs various processes based on the video stored on the server device, wherein the information processing device includes a determination unit that determines a detection range corresponding to the restricted position of the moving body shown in the image generated by the imaging unit imaging the movement path and a reference direction corresponding to the detection range, a position identification unit that identifies the position of the moving body shown in the image, a behavior identification unit that identifies the behavior of the moving body shown in the video based on the video, a direction identification unit that identifies the traveling direction of the moving body shown in the video based on the video, and a determination unit that determines that a violation event has occurred when the identified traveling direction of the moving body matches the reference direction and if at least a part of the moving body has not performed a predetermined behavior within the detection range since entering the detection range.

[0166] [Aspect 20] An information processing system includes an imaging unit installed at a predetermined position capable of imaging the movement path of a moving body, one or more management devices that store in the information processing device a video of a chronological series of images generated by the imaging unit imaging the movement path, and an information processing device that performs various processes based on the video, wherein the information processing device includes a determination unit that determines a detection range corresponding to the restricted position of the moving body shown in the image generated by the imaging unit imaging the movement path and a reference direction corresponding to the detection range, a position identification unit that identifies the position of the moving body shown in the image, a behavior identification unit that identifies the behavior of the moving body shown in the video based on the video, a direction identification unit that identifies the direction of travel of the moving body shown in the video based on the video, and a determination unit that determines that a violation event has occurred when the identified direction of travel of the moving body matches the reference direction and if at least a part of the moving body has not performed a predetermined behavior within the detection range since entering the detection range. [Explanation of symbols]

[0167] 1...information processing system, 10...information processing device, 20...management device, 21, 21a, 21b, 21c, 21d, 21e, 21f, 21g...imaging unit, 30...server device, 101...determination unit, 103...position identification unit, 104...behavior identification unit, 105...direction identification unit, 106...judgment unit, 151...trained model, 154...judgment result information, AR...detection range, ARa...entry range, BX...bounding box, DR...reference direction, EV1...violation event, EV2...compliance event, IM1...preset image, IMa...image, IMd...map image, IMw...frame image, JP...lower front end, MV...video, MV1...violation video, MV2...compliance video, TG...moving object, TG1...forklift, TG2...person.

Claims

1. The computer a detection range corresponding to a restriction position of the moving body shown in an image generated by capturing an image of the moving path using an imaging unit provided at a predetermined position capable of capturing an image of the moving path of the moving body, and a reference direction associated with the detection range; Identifying the position of the moving object shown in the image; Identifying the behavior of the moving object shown in the video based on the video in which the images are successive in time series; Identifying a traveling direction of the moving object shown in the video based on the video; When the identified traveling direction of the moving object matches the reference direction, and when at least a part of the moving object does not perform a predetermined behavior within the detection range after entering the detection range, it is determined that a violation event has occurred. Information processing methods.

2. The predetermined behavior includes a temporary stop or slowing down of the moving body. The information processing method according to claim 1 .

3. If the time during which the speed of the moving object within the detection range continues to be equal to or less than the speed threshold is shorter than a predetermined time, it is determined that the moving object is not stopping or moving slowly. The information processing method according to claim 2 .

4. If the speed of the moving object within the detection range does not become equal to or less than a speed threshold, it is determined that the moving object is not stopping or slowing down. The information processing method according to claim 2 .

5. generating a violation video by extracting a part of the video that includes the image of the scene where the violation event occurred, when it is determined that the violation event has occurred; The information processing method according to claim 1 .

6. generating determination result information indicating a determination result corresponding to the violation event when it is determined that the violation event has occurred; The information processing method according to claim 1 .

7. Identifying the mobile object involved in the violation event based on identification information corresponding to the mobile object; generating the determination result information by further including moving body identification information that can identify the specified moving body; The information processing method according to claim 6.

8. the determination result information includes imaging unit identification information that can identify the imaging unit, generating a confirmation screen that displays the occurrence status of the violation event for each imaging unit on the map image based on a map image that indicates the predetermined position where the imaging unit corresponding to the imaging unit identification information is provided and the movement route, and the determination result information; The information processing method according to claim 6.

9. If it is determined that the violation event has occurred, notify the mobile body that the violation event has occurred. The information processing method according to claim 1 .

10. When the moving object is performing a predetermined behavior within the detection range, it is determined that a compliance event has occurred. The information processing method according to claim 1 .

11. generating a compliance video by extracting a part of the video including the image of the scene where the compliance event occurred, when it is determined that the compliance event has occurred; The information processing method according to claim 10.

12. generating judgment result information indicating a judgment result corresponding to the compliance event; The information processing method according to claim 11.

13. the preset image used to specify the detection range includes a frame image that is one size larger than the image, and the image arranged within the frame image; determining the detection range based on a user's operation on an operation area set within the frame image; The information processing method according to claim 1 .

14. determining a plurality of detection ranges and the reference directions respectively associated with the plurality of detection ranges on the movement path; determining whether the violation event has occurred for each of the plurality of detection ranges; The information processing method according to claim 1 .

15. At least a part of the moving object is a front end portion of the moving object in the traveling direction on the image. The information processing method according to claim 1 .

16. Identifying the position of the moving object based on information output by inputting each of the images constituting the video into a trained model that outputs the position of the moving object using a bounding box when the images constituting the video are input; At least a part of the moving object is a lower part of the front end of the bounding box representing the moving object on the image in the traveling direction. The information processing method according to claim 1 .

17. the reference direction is a direction connecting the detection range and an entry range into which the vehicle enters after passing through the detection range, If the identified position of the moving object indicates that it has moved from the detection range to the entry range, it is determined that the traveling direction matches the reference direction. The information processing method according to claim 1 .

18. further determining the degree of the violation event based on a movement pattern of the moving object within the detection range; The information processing method according to claim 1 .

19. An information processing system comprising: an imaging unit provided at a predetermined position capable of capturing an image of a moving path of a moving object; one or more management devices that store, in a server device on a cloud, a video in which images generated by the imaging unit capturing the moving path are successive in time series; and an information processing device that executes various processes based on the video stored in the server device, The information processing device includes: a determination unit that determines a detection range corresponding to a restriction position of the moving object shown in an image generated by the imaging unit by capturing an image of the moving path, and a reference direction associated with the detection range; a position identification unit that identifies the position of the moving object shown in the image; a behavior identification unit that identifies, based on the video, a behavior of the moving object shown in the video; a direction identification unit that identifies a traveling direction of the moving object shown in the video based on the video; and a determination unit that determines that a violation event has occurred when the identified traveling direction of the moving object matches the reference direction and when at least a part of the moving object has not performed a predetermined behavior within the detection range since entering the detection range. Information processing system.

20. An information processing system comprising: an imaging unit provided at a predetermined position capable of capturing an image of a moving path of a moving object; one or more management devices that store, in an information processing device, a video in which images generated by the imaging unit capturing the moving path are successive in time series; and an information processing device that executes various processes based on the video, The information processing device includes: a determination unit that determines a detection range corresponding to a restriction position of the moving object shown in an image generated by the imaging unit by capturing an image of the moving path, and a reference direction associated with the detection range; a position identification unit that identifies the position of the moving object shown in the image; a behavior identification unit that identifies, based on the video, a behavior of the moving object shown in the video; a direction identification unit that identifies a traveling direction of the moving object shown in the video based on the video; and a determination unit that determines that a violation event has occurred when the identified traveling direction of the moving object matches the reference direction and when at least a part of the moving object has not performed a predetermined behavior within the detection range since entering the detection range. Information processing system.

Images