Road monitoring system, information processing device, road monitoring method and program

Abstract

To support efficient road monitoring.SOLUTION: A road monitoring system 100 includes a video acquisition unit 121, an object determination unit 122, and a display control unit 133. The video acquisition unit 121 obtains a video of a road. The object determination unit 122 determines whether at least one frame image constituting the video is an analysis target, based on both a result of comparison between the at least one frame image and a reference image and a first reference. The display control unit 133 makes a display unit display information based on a result of the determination. The display control unit 133 includes a first control unit 133b and a second control unit 133c. The first control unit 133b makes the display unit display an analysis result for the at least one frame image that is determined as being an analysis target. The second control unit 133c makes the display unit display detection information indicating that a frame image being not an analysis target is detected.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a road monitoring system, an information processing device, a road monitoring method, and a program.

Background Art

[0002] In a system for monitoring a road, a system for detecting an abnormality of a camera that photographs the road has been proposed. [[ID=1...]]

[0003] For example, the technique disclosed in Patent Document 1 detects camera abnormality based on the difference in luminance values between an input image acquired from a camera and a reference image acquired in advance, and notifies the type of the abnormality and the like. The camera abnormalities disclosed in Patent Document 1 include partial shielding, overall shielding, deviation of the shooting angle (deviation of the camera direction), blurring, noise, halation, and the like.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, more efficient support for road monitoring is required. For example, the technique disclosed in Patent Document 1 does not disclose how to use an input image from a camera when there is no camera abnormality.

[0006] An example of an object of the present invention is to provide a road monitoring system, an information processing device, a road monitoring method, a program, and the like that solve the above-described problems and support efficient road monitoring in view of the above-described problems.

Means for Solving the Problems

[0007] According to one aspect of the present invention, video acquisition means for acquiring video obtained by photographing a road, A target determination means that determines whether or not the at least one frame image is the subject of analysis based on the result of comparing the at least one frame image constituting the video with a reference image and a first criterion, The system includes a display control means for displaying information based on the result of the aforementioned determination on a display means, The display control means is A first control means that causes the display means to display the analysis results of the at least one frame image that has been determined to be the subject of analysis, Includes a second control means that causes the display means to display detection information indicating that a frame image that is not the subject of analysis has been detected. A road monitoring system will be provided.

[0008] According to one aspect of the present invention, A means of acquiring video footage of a road, A target determination means that determines whether or not the at least one frame image is the subject of analysis based on the result of comparing the at least one frame image constituting the video with a reference image and a first criterion, The system includes a transmission means for transmitting information based on the results of the aforementioned determination, The aforementioned transmission means is A first transmission means for transmitting the analysis results of the at least one frame image that has been determined to be the subject of analysis, Includes a second transmission means that transmits detection information indicating that a frame image not subject to analysis has been detected. An information processing device is provided.

[0009] According to one aspect of the present invention, One or more computers, We obtained video footage of the road, Based on the results of comparing at least one frame image constituting the video with a reference image, and the first criterion, it is determined whether or not the at least one frame image is the subject of analysis. This includes displaying information based on the results of the aforementioned determination on a display means, The causing to display includes causing the display means to display the analysis result of the at least one frame image determined to be the analysis target, and causing the display means to display detection information indicating that a frame image that is not the analysis target has been detected. A road monitoring method is provided.

[0010] According to one aspect of the present invention, one or more computers are caused to acquire an image of a road, determine whether the at least one frame image is an analysis target based on a result of comparing the at least one frame image constituting the image with a reference image and a first criterion, and is for causing information based on the result of the determination to be displayed on display means. The causing to display includes causing the display means to display the analysis result of the at least one frame image determined to be the analysis target, and causing the display means to display detection information indicating that a frame image that is not the analysis target has been detected. A program is provided.

Advantages of the Invention

[0011] According to one aspect of the present invention, it becomes possible to support efficient road monitoring.

Brief Description of the Drawings

[0012] [Figure 1] It is a diagram showing an overview of a road monitoring system according to Embodiment 1. [Figure 2] It is a diagram showing an overview of an information processing apparatus according to Embodiment 1. [Figure 3] It is a flowchart showing an overview of a road monitoring method according to Embodiment 1. [Figure 4] It is a diagram showing a configuration example of a road monitoring system according to Embodiment 1. [Figure 5] It is a diagram showing a configuration example of image information according to Embodiment 1. [Figure 6] It is a diagram showing a functional configuration example of the first information processing apparatus according to Embodiment 1. [Figure 7] It is a diagram showing a configuration example of analysis information according to Embodiment 1. [Figure 8] It is a diagram showing a functional configuration example of the second information processing apparatus according to Embodiment 1. [Figure 9] It is a diagram showing a physical configuration example of the imaging apparatus according to Embodiment 1. [Figure 10] It is a diagram showing a physical configuration example of the first information processing apparatus according to Embodiment 1. [Figure 11] It is a flowchart showing an example of imaging processing according to Embodiment 1. [Figure 12] It is a diagram showing an example of the road R to be imaged. [Figure 13] It is a diagram showing an example of image information IMD including a frame image IM1 obtained by imaging the road R illustrated in FIG. 12. [Figure 14] It is a flowchart showing an example of the first information processing according to Embodiment 1. [Figure 15] It is a diagram showing an example of the frame image IM1 to be compared and a reference image. [Figure 16] It is a diagram showing another example of the frame image IM2 to be compared and a reference image. [Figure 17] It is a flowchart showing an example of the second information processing according to Embodiment 1. [Figure 18] It is a diagram showing an example of a screen for displaying an analysis result. [Figure 19] It is a diagram showing an example of a screen for displaying detection information. [Figure 20] It is a diagram showing a functional configuration example of the target determination unit according to Embodiment 2. [Figure 21] It is a flowchart showing an example of the first information processing according to Embodiment 2. [Figure 22] It is a flowchart showing an example of the selection process according to Embodiment 2. [Figure 23] It is a diagram showing a functional configuration example of the target determination unit according to Embodiment 3. [Figure 24] This is a flowchart showing an example of the first information processing according to Embodiment 3. [Figure 25] This figure shows an example configuration of a road monitoring system according to Embodiment 4. [Modes for carrying out the invention]

[0013] Embodiments of the present invention will be described below with reference to the drawings. In all drawings, similar components are denoted by the same reference numerals, and their descriptions are omitted where appropriate.

[0014] <Embodiment 1> (overview) Figure 1 is a diagram showing an overview of the road monitoring system 100 according to Embodiment 1. The road monitoring system 100 includes an image acquisition unit 121, a target determination unit 122, and a display control unit 133.

[0015] The video acquisition unit 121 acquires video footage of the road.

[0016] The target determination unit 122 determines whether at least one frame image is the target of analysis based on the result of comparing at least one frame image constituting the video with a reference image and the first criterion.

[0017] The display control unit 133 causes the display unit to display information based on the result of the decision.

[0018] The display control unit 133 includes a first control unit 133b and a second control unit 133c. The first control unit 133b causes the display unit to display the analysis results of at least one frame image that has been determined to be the target of analysis. The second control unit 133c causes the display unit to display detection information indicating that a frame image that is not the target of analysis has been detected.

[0019] This road monitoring system 100 makes it possible to support efficient road monitoring.

[0020] Figure 2 shows an overview of the information processing device 102 according to Embodiment 1. The information processing device 102 includes an image acquisition unit 121, a target determination unit 122, and a transmission unit 124.

[0021] The video acquisition unit 121 acquires video footage of the road.

[0022] The target determination unit 122 determines whether at least one frame image is the target of analysis based on the result of comparing at least one frame image constituting the video with a reference image and the first criterion.

[0023] The transmitting unit 124 transmits information based on the result of the judgment.

[0024] The transmitting unit 124 includes a first transmitting unit 124a and a second transmitting unit 124b. The first transmitting unit 124a transmits the analysis results of at least one frame image that has been determined to be the subject of analysis. The second transmitting unit 124b transmits detection information indicating that a frame image that is not the subject of analysis has been detected.

[0025] This information processing device 102 makes it possible to support efficient road monitoring.

[0026] Figure 3 is a flowchart showing an overview of the road monitoring method according to Embodiment 1.

[0027] The video acquisition unit 121 acquires video footage of the road (step S201).

[0028] The target determination unit 122 determines whether at least one frame image is the target of analysis based on the result of comparing at least one frame image constituting the video with a reference image and the first criterion (step S202).

[0029] The display control unit 133 causes the display unit to display information based on the result of the decision (step S302).

[0030] Displaying includes displaying the analysis results of at least one frame image that has been determined to be the subject of analysis on the display unit, and displaying detection information on the display unit indicating that a frame image that is not the subject of analysis has been detected.

[0031] This road monitoring method makes it possible to support efficient road monitoring.

[0032] The following describes a detailed example of the road monitoring system 100 according to Embodiment 1.

[0033] (detail) Figure 4 shows an example configuration of the road monitoring system 100 according to Embodiment 1. The road monitoring system 100 is a system for supporting road monitoring by a user. The road monitoring system 100 comprises a camera 101, a first information processing device 102 (corresponding to the "information processing device 102" described above), and a second information processing device 103.

[0034] The imaging device 101, the first information processing device 102, and the second information processing device 103 are connected to each other via a network N, which is configured, for example, by wire, wireless, or a combination thereof, and can send and receive information from each other via the network N.

[0035] (Example of functional configuration of imaging device 101) The camera 101 photographs the road and generates video. The video consists of a series of frame images, i.e., multiple images, captured by the camera 101.

[0036] In detail, for example, the imaging device 101 photographs a predetermined location on the road at a predetermined frequency (frame rate) and generates a frame image with each photograph. In this embodiment, the imaging area of ​​the imaging device 101, i.e., the area that appears in the frame image, is pre-set for the imaging device 101.

[0037] The imaging device 101 generates image information including the generated frame image and transmits it to the first information processing device 102. More specifically, for example, when the imaging device 101 generates a frame image, it may generate image information in real time and transmit it to the first information processing device 102. This allows the imaging device 101 to transmit video in real time.

[0038] The frame image can be either color or monochrome, and its pixel count should be selected appropriately.

[0039] Figure 5 shows an example of the configuration of image information according to Embodiment 1. The image information is information in which image-related information is associated with a frame image generated by the imaging device 101. The image-related information includes, for example, image identification information, imaging device identification information, shooting date, and shooting location.

[0040] Image identification information is information used to identify image information. Since image information is generated for each frame image, image identification information is also information used to identify the frame image. Image identification information will also be referred to as "image ID (Identification)" below.

[0041] The imaging device identification information is information used to identify the imaging device 101. The imaging device identification information will also be referred to as the "imaging ID" below.

[0042] The shooting date is information indicating when the photograph was taken. For example, the shooting date consists of the year, month, day, and time. The time may be expressed in predetermined increments such as 1 / 10th of a second or 1 / 100th of a second.

[0043] The shooting location is information indicating the place where the image was taken. For example, the shooting location is information indicating the location where the shooting device 101 is installed, and consists of the latitude and longitude of that location. The shooting location may be acquired using the position detection function if the shooting device 101 is equipped with a position detection function, or it may be set in advance by the installer or the like. The position detection function is a function that detects the position of the shooting device 101 using GPS (Global Positioning System) or the like.

[0044] The image information only needs to include at least a frame image of the road, and does not need to include any accompanying image information. The accompanying image information is not limited to the examples above, and may include one or more items such as image ID, shooting ID, shooting date, and shooting location.

[0045] (Example of functional configuration of the first information processing device 102) Figure 6 shows an example of the functional configuration of the first information processing device 102 according to Embodiment 1. The first information processing device 102 is a device that analyzes video footage of a road. The first information processing device 102 includes a video acquisition unit 121, a target determination unit 122, an analysis unit 123, and a transmission unit 124.

[0046] The video acquisition unit 121 acquires video footage of the road. For example, the video acquisition unit 121 acquires image information from the camera 101 in real time. As a result, the video acquisition unit 121 acquires video footage of the road in real time.

[0047] The target determination unit 122 determines whether or not the at least one frame image constituting the video is the target of analysis, based on the result of comparing the frame image with the reference image and the first criterion.

[0048] The first criterion is the standard that the frame image to be analyzed must meet. The subject of analysis is, for example, the subject of analysis by the analysis unit 123, which will be described in detail later.

[0049] For comparison between at least one frame image and a reference image, it is preferable to use the image features of each image. Image features are, for example, features that indicate the area (capture area) captured in the frame image.

[0050] In detail, for example, the target determination unit 122 includes a reference image storage unit 122a, a comparison unit 122b, and a first determination unit 122c, as shown in Figure 6.

[0051] The reference image storage unit 122a is a storage unit in which reference images are stored in advance. The reference image is an image that shows a pre-set shooting area for the shooting device 101. In other words, the reference image is an image that shows the appropriate shooting area for the shooting device 101. Note that the reference image storage unit 122a may also store the image feature quantities of the reference image in advance, either along with the reference image or in place of the reference image.

[0052] The comparison unit 122b compares each image feature of the frame image with the image feature of the reference image. In this embodiment, we will explain using an example where the result of the comparison is the similarity between each image feature of the frame image and the image feature of the reference image. That is, the comparison unit 122b according to this embodiment determines the similarity between each image feature of the frame image and the image feature of the reference image.

[0053] The first determination unit 122c determines whether each frame image is a target for analysis based on the results of the comparison by the comparison unit 122b and the first criterion.

[0054] In detail, for example, the first determination unit 122c determines whether the comparison result of the comparison unit 122b for each frame image satisfies the first criterion. The first determination unit 122c then determines that the frame image is a target for analysis if the comparison result satisfies the first criterion. The first determination unit 122c determines that the frame image is not a target for analysis if the comparison result does not satisfy the first criterion.

[0055] As described above, when similarity is adopted as the "result of comparison," the first criterion may be, for example, a relationship between similarity and a threshold. In this embodiment, we will explain using an example where the first criterion is that the similarity sought by the comparison unit 122b is equal to or greater than a threshold.

[0056] The analysis unit 123 analyzes at least one frame image if it determines that at least one frame image is the target of analysis. The analysis unit 123 then generates analysis information including the results of that analysis (i.e., analysis results).

[0057] If multiple frame images are determined to be subject to analysis, the analysis unit 123 according to this embodiment analyzes all of the frame images determined to be subject to analysis. The analysis unit 123 according to this embodiment then analyzes each of the frame images determined to be subject to analysis and generates analysis information including the analysis results for each frame image.

[0058] Furthermore, the analysis unit 123 may analyze some of the frame images that have been determined to be the target of analysis, such as frame images at predetermined time intervals. Also, the analysis unit 123 does not have to analyze frame images that have been determined not to be the target of analysis.

[0059] (Example of analysis by Analysis Unit 123) The analysis performed by the analysis unit 123 may be selected as appropriate. For example, the analysis unit 123 may perform road monitoring analysis by processing frame images that have been determined to be the target of analysis. The analysis unit 123 may then generate analysis information including the analysis results.

[0060] Here, we will explain using the example of an analysis that involves detecting road conditions and events.

[0061] The analysis unit 123 detects road conditions by processing frame images that have been determined to be the target of analysis.

[0062] Road conditions refer to the state of objects on the road. Objects on the road include, for example, one or more vehicles, fallen objects, etc. Vehicles include, for example, one or more passenger cars, trucks, trailers, construction vehicles, emergency vehicles, motorcycles, bicycles, etc. Fallen objects include things that have fallen onto the road from vehicles, etc., and things that have been blown onto the road by wind, etc.

[0063] Road conditions may include the presence or absence of objects on the road. If there are objects on the road, road conditions may include object identification information and object conditions for each object. The presence or absence of objects on the road may be represented by a flag indicating their presence or absence, the number of objects on the road, or by whether or not object identification information is included in the road conditions.

[0064] Object identification information is information used to identify objects on the road. This information will also be referred to as "object ID" below. Object status refers to the state of each object.

[0065] The items included in the object state may differ in some or all ways depending on the type of object. For example, the object state (vehicle state) for a vehicle may include one or more items such as the vehicle's position, direction of travel, speed, movement path (trajectory), and attributes. The attributes of a vehicle may include one or more items such as the type of vehicle, size, color, and license plate number. The type of vehicle may include one or more items such as the passenger car, truck, trailer, construction vehicle, emergency vehicle, motorcycle, and bicycle mentioned above.

[0066] For example, the object state of a falling object (falling object state) may be one or more of the following: the position of the falling object, the direction of movement, the speed of movement, the path of movement, and the attributes of the falling object. The attributes of a falling object may be one or more of the following: the type of falling object, the size of the falling object, and the color of the falling object. The type of falling object may be one or more of the following: wood, packaged goods, etc.

[0067] Common techniques such as pattern matching and pre-trained machine learning models can be used to detect such road conditions.

[0068] When a learning model is used, the analysis unit 123 detects the road condition by inputting frame images acquired by the video acquisition unit 121 into a pre-trained learning model that has performed machine learning to detect road conditions, for example. In machine learning, it is preferable to perform supervised learning using, for example, training data in which images of roads are labeled as input data.

[0069] Furthermore, the analysis unit 123 detects predetermined events on the road based on the detected road conditions. These events include one or more of the following: (1) traffic congestion, (2) vehicles driving in the wrong direction, (3) vehicles driving at a low speed, (4) vehicles stopped, (5) fallen objects, and (6) erratic driving. However, the events are not limited to these.

[0070] (1) Traffic congestion is detected when, for example, a line of vehicles consisting of vehicles traveling at a slow speed or vehicles that repeatedly stop and start is longer than a predetermined distance and continues for a predetermined duration. In this context, slow speed means traveling at a speed below a predetermined speed.

[0071] (2) A vehicle driving in the wrong direction is detected using, for example, the following conditions: (2-A) that the road in question or each lane constituting the road has a predetermined direction of travel; and (2-B) that the direction of travel of the vehicle and the direction of travel of the road or lane on which the vehicle is traveling differ by more than a predetermined angle (e.g., 90 degrees). A vehicle driving in the wrong direction is detected when both (2-A) and (2-B) are met, and a vehicle driving in the wrong direction is not detected when at least one of (2-A) and (2-B) is not met.

[0072] (3) Low-speed driving of a vehicle is detected, for example, when the vehicle continues to travel at or below a predetermined speed for a predetermined duration or longer. The predetermined speed here may be the same as, or different from, the predetermined speed defined for low-speed driving in traffic congestion in (1).

[0073] (4) Vehicle stopping is detected, for example, when the vehicle remains stopped for a predetermined period of time or longer (vehicle position is within a predetermined range).

[0074] (5) Fallen objects are detected if, for example, (5-A) there is an object other than a vehicle on the road, or (5-B) a specified number of vehicles or more make a temporary lane change within a common area, or both. Fallen objects are detected if either (5-A) or (5-B) is met, and are not detected if neither (5-A) nor (5-B) is met.

[0075] A temporary lane change refers to changing lanes and then returning to the original lane within a predetermined distance or time. Generally, vehicles will move to avoid fallen objects, so it is presumed that fallen objects are located in an area that multiple vehicles will commonly avoid. Therefore, fallen objects can be detected using (5-B).

[0076] (6) Swerving is detected, for example, when a vehicle makes temporary lane changes more than a predetermined number of times. A temporary lane change, as described above, is when a vehicle changes lanes and then returns to its original lane within a predetermined distance or time.

[0077] When the analysis unit 123 detects road conditions and events, it generates analysis information that includes the detected road conditions and events as analysis results.

[0078] Figure 7 shows an example of the structure of analysis information according to Embodiment 1. The analysis information is information that associates an event and state-related information with the detected road condition. The event is information that indicates an event detected based on the associated road condition, such as "traffic jam" or "driving in the wrong direction." The state-related information includes a frame image used to detect the associated road condition and image-related information associated with the frame image in the image information.

[0079] Furthermore, the analysis information only needs to include, but is not limited to, the analysis results. The analysis information may include, for example, only road conditions or events, and may be modified as appropriate depending on the content of the analysis performed by the analysis unit 123.

[0080] The transmission unit 124 transmits information based on the judgment result. This information may be, for example, analysis information or detection information. Detection information indicates that a frame image that is not the target of analysis has been detected.

[0081] In detail, for example, the transmitting unit 124 includes a first transmitting unit 124a and a second transmitting unit 124b, as shown in Figure 6.

[0082] The first transmitting unit 124a transmits analysis information regarding the frame image that has been determined to be the subject of analysis. In other words, the first transmitting unit 124a is an example of a means for transmitting the analysis results of at least one frame image that has been determined to be the subject of analysis.

[0083] In this embodiment, the first transmission unit 124a transmits all analysis information relating to the frame image that has been determined to be the target of analysis.

[0084] The first transmission unit 124a may transmit some of the analysis information, such as analysis information relating to frame images at predetermined time intervals, from among all the analysis information.

[0085] The second transmission unit 124b analyzes at least one frame image. There isn't If this is determined, detection information for at least one frame image is transmitted. In other words, the second transmission unit 124b is an example of a means for transmitting detection information indicating that a frame image that is not the subject of analysis has been detected.

[0086] In this embodiment, the second transmission unit 124b analyzes multiple frame images. There isn't If this is determined, detection information for each of the multiple frame images will be sent.

[0087] As described above, the detection information indicates that a frame image that was not the target of analysis was detected. The detection information may also include information about the frame image used to determine that it was not the target of analysis.

[0088] Information regarding the frame image may include at least one of the following (1) to (3): (1) Camera identification information for identifying the camera that captured the frame image. (2) At least one of the frame image and the reference image used in the determination (3) Misalignment information regarding the misalignment of the area shown in the frame image

[0089] The misalignment information is, for example, information indicating the degree of misalignment between the area shown in the frame image (the shooting area of ​​the shooting device 101) and the area shown in the reference image (the reference area). More specifically, the misalignment information is at least one of the following: the similarity determined by the comparison unit 122b, and an index such as a symbol associated with the similarity in stages.

[0090] (Example of the functional configuration of the second information processing device 103) Figure 8 shows an example of the functional configuration of the second information processing device 103 according to Embodiment 1. The second information processing device 103 is a device that displays images for monitoring roads. The second information processing device 103 includes a receiving unit 131, a display unit 132, and a display control unit 133.

[0091] The receiving unit 131 receives information transmitted from the transmitting unit 124. The receiving unit 131 receives, for example, analysis information or detection information relating to at least one frame image. In this embodiment, the receiving unit 131 receives analysis information or detection information relating to all frame images.

[0092] The display unit 132 displays various information under the control of the display control unit 133.

[0093] The display control unit 133 displays information on the display unit 132 based on the result of a judgment, according to the information received by the receiving unit 131. In other words, the display control unit 133 is an example of a means for displaying information on a display means based on the result of a judgment.

[0094] In detail, for example, the display control unit 133 includes a content determination unit 133a, a first control unit 133b, and a second control unit 133c, as shown in Figure 8.

[0095] The content determination unit 133a determines the content of the information when the receiving unit 131 receives it. For example, the content determination unit 133a determines whether the information received by the receiving unit 131 is analysis information or detection information.

[0096] The first control unit 133b displays the analysis information on the display unit 132 when it determines that the content determination unit 133a has received the analysis information. As can be seen from the above explanation, the analysis information is information about the frame image that has been determined to be the target of analysis, and includes the analysis results of the frame image. In other words, the first control unit 133b is an example of a means for displaying the analysis results of at least one frame image that has been determined to be the target of analysis on the display means.

[0097] In this embodiment, the first control unit 133b displays all analysis information related to the frame image that has been determined to be the subject of analysis on the display unit 132. When the first control unit 133b displays the analysis information on the display unit 132, it is preferable to display at least some or all of the analysis results from the information contained in the analysis information on the display unit 132.

[0098] The first control unit 133b may display some of the analysis information, such as analysis information related to frame images at predetermined time intervals, on the display unit 132.

[0099] The second control unit 133c displays the detection information on the display unit 132 when it determines that the content determination unit 133a has received the detection information. In other words, the second control unit 133c is one example of a means for displaying detection information on the display unit 132 that indicates that a frame image that is not the target of analysis has been detected.

[0100] The second embodiment of this embodiment Control unit 133c When detection information relating to multiple frame images is received, all of that detection information is displayed on the display unit 132.

[0101] Also, the second Control unit 133c If the detection information includes a frame image and a reference image, the frame image and the reference image may be displayed side by side on the display unit 132.

[0102] Up to this point, we have mainly described an example of the functional configuration of the road monitoring system 100 according to Embodiment 1. From here, we will describe an example of the physical configuration of the road monitoring system 100 according to Embodiment 1.

[0103] (Example of physical configuration of road monitoring system 100) The road monitoring system 100 physically comprises, for example, a camera 101, a first information processing device 102, and a second information processing device 103.

[0104] The physical configuration of the road monitoring system 100 is not limited to this. For example, the functions of the imaging device 101 and the two information processing devices 102 and 103 described in this embodiment may be physically provided in a single device, or they may be divided and provided in each of multiple devices in a manner different from this embodiment. When the functions of transmitting or receiving information between devices 101 to 103 in this embodiment are incorporated into a common physical device, it is preferable to transmit or receive information via an internal bus or the like instead of the network N.

[0105] (Example of the physical configuration of the imaging device 101) Figure 9 shows an example of the physical configuration of the imaging device 101 according to Embodiment 1. The imaging device 101 physically includes, for example, a bus 1010, a processor 1020, a memory 1030, a storage device 1040, a network interface 1050, a user interface 1060, and a camera 1070.

[0106] Bus 1010 is a data transmission path for the processor 1020, memory 1030, storage device 1040, network interface 1050, user interface 1060, and camera 1070 to send and receive data to and from each other. However, the method of connecting the processor 1020 and the other components to each other is not limited to bus connection.

[0107] The 1020 processor is a processor implemented in components such as the CPU (Central Processing Unit) and GPU (Graphics Processing Unit).

[0108] Memory 1030 is a main memory device implemented using RAM (Random Access Memory), etc.

[0109] The storage device 1040 is an auxiliary storage device implemented as an HDD (Hard Disk Drive), SSD (Solid State Drive), memory card, or ROM (Read Only Memory). The storage device 1040 stores program modules for realizing each function of the imaging device 101. The processor 1020 reads these program modules into memory 1030 and executes them, thereby realizing each function corresponding to that program module.

[0110] The network interface 1050 is an interface for connecting the imaging device 101 to the network N.

[0111] The user interface 1060 includes touch panels, keyboards, mice, etc., as interfaces for the user to input information, and liquid crystal panels, organic EL (Electro-Luminescence) panels, etc., as interfaces for presenting information to the user.

[0112] Camera 1070 photographs subjects such as roads and generates images of those subjects. The imaging device 101 is installed, for example, on the side of the road or above the road so that camera 1070 can photograph a predetermined location on the road.

[0113] Furthermore, the imaging device 101 may accept input from the user and present information to the user via an external device connected to the network N (for example, the first information processing device 102, the second information processing device 103, etc.). In this case, the imaging device 101 does not need to have a user interface 1060.

[0114] (Example of physical configuration of the first information processing device 102 and the second information processing device 103) Figure 10 shows an example of the physical configuration of the first information processing device 102 according to Embodiment 1. The first information processing device 102 physically has, for example, a bus 1010, a processor 1020, a memory 1030, a storage device 1040, and a network interface 1050, similar to those of the imaging device 101. The first information processing device 102 further physically has, for example, an input interface 2060 and an output interface 2070.

[0115] However, the storage device 1040 of the first information processing device 102 stores program modules for realizing each function of the first information processing device 102. Furthermore, the network interface 1050 of the first information processing device 102 is an interface for connecting the first information processing device 102 to the network N.

[0116] The input interface 2060 is an interface for the user to input information, and includes, for example, a touch panel, keyboard, mouse, etc. The output interface 2070 is an interface for presenting information to the user, for example liquid This includes crystal panels, organic EL panels, etc.

[0117] The second information processing device 103 according to Embodiment 1 may be physically configured in the same way as the first information processing device 102, for example. However, the storage device 1040 of the second information processing device 103 stores program modules for realizing each function of the second information processing device 103. Furthermore, the network interface 1050 of the second information processing device 103 is an interface for connecting the second information processing device 103 to the network N.

[0118] We have now described an example of the configuration of the road monitoring system 100 according to Embodiment 1. From here, we will describe an example of the operation of the road monitoring system 100 according to Embodiment 1.

[0119] (Example of operation of road monitoring system 100) The road monitoring system 100 performs road monitoring processing to monitor roads. This road monitoring processing includes, for example, imaging processing performed by the imaging device 101, first information processing performed by the first information processing device 102, and second information processing performed by the second information processing device 103. These processes will be explained with reference to the diagram.

[0120] (Example of the imaging process according to Embodiment 1) Figure 11 is a flowchart illustrating an example of the imaging process according to Embodiment 1. The imaging process is a process for photographing a road. When the imaging device 101 receives a start instruction from the user via, for example, the second information processing device 103, it repeatedly executes the imaging process at a predetermined frequency until it receives a stop instruction from the user. Note that the method for starting or ending the imaging process is not limited to these.

[0121] The imaging device 101 photographs the road and generates image information (step S101).

[0122] In detail, for example, when the camera 1070 photographs a predetermined location on the road, the imaging device 101 generates image information including the frame image obtained by this photography.

[0123] Figure 12 shows an example of a road R being photographed.

[0124] Road R includes shoulders RS1 and RS2 provided along both sides of Road R, and a median strip SZ provided roughly in the center along the road. Road R further includes lanes L1 and L2 provided between shoulder RS1 and median strip SZ, and lanes L3 and L4 provided between shoulder RS2 and median strip SZ. Road lights M1 to M4 are provided on the sides of Road R to illuminate lanes L1 to L4, respectively.

[0125] In Figure 12, the dotted arrows indicate the designated direction of travel for each lane. Vehicles C1, C2, C3, and C4 are traveling on road R. In Figure 12, the solid arrows indicate the direction of travel for each vehicle.

[0126] Figure 13 shows an example of image information IMD including frame image IM1 of road R as illustrated in Figure 12. The image information IMD illustrated in Figure 13 associates image-related information with frame image IM1. The image-related information illustrated in Figure 13 associates image ID "P1", shooting ID "CM1", shooting date "T1", and shooting location "L1".

[0127] "P1" is the image ID assigned to the frame image IM1. The imaging device 101 may, for example, assign an image ID to the image IM1 according to a predetermined rule and set the image ID in the image information IMD.

[0128] "CM1" is the shooting ID of the shooting device 101. The shooting device 101 may, for example, store a shooting ID that has been set in advance by the user via the second information processing device 103, and set the shooting ID in the image information IMD.

[0129] "T1" is a frame image IM 1 This indicates the time of capture. The imaging device 101 may, for example, be equipped with a timing function and set the time of capture as the capture date in the image information IMD.

[0130] "L1" is information indicating the location where the imaging device 101 will take a photograph. The imaging device 101 may, for example, store in advance a shooting location (for example, the installation location of the imaging device 101) set by the user via the second information processing device 103, and set that shooting location in the image information IMD.

[0131] Refer to Figure 11 again. The imaging device 101 transmits the image information generated in step S101 to the first information processing device 102 (step S102), and then returns to step S101.

[0132] This type of shooting process allows the video (i.e., each frame image captured at a predetermined frame rate) to be transmitted to the first information processing device 102 in near real-time. Step S102 may also be performed at pre-set time intervals to transmit image information for a portion of the captured frame images.

[0133] (Example of the first information processing according to Embodiment 1) Figure 14 is a flowchart showing an example of the first information processing according to Embodiment 1. The first information processing is a process for analyzing video footage of a road. The first information processing device 102, for example, in the same way as the camera 101, receives a start command from the user via the second information processing device 103 and repeatedly executes the first information processing until it receives a stop command from the user. Note that the method for starting or ending the first information processing is not limited to these.

[0134] The video acquisition unit 121 acquires video footage of the road (step S201).

[0135] In detail, for example, the video acquisition unit 121 acquires image information from the shooting device 101 in real time. As a result, the video acquisition unit 121 acquires frame images of the road in real time.

[0136] The target determination unit 122 determines whether the frame image is the target of analysis based on the result of comparing the frame image included in the image information acquired in step S201 with the reference image, and the first criterion (step S202).

[0137] In detail, for example as shown in Figure 14, the comparison unit 122b compares the frame image acquired in step S201 with the reference image stored in the reference image storage unit 122a (step S202a).

[0138] Figure 15 shows an example of a comparison between a frame image IM1 and a reference image. Figure 15(a) shows the frame image IM1, and Figure 15(b) shows the reference image. The reference image is, for example, a frame image of road R previously captured by the imaging device 101.

[0139] The comparison unit 122b acquires image features from comparison region A1 and comparison region AR, which are set to be common ranges for both the frame image IM1 and the reference image. A common range means that the positions (pixel positions) of comparison region A1 and comparison region AR are common to both the frame image IM1 and the reference image, respectively.

[0140] For acquiring image features, general techniques can be used; for example, a pre-trained machine learning model can be used. In this case, the comparison unit 122b obtains the image features of comparison region A1 by inputting comparison region A1 into a pre-trained machine learning model that has been used to detect image features (e.g., edges) of the comparison region. Similarly, for comparison region AR, the image features of comparison region AR can be obtained by inputting comparison region AR into this machine learning model. In machine learning, for example, supervised learning can be performed using labeled training data of images of roads as input data.

[0141] Then, the comparison unit 122b calculates the similarity of image features between comparison region A1 and comparison region AR.

[0142] Furthermore, the techniques for determining the similarity between the frame image IM1 and the reference image are not limited to this; general techniques such as pattern matching may also be used.

[0143] Here, the comparison region AR may be the entirety of the reference image or an appropriate portion thereof, but it is desirable that it be set in a part of the reference image where vehicles do not normally travel. If the area where vehicles travel is used as the comparison region AR, the similarity may be affected by the traffic conditions on road R, potentially lowering the accuracy of the similarity between the current shooting area of ​​the shooting device 101 and the area shown in the reference image. By setting the comparison region AR to a part where vehicles do not normally travel and comparing the comparison region AR and comparison region A1 in a common area, a similarity that is almost unaffected by the traffic conditions on road R can be obtained.

[0144] Refer to Figure 14 again. The first determination unit 122c determines whether or not the frame image is the subject of analysis based on the similarity obtained from the comparison in step S202a and the first criterion (step S202b).

[0145] For example, comparison region A1 and comparison region AR shown in Figure 15 are generally the same. Therefore, the similarity between comparison region A1 and comparison region AR is, for example, above the threshold included in the first criterion. In this case, since the first criterion according to this embodiment is met, the first determination unit 122c determines that the frame image IM1 is the target of analysis.

[0146] Conversely, if the first criterion according to this embodiment is not met, the first determination unit 122c determines that the frame image IM1 is not subject to analysis.

[0147] Figure 16 shows another example of the frame image IM2 being compared with the reference image. Figure 16(a) shows the frame image IM2, and Figure 16(b) shows the same reference image as Figure 15(b). Frame image IM2 is an example of an image in which the imaging area of ​​the imaging device 101 is shifted to the lower left region compared to Figure 15(a).

[0148] The comparison region A2 and comparison region AR, which are set as common ranges between the frame image IM2 and the reference image shown in Figure 16, are different. The similarity between comparison region A2 and comparison region AR is, for example, below the threshold included in the first criterion. In this case, since the first criterion according to this embodiment is not met, the first determination unit 122c determines that the frame image IM2 is not the subject of analysis.

[0149] Generally, during maintenance of the imaging device 101, the orientation of the imaging device 101 may change due to unintentional contact with the device, and as a result, the imaging area of ​​the imaging device 101 may change.

[0150] For example, if the position of the road or lane is misrecognized, it may incorrectly detect a vehicle driving in the wrong direction. For example, on a two-lane road where the directions of travel are opposite, if one lane is recognized as the other lane, actual This could lead to a vehicle traveling in one lane being perceived as traveling in the other lane. As a result, even if a vehicle is traveling in the correct direction on one of the roads or lanes, it may be judged as traveling in the wrong direction.

[0151] Also, for example, Mu-ga A value is set to convert the length of a predetermined portion in the image into actual distance, and vehicle speed, etc., may be detected using this set value. In such cases, if the orientation of the imaging device 101 changes, the predetermined portion in the image changes, and the length of this predetermined portion may no longer correspond to the set actual distance. As a result, an incorrect vehicle speed may be detected.

[0152] Thus, if the area captured in the frame image deviates from the pre-defined shooting area, errors may occur in the analysis results.

[0153] If the comparison result in step S202a does not meet the first criterion, the area captured in the frame image is deviated from the pre-set shooting area, which may lead to errors in the analysis results. Therefore, if the first criterion is not met, the first determination unit 122c will analyze the frame image IM 2 We have determined that it is not the subject of analysis.

[0154] Refer to Figure 14 again. If it is determined that the frame image is the target of analysis (step S202b; Yes), the analysis unit 123 analyzes the frame image and generates analysis information that includes at least the analysis results (step S203). The first transmission unit 124a transmits the analysis information generated in step S203 to the second information processing device 103 (step S204), and the first information processing is completed.

[0155] If it is determined that the frame image is not the target of analysis (step S202b; No), the second transmission unit 124b generates detection information related to the frame image (step S205). The second transmission unit 124b transmits the detection information generated in step S205 to the second information processing device 103 (step S206), and the first information processing is terminated.

[0156] This first information processing method allows analysis using frame images that capture the appropriate shooting area. Furthermore, the analysis results using frame images that capture the appropriate shooting area can be transmitted to the second information processing device 103. In addition, if the appropriate shooting area is not captured in the frame image, detection information can be transmitted to the second information processing device 103 to inform the user of the second information processing device 103 that a frame image that is not the target of analysis has been detected.

[0157] (Example of the second information processing according to Embodiment 1) Figure 17 is a flowchart showing an example of the second information processing according to Embodiment 1. The second information processing is a process for displaying an image for monitoring roads.

[0158] For example, when the second information processing device 103 receives a start command from the user, it transmits a start command to the imaging device 101 and the first information processing device 102, and starts the second information processing. Then, for example, when the second information processing device 103 receives a stop command from the user, it transmits a stop command to the imaging device 101 and the first information processing device 102, and ends the second information processing. In other words, for example, when the second information processing device 103 receives a start command from the user, it repeatedly executes the second information processing until it receives a stop command from the user. Note that the method of starting or ending the second information processing is not limited to these.

[0159] The receiving unit 131 receives the information transmitted from the transmitting unit 124 (step S301).

[0160] In detail, for example, the receiving unit 131 receives the analysis information or detection information transmitted in step S204 or S205 (see Figure 14).

[0161] The display control unit 133 displays information on the display unit 132 based on the result of the decision, according to the information received in step S301 (step S302).

[0162] In detail, for example as shown in Figure 17, the content determination unit 133a determines whether the content of the information received in step S301 is analysis information or detection information (step S302a).

[0163] If it is determined that the information is analytical information (step S302a; analytical information), the first control unit 133b displays the analytical information received in step S301 on the display unit 132 (step S302b), and terminates the second information processing.

[0164] Figure 18 shows an example of a screen displaying analysis results.

[0165] The screen shown in Figure 18 displays the frame image IM1 and the image captured by the imaging device 101. photograph The ID, the time and location of the frame image IM1's capture, and other information are included. The screen shown in Figure 18 also includes a frame F, which is an indicator for identifying each object contained in the frame image IM1. Frame F is superimposed on the frame image IM1, enclosing each of the vehicles C1 to C4, which are objects contained in the frame image IM1. Furthermore, since no event was detected, the object ID and event type, which are examples of event-related information, are blank in the screen shown in Figure 18.

[0166] The display control unit 133 continues to display the screen illustrated in Figure 18 on the display unit 132. Then, when a predetermined operation is performed, such as placing the cursor over the "close" button on the screen and pressing the mouse button, the display control unit 133 terminates the display of the screen and ends the second information processing.

[0167] The screen displaying the analysis results is not limited to this example; for instance, the information included on the screen may be modified as appropriate.

[0168] Refer to Figure 17 again. If it is determined that the information is detected (step S302a; detected information), the second control unit 133c displays the detected information received in step S301 on the display unit 132 (step S302c), and terminates the second information processing.

[0169] Figure 19 shows an example of a screen that displays detection information.

[0170] The screen shown in Figure 19 displays the frame image IM2 and the reference image, the degree of misalignment of the area shown in the frame image IM2, and the image captured by the imaging device 101 that captured the frame image IM2. photograph This includes the ID, the date and location where the frame image IM2 was taken.

[0171] The display control unit 133 continues to display the screen illustrated in Figure 19 on the display unit 132. Then, when a predetermined operation is performed, such as placing the cursor over the "close" button on the screen and pressing the mouse button, the display control unit 133 terminates the display of the screen and ends the second information processing.

[0172] The screen displaying the detection information is not limited to this example; for instance, the information included on the screen may be modified as appropriate.

[0173] This second information processing method allows the user to be notified of the analysis results using frame images that contain the appropriate shooting area by displaying them on the display unit 132. Furthermore, if the appropriate shooting area is not present in the frame image, detection information can be displayed on the display unit 132 to inform the user that a frame image that is not the target of analysis has been detected.

[0174] (Effects / Actions) As described above, according to Embodiment 1, the road monitoring system 100 comprises a video acquisition unit 121, a target determination unit 122, and a display control unit 133.

[0175] The video acquisition unit 121 acquires video footage of the road. The target determination unit 122 determines whether at least one frame image is the target of analysis based on the result of comparing at least one frame image constituting the video with a reference image and a first criterion.

[0176] The display control unit 133 causes the display unit to display information based on the judgment result. The display control unit 133 includes a first control unit 133b and a second control unit 133c. The first control unit 133b causes the display unit 132 to display the analysis result of at least one frame image that has been determined to be the target of analysis. The second control unit 133c causes the display unit 132 to display detection information indicating that a frame image that is not the target of analysis has been detected.

[0177] This allows the system to determine whether or not a frame image is subject to analysis using the first criterion, and for frame images that are subject to analysis, the analysis results are displayed on the display unit 132 to inform the user. As a result, the user can monitor roads by referring to appropriate analysis results based on the frame images being analyzed. This makes it possible to support efficient road monitoring.

[0178] Furthermore, for frame images that are not to be analyzed, detection information can be displayed on the display unit 132 to inform the user that frame images that are not to be analyzed have been detected. Upon learning this, the user can take appropriate action, such as correcting the orientation of the camera 101 based on the detection information, to ensure that appropriate frame images are captured. After taking action, the road can then be monitored based on the appropriate analysis results. Thus, it becomes possible to support efficient road monitoring.

[0179] According to Embodiment 1, the target determination unit 122 determines whether at least one frame image is the target of analysis based on the result of comparing the image features of at least one frame image with the image features of a reference image and a first criterion.

[0180] Image features can be easily obtained using common techniques. Therefore, it is also easy to determine whether or not an image is suitable for analysis. Consequently, it becomes possible to easily support efficient road monitoring.

[0181] According to Embodiment 1, the image feature is a feature that indicates a region that appears in at least one frame image.

[0182] By using these image features, it is possible to determine whether the area captured in a frame image is shifted to an extent that it is not relevant to the analysis. This allows users to monitor roads by referring to appropriate analysis results based on the frame images that are to be analyzed. Furthermore, by appropriately addressing frame images that are not to be analyzed, road monitoring can then be performed based on appropriate analysis results. Thus, it becomes possible to support efficient road monitoring.

[0183] According to Embodiment 1, the detection information includes information about the frame image used to determine that it is not the target of analysis.

[0184] This allows users to learn about frame images that are not being analyzed, making it easier to take steps to ensure that appropriate frame images are captured. Therefore, it becomes possible to support more efficient road monitoring.

[0185] According to Embodiment 1, the information relating to the frame image includes at least one of the following: (1) imaging device identification information for identifying the imaging device that captured the frame image; (2) at least one of the frame image and the reference image used for the determination; and (3) displacement information relating to the displacement of the area shown in the frame image.

[0186] This allows the user to know at least one of the above (1) to (3), making it easier to take measures to ensure that appropriate frame images are captured. Thus, it becomes possible to support efficient road monitoring.

[0187] According to Embodiment 1, when the detection information includes a frame image and a reference image, the second control unit 133c displays the frame image and the reference image side by side on the display unit 132.

[0188] This allows users to easily compare frame images and reference images used to determine that an image was not for analysis, and easily infer the reason why it was deemed unsuitable. Therefore, it becomes easier to take steps to ensure that appropriate frame images are captured. Consequently, it becomes possible to support efficient road monitoring.

[0189] According to Embodiment 1, the information processing device 102 includes an image acquisition unit 121, a target determination unit 122, and a transmission unit 124.

[0190] The video acquisition unit 121 acquires video footage of the road. The target determination unit 122 determines whether at least one frame image is the target of analysis based on the result of comparing at least one frame image constituting the video with a reference image and a first criterion.

[0191] The transmitting unit 124 transmits information based on the results of the judgment. The transmitting unit 124 includes a first transmitting unit 124a and a second transmitting unit 124b. The first transmitting unit 124a transmits the analysis results of at least one frame image that has been determined to be the subject of analysis. The second transmitting unit 124b transmits detection information indicating that a frame image that is not the subject of analysis has been detected.

[0192] This allows the system to determine whether or not a frame image is suitable for analysis using the first criterion, and for frame images that are suitable, it can transmit the analysis results to the user. Therefore, users can monitor roads by referring to appropriate analysis results based on the frame images being analyzed. This, in turn, makes it possible to support efficient road monitoring.

[0193] Furthermore, for frame images that are not to be analyzed, detection information can be transmitted to inform the user that non-analyzable frame images have been detected. Upon learning this, the user can take appropriate action, such as correcting the orientation of the camera 101 based on the detection information, to ensure that appropriate frame images are captured. After taking action, the road can then be monitored based on the appropriate analysis results. Thus, it becomes possible to support efficient road monitoring.

[0194] <Embodiment 2> Generally, even if the shooting area of ​​the shooting device 101 is appropriate, the captured frame images may differ significantly depending on the shooting environment (e.g., time of day, weather, season, etc.). In such cases, if a common reference image is used to determine whether or not a frame image is suitable for analysis in different shooting environments, an incorrect judgment may be made.

[0195] In this embodiment, we describe an example in which a reference image is selected from a plurality of candidate images, and the frame image is used to determine whether or not it is the target of analysis.

[0196] In this embodiment, for the sake of brevity, the explanation of configurations similar to those in Embodiment 1 will be omitted as appropriate.

[0197] (Example of the configuration of the road monitoring system according to Embodiment 2) The road monitoring system according to Embodiment 2 functionally includes a first information processing device that includes a target determination unit 222 which replaces the target determination unit 122 according to Embodiment 1.

[0198] Figure 20 shows an example of the functional configuration of the target determination unit 222 according to Embodiment 2. The target determination unit 222 includes a reference image storage unit 122a and a comparison unit 122b similar to those in Embodiment 1, and a first determination unit 222c that replaces the first determination unit 122c according to Embodiment 1. Furthermore, the target determination unit 222 includes a candidate storage unit 222a and a selection unit 222b.

[0199] The first determination unit 222c has the same functions as the first determination unit 122c according to Embodiment 1. In addition, the first determination unit 222c stores frame images that it has determined to satisfy the first criterion as candidate images in the candidate storage unit 222a.

[0200] The first determination unit 222c may store in the candidate storage unit 222a, associating the shooting environment of the frame image with the frame image that it has determined to meet the first criterion.

[0201] The shooting environment of a frame image refers to the environment at the time the frame image was taken, and includes, for example, attributes of the time the frame image was taken and the weather conditions at the time the frame image was taken. The attributes of the time of shooting include, for example, at least one such as time of day or season. The weather conditions include, for example, at least one such as weather or climate.

[0202] Furthermore, the shooting environment may be, but is not limited to, at least one of the attributes of the shooting period and / or weather conditions.

[0203] The frame images stored in the candidate storage unit 222a as candidate images may be all of the frame images that are determined to satisfy the first criterion, or they may be only a part of the frame images that are determined to satisfy the first criterion.

[0204] When a portion of frame images that are determined to satisfy the first criterion are stored in the candidate storage unit 222a, the portion of frame images is selected from among the frame images that are determined to satisfy the first criterion according to predetermined rules. For example, the first determination unit 222c may store frame images that are determined to satisfy the first criterion at predetermined time intervals in the candidate storage unit 222a.

[0205] The candidate storage unit 222a is a storage unit in which multiple candidate images are stored in advance. candidate Each image should ideally be associated with the shooting environment in which it was taken. Multiple candidate images should include images associated with different shooting environments.

[0206] The selection unit 222b selects a reference image from among multiple candidate images stored in the candidate storage unit 222a according to predetermined selection criteria.

[0207] The selection criteria are the conditions for selecting a reference image. The selection criteria include, for example, (1) conditions related to the shooting environment (environmental conditions) and (2) conditions related to the timing of switching the reference image (switching timing conditions). The selection criteria may include, but are not limited to, at least one of (1) environmental conditions and (2) switching timing conditions.

[0208] (1) Environmental conditions include, for example, that the reference image is an image that conforms to the current shooting environment (for example, when the most recent frame image was taken). "Conforms" means, for example, that the shooting environment associated with the reference image is partially or completely the same as the shooting environment of the current frame image.

[0209] For example, the selection unit 222b selects a reference image from one or more candidate images associated with at least one frame image and a common shooting environment.

[0210] (2) The switching timing conditions are defined using at least one of a time interval and a work schedule. The work schedule is a schedule of work relating to at least one of the imaging device 101, the first information processing device 102, and the second information processing device 103, and may be entered by the user or obtained from an external device (not shown).

[0211] For example, the selection unit 222b selects a reference image from among multiple candidate images at the time specified in the switching time conditions, that is, at the time to switch the reference image.

[0212] Aside from these, the road monitoring system according to this embodiment may be functionally configured in a manner similar to the road monitoring system 100 according to Embodiment 1. Physically, the road monitoring system according to this embodiment may be configured in a manner similar to the road monitoring system 100 according to Embodiment 1.

[0213] (Example of operation of the road monitoring system according to Embodiment 2) The road monitoring system according to this embodiment performs road monitoring processing in the same manner as the road monitoring system according to Embodiment 1. The road monitoring processing according to this embodiment includes the same imaging processing and second information processing as in Embodiment 1, and a first information processing that is different from that of Embodiment 1.

[0214] Figure 21 is a flowchart showing an example of the first information processing according to Embodiment 2. 1st information The process includes steps S201 to S206, similar to those in Embodiment 1. 1st information The process further includes step S301, which is performed following step S201, and step S302, which is performed following step S204. Except for these, the embodiment described herein 1st information The process relates to Embodiment 1. 1st information The process can be the same as before.

[0215] Following step S201, the selection unit 222b selects a reference image from among the multiple candidate images stored in the candidate storage unit 222a according to predetermined selection conditions (step S301).

[0216] Figure 22 is a flowchart showing an example of the selection process (step S301) according to Embodiment 2.

[0217] The selection unit 222b determines whether the environmental conditions of the reference image and the frame image are compatible (step S301a).

[0218] In detail, for example, the reference image stored in the reference image storage unit 122a according to this embodiment is associated with the time the reference image was taken and the weather conditions (e.g., the weather) at the time the reference image was taken. Furthermore, for example, the frame image included in the image information according to this embodiment is further associated with the weather conditions (e.g., the weather) at the time the frame image was taken.

[0219] The selection unit 222b determines whether the environmental conditions associated with the reference image in the reference image storage unit 122a and the environmental conditions associated with the frame image in the image information acquired in step S201 are compatible.

[0220] For example, if the reference image and the frame image were taken during the same time period and the associated weather is the same, the selection unit 222b determines that the environmental conditions are suitable. This time period may be predetermined to correspond to, for example, daytime or nighttime. Conversely, if the reference image and the frame image were taken during different time periods or the associated weather is different, the selection unit 222b determines that the environmental conditions are not suitable.

[0221] If it is determined that the environmental conditions are suitable (step S301a; Yes), the selection unit 222b determines whether or not the switching timing conditions are met (step S301b).

[0222] More specifically, if the switching timing conditions are defined using time, the selection unit 222b should determine whether the current time is included in the switching timing conditions. For example, if the current time is included in the switching timing conditions, the selection unit 222b should determine that the switching timing conditions are met. If the current time is not included in the switching timing conditions, the selection unit 222b should determine that the switching timing conditions are not met.

[0223] The time included in the switching time conditions may be defined as a time range. Furthermore, the selection unit 222b may determine whether the current time is included in the switching time conditions based on whether it falls within a predetermined time range relative to the time included in the switching time conditions.

[0224] For example, if the switching timing condition is defined using a time interval, the selection unit 222b may determine whether the switching timing condition is met based on the elapsed time since the time the current reference image was stored in the reference image storage unit 122a (storage time). For example, if the elapsed time is equal to or greater than the time interval, the selection unit 222b may determine that the switching timing condition is met. If the elapsed time is less than the time interval, the selection unit 222b may determine that the switching timing condition is not met. The storage time may be stored in the reference image storage unit 122a, for example.

[0225] Furthermore, for example, if the transition timing conditions are defined using a work schedule, the selection unit 222b may determine whether the transition timing conditions are met based on whether the current time is a predetermined time before the work start time indicated in the work schedule. For example, if it is a predetermined time before the work start time, the selection unit 222b may determine that the transition timing conditions are met. If it is not a predetermined time before the work start time, the selection unit 222b may determine that the transition timing conditions are not met.

[0226] If it is determined that the switching timing conditions are not met (step S301b; No), the selection unit 222b returns to the first information processing (see Figure 21).

[0227] If it is determined that the environmental conditions are not suitable (step S301a; No), or if it is determined that the switching timing conditions are met (step S301b; Yes), the selection unit 222b selects a reference image (step S301c).

[0228] In detail, for example, the selection unit 222b selects a reference image from among multiple candidate images stored in the candidate storage unit 222a. At this time, the selection unit 222b selects as the reference image a candidate image whose associated environmental conditions match those of the current frame image. Furthermore, if there are multiple candidate images whose environmental conditions match those of the current frame image, the selection unit 222b selects as the reference image, for example, the candidate image whose shooting date is closest to the present (most recent).

[0229] The selection unit 222b stores the reference image selected in step S301c in the reference image storage unit 122a (step S301d), and returns to the first information processing (see Figure 21).

[0230] Refer to Figure 21 again. Following step S204, the first determination unit 222c stores the frame image that it has determined to satisfy the first criterion as a candidate image in the candidate storage unit 222a (step S302), and terminates the first information processing.

[0231] (Effects / Actions) According to Embodiment 2, the road monitoring system further includes a selection unit 222b that selects a reference image from among a plurality of candidate images according to selection conditions that define the conditions for selecting a reference image.

[0232] This allows you to select the appropriate image from among multiple candidate images. reference Since images can be selected, it becomes possible to more accurately determine whether or not a frame image is the target of analysis. Therefore, it becomes possible to further support efficient road monitoring.

[0233] According to Embodiment 2, the selection criteria include at least one of (1) conditions relating to the shooting environment and (2) conditions relating to the timing of switching the reference image.

[0234] This allows for the selection of an appropriate image as a reference image from among multiple candidate images according to at least one of (1) environmental conditions or (2) switching timing conditions, thereby enabling a more accurate determination of whether or not a frame image is the target of analysis. Consequently, it becomes possible to further support efficient road monitoring.

[0235] According to Embodiment 2, the selection criteria include conditions related to the shooting environment. Furthermore, the multiple candidate images include images associated with different shooting environments. The selection unit 222b selects a reference image from one or more candidate images associated with a common shooting environment with at least one frame image.

[0236] This allows for the selection of appropriate images as reference images based on the shooting environment, enabling a more accurate determination of whether or not a frame image is the target of analysis. Consequently, it becomes possible to further support efficient road monitoring.

[0237] According to Embodiment 2, the shooting environment includes at least one of the attributes of the shooting time and weather conditions.

[0238] Generally, frame images can differ depending on the attributes of the time of year the image was taken. For example, the frame image may differ because the brightness varies depending on the time of day. Also, the frame image may differ because the density of vegetation varies depending on the season. The same applies to weather conditions.

[0239] By including at least one of the following attributes in the shooting environment: the shooting time and weather conditions, it becomes possible to select an appropriate image as a reference image based on at least one of these attributes. Therefore, it becomes possible to more accurately determine whether or not a frame image is the target of analysis. Consequently, it becomes possible to further support efficient road monitoring.

[0240] According to Embodiment 2, the selection criteria include a condition regarding the timing of switching the reference image. The selection unit 222b selects a reference image from among a plurality of candidate images at the time of switching the reference image.

[0241] This allows for the selection of appropriate images as reference images at predetermined times. Therefore, it becomes possible to more accurately determine whether a frame image is suitable for analysis. Consequently, it becomes possible to further support efficient road monitoring.

[0242] According to Embodiment 2, the conditions regarding the timing of switching the reference image are defined using at least one of a time interval and a work schedule.

[0243] This allows for the selection of appropriate images as reference images at predetermined time intervals or according to the work schedule. Therefore, it becomes possible to more accurately determine whether a frame image is suitable for analysis. Consequently, it becomes possible to further support efficient road monitoring.

[0244] <Embodiment 3> In Embodiment 1, if it is determined even once that the first criterion is not met, Out Information is displayed. However, for example, lens nearby Birds or insects may appear in the frame image, or the frame image may be affected by communication problems, etc. The statue It may become temporarily distorted. In such cases, even if the imaging area of ​​the imaging device 101 is appropriate, Frame image and The similarity to the reference image decreases, Frame image The frame image will be deemed ineligible for analysis. If detection information is displayed when the frame image is temporarily deemed ineligible for analysis, it could reduce the efficiency of road monitoring.

[0245] In this embodiment, in order to suppress the display of such detection information, This section describes a configuration that reduces the possibility of frame images being temporarily deemed ineligible for analysis.

[0246] In this embodiment, for the sake of brevity, the explanation of configurations similar to those in Embodiment 1 will be omitted as appropriate.

[0247] (Example of the configuration of the road monitoring system according to Embodiment 3) The road monitoring system according to Embodiment 3 functionally includes a first information processing device that includes a target determination unit 322, which replaces the target determination unit 122 according to Embodiment 1.

[0248] Figure 23 shows the target determination unit according to Embodiment 3. 3It is a diagram showing a functional configuration example of 22. The target determination unit 322 includes a reference image storage unit 122a and a comparison unit 122b similar to those in the first embodiment, and a first determination unit 322c replacing the first determination unit 122c according to the first embodiment. Further, the target determination unit 322 includes a second determination unit 322d.

[0249] Similar to the first determination unit 122c according to the first embodiment, the first determination unit 322c determines whether the result compared by the comparison unit 122b satisfies the first criterion for each of the time-series frame images. However, the first determination unit 322c is different from the first determination unit 122c according to the first embodiment in that it does not make a determination as to whether each of the frame images is an analysis target based on this determination result.

[0250] <^ The second determination unit 322d determines whether each of the frame images is an analysis target based on the determination result of the first determination unit 322c and the second criterion. That is, the second determination unit 322d determines whether each of the time-series frame images is an analysis target based on the determination result of whether each satisfies the first criterion and the second criterion.

[0251] The second criterion is defined using at least one of the number of times, frequency, and ratio of determinations that the first criterion is not satisfied for the time-series frame images.

[0252] Except for these, the road monitoring system according to this embodiment may be functionally configured in substantially the same manner as the road monitoring system 100 according to the first embodiment. Also, physically, the road monitoring system according to this embodiment may be configured in the same manner as the road monitoring system 100 according to the first embodiment.

[0253] (Operation example of the road monitoring system according to Embodiment 3) The road monitoring system according to this embodiment executes road monitoring processing in the same manner as the road monitoring system according to the first embodiment. The road monitoring processing according to this embodiment includes a shooting process and a second information process similar to those in the first embodiment, and a first information process different from that in the first embodiment.

[0254] Figure 24 is a flowchart showing an example of the first information processing according to Embodiment 3. The first information processing according to this embodiment includes an analysis target determination process (step S402) that replaces the analysis target determination process (step S202) according to Embodiment 1. Except for this point, the road monitoring process according to this embodiment may be the same as that of Embodiment 1. Figure 24 is a diagram showing the part of the flowchart of the analysis target determination process (step S402) according to this embodiment that differs from the analysis target determination process (step S202) according to Embodiment 1.

[0255] The analysis target determination process (step S402) includes step S202a, which is the same as in Embodiment 1. Subsequently, the first determination unit 322c determines whether the similarity obtained from the comparison in step S202a satisfies the first criterion (step S402b).

[0256] If it is determined that the first criterion is met (step S402b; Yes), the same processing from step S203 onwards as in Embodiment 1 is performed.

[0257] If it is determined that the first criterion is not met (step S402b; No), the second determination unit 322d determines whether the result of the determination in step S402b meets the second criterion (step S402c).

[0258] For example, the second criterion is one of the following: the number of times the first criterion is not met within a specified period is below a threshold; the frequency is below a threshold; or the percentage is below a threshold.

[0259] The second judgment unit 322d determines that if the judgment result in step S402b does not meet the second criterion, the frame image is the subject of analysis. be The second determination unit 322d determines that the frame image is not subject to analysis if the determination result in step S402b satisfies the second criterion.

[0260] If it is determined that the second criterion is not met (step S402c; No), steps S205 to S206, similar to those in Embodiment 1, are executed. If it is determined that the second criterion is met (step S402c; Yes), the second determination unit 322d terminates the first information processing.

[0261] Thus, in the first information processing according to this embodiment, if the number of times, frequency, or percentage of times the first criterion is not met within a predetermined period is below a predetermined threshold, the generation and transmission of detection information can be prevented.

[0262] (Effects / Actions) According to this embodiment, the video is composed of a time-series of frame images, each containing at least one frame image. The target determination unit 322 includes a first determination unit 322c and a second determination unit 322d.

[0263] The first determination unit 322c determines whether the result of comparing each frame image in the time series with a reference image satisfies the first criterion. The second determination unit 322d determines whether each frame image is a target for analysis based on the result of determining whether each frame image in the time series satisfies the first criterion and the second criterion.

[0264] This allows us to determine whether each frame image in the time series is a target for analysis based on whether the result of determining whether each frame image meets the first criterion meets the second criterion. Furthermore, if each frame image is not a target for analysis, the detection information will not be displayed on the display unit 132.

[0265] In general, when a communication failure occurs, or when insects or birds are prominently featured in the frame image, frame images that do not meet the first criterion may be temporarily generated. In such cases, the shooting area of ​​the shooting device 101 may not have actually changed to the extent that it does not meet the first criterion, and informing the user of the detection information would require the user to take the time to verify the detection information.

[0266] In this embodiment, as described above, since the detection information can be prevented from being displayed on the display unit 132, the labor of the user can be reduced. Therefore, it becomes possible to further support efficient road monitoring.

[0267] According to this embodiment, the second criterion is defined using at least one of the number, frequency, and ratio of times that the first criterion is determined not to be satisfied for time-series frame images.

[0268] When the number, frequency, and ratio of times that the first criterion is determined not to be satisfied are small, as described above, there is a high possibility that a frame image that does not satisfy the first criterion is temporarily generated. Therefore, while suppressing a decrease in the detection accuracy of frame images that do not satisfy the first criterion, the labor of the user as described above can be reduced. Therefore, it becomes possible to further support efficient road monitoring.

[0269] <Embodiment 4> In Embodiment 1, an example in which the road monitoring system includes one imaging device 101, one first information processing device 102, and one second information processing device 103 has been described. However, the road monitoring system may include a plurality of imaging devices 101 installed to image different locations on the road. The road in this case may be a specific road such as the X Expressway, or may include a plurality of specific roads such as the X Expressway and the Y Expressway. Note that the road may include a pedestrian passage.

[0270] Further, the road monitoring system may include a plurality of first information processing devices 102. In this case, each of the first information processing devices 102 may be connected via a network N so as to be able to transmit and receive information to and from one or more imaging devices 101. Thereby, each of the first information processing devices 102 can detect the road state, which is the state of an object on the road, by processing an image captured by one or more imaging devices 101.

[0271] Note that such a modification can be applied not only to Embodiment 1 but also to Embodiments 2 to 3.

[0272] Figure 25 shows an example configuration of a road monitoring system 400 according to Embodiment 4. The road monitoring system 400 comprises a plurality of imaging devices 101_1_1 to 101_1_M1, 101_X_1 to 101_X_MX, one or more first information processing devices 102_1 to 102_X, and a second information processing device 103. Each of M1, MX, and X is an integer of 1 or more.

[0273] Multiple imaging devices 101_1_1~101_1_M1, 101_X_1~101_X_MX are installed to photograph different locations on the road. Each of the multiple imaging devices 101_1_1~101_1_M1, 101_X_1~101_X_MX corresponds to, for example, the imaging device 101 according to Embodiment 1. Therefore, each of the multiple imaging devices 101_1_1~101_1_M1, 101_X_1~101_X_MX photographs the road and generates video. As described above, the video consists of time-series frame images, i.e., multiple frame images.

[0274] If X is 2 or more, each of the first information processing devices 102_1 to 102_X corresponds to the first information processing device 102 according to Embodiment 1. For example, in each of the first information processing devices 102_1 to 102_X according to this embodiment, the video acquisition unit 121 acquires multiple videos of the road taken by each of the multiple shooting devices 101_1_1 to 101_1_M1 and 101_X_1 to 101_X_MX.

[0275] The target determination unit 122 determines whether each of the multiple frame images is a target for analysis, based on, for example, the result of comparing each of the multiple frame images constituting each of the multiple videos acquired by the video acquisition unit 121 with a reference image, and a first criterion.

[0276] (Effects / Actions) As described above, according to Embodiment 4, the road monitoring system 400 further comprises a plurality of shooting devices 101_1_1~101_1_M1, 101_X_1~101_X_MX that photograph the road and generate video.

[0277] The video acquisition unit 121 acquires multiple video images of the road taken by each of the multiple shooting devices 101_1_1~101_1_M1, 101_X_1~101_X_MX. The target determination unit 122 determines whether each of the multiple frame images is a target for analysis based on the result of comparing each of the multiple frame images constituting each of the multiple video images with a reference image, and a first criterion.

[0278] This allows for the determination of whether or not the video footage generated by multiple imaging devices 101_1_1~101_1_M1, 101_X_1~101_X_MX is suitable for analysis using the first criterion. For frame images that are suitable for analysis, the analysis results are displayed on the display unit 132 to inform the user. Therefore, the user can monitor a wide area of ​​road by referring to the appropriate analysis results based on the frame images being analyzed. This, in turn, enables efficient road monitoring.

[0279] Furthermore, for frame images that are not to be analyzed, detection information can be displayed on the display unit 132 to inform the user that frame images that are not to be analyzed have been detected. Upon learning this, the user can take appropriate action, such as correcting the orientation of the camera 101 based on the detection information, to ensure that a wide range of frame images are properly captured. After taking action, a wide range of roads can be monitored based on appropriate analysis results. Thus, it becomes possible to support efficient road monitoring.

[0280] The embodiments and modifications of the present invention have been described above with reference to the drawings, but these are merely examples of the present invention, and various other configurations can also be adopted.

[0281] Furthermore, while the flowcharts used in the above description show multiple steps (processes) in sequence, the execution order of the steps performed in each embodiment is not limited to the order in which they are described. In each embodiment, the order of the illustrated steps can be changed to the extent that it does not impede the content. Also, the above embodiments and modifications can be combined to the extent that their content is not contradictory.

[0282] Some or all of the above embodiments may also be described as follows, but are not limited to the following:

[0283] 1. A means for acquiring video footage of the road, A target determination means that determines whether or not the at least one frame image is the subject of analysis based on the result of comparing the at least one frame image constituting the video with a reference image and a first criterion, The system includes a display control means for displaying information based on the result of the aforementioned determination on a display means, The display control means is A first control means that causes the display means to display the analysis results of the at least one frame image that has been determined to be the subject of analysis, Includes a second control means that causes the display means to display detection information indicating that a frame image that is not the subject of analysis has been detected. Road monitoring system. 2. The target determination means determines whether the at least one frame image is the target of analysis based on the result of comparing the image features of the at least one frame image with the image features of the reference image and the first criterion. 1. The road monitoring system described in 1. 3. The image feature is a feature that indicates a region that appears in the at least one frame image. The road monitoring system described in 2. 4. The system further comprises a selection means for selecting the reference image from among a plurality of candidate images in accordance with selection conditions that define the conditions for selecting the reference image. A road monitoring system as described in any one of the following three items. 5. The selection criteria include at least one of the following: (1) conditions relating to the shooting environment, and (2) conditions relating to the timing of switching the reference image. The road monitoring system described in 4. 6. The selection criteria include the conditions relating to the shooting environment. The aforementioned multiple candidate images include images associated with different shooting environments, The selection means selects the reference image from one or more candidate images associated with the shooting environment common to the at least one frame image. The road monitoring system described in 5. 7. The shooting environment includes at least one of the attributes of the shooting time and weather conditions. The road monitoring system described in 6. 8. The selection criteria include conditions relating to the timing of switching the reference image. The selection means selects the reference image from among the multiple candidate images when it is time to switch the reference image. A road monitoring system as described in any one of items 5 through 7. 9. The conditions for when to switch the aforementioned reference image shall be defined by at least one of the time interval and / or work schedule. The road monitoring system described in 8. 10. The video consists of a time-series of frame images, including at least one frame image. The means for determining the target is, A first determination means for determining whether the result of comparing each of the frame images in the time series with the reference image satisfies the first criterion, The system includes a second determination means for determining whether each of the frame images in the time series is subject to analysis, based on the result of determining whether each of the frame images in the time series satisfies the first criterion and a second criterion. A road monitoring system as described in any one of items 1 through 9. 11. The second criterion is defined using at least one of the number, frequency, or percentage of times the frame images in the time series are determined not to meet the first criterion. The road monitoring system described in 10. 12. Further comprising multiple shooting means for photographing the road and generating video, The video acquisition means acquires a plurality of videos of the road, each of the plurality of shooting means. The target determination means determines whether each of the multiple frame images is the target of analysis based on the result of comparing each of the multiple frame images constituting each of the multiple videos with a reference image, and the first criterion. A road monitoring system as described in any one of items 1 through 11. 13. The detection information includes information about the frame image used to determine that it was not subject to analysis. A road monitoring system as described in any one of items 1 through 12. 14. The information relating to the frame image includes at least one of the following: (1) imaging device identification information for identifying the imaging device that captured the frame image; (2) at least one of the frame image and the reference image used for the determination; and (3) displacement information relating to the displacement of the area shown in the frame image. The road monitoring system described in 13. 15. The second control means causes the frame image and the reference image to be displayed side by side on the display means when the detection information includes the frame image and the reference image. The road monitoring system described in 14. 16. A means for acquiring video footage of a road, A target determination means that determines whether or not the at least one frame image is the subject of analysis based on the result of comparing the at least one frame image constituting the video with a reference image and a first criterion, The system includes a transmission means for transmitting information based on the results of the aforementioned determination, The aforementioned transmission means is A first transmission means for transmitting the analysis results of the at least one frame image that has been determined to be the subject of analysis, Includes a second transmission means that transmits detection information indicating that a frame image not subject to analysis has been detected. Information processing device. 17. One or more computers, We obtained video footage of the road, Based on the results of comparing at least one frame image constituting the video with a reference image, and the first criterion, it is determined whether or not the at least one frame image is the subject of analysis. This includes displaying information based on the results of the aforementioned determination on a display means, The display includes displaying the analysis results of at least one frame image that has been determined to be the subject of analysis on the display means, and displaying detection information on the display means indicating that a frame image that is not the subject of analysis has been detected. Road monitoring method. 18. On one or more computers, We obtained video footage of the road, Based on the results of comparing at least one frame image constituting the video with a reference image, and the first criterion, it is determined whether or not the at least one frame image is the subject of analysis. This is intended to cause the information based on the results of the aforementioned judgment to be displayed on the display means. To display the above means, This includes displaying the analysis results of at least one frame image that has been determined to be the subject of analysis on the display means, and displaying detection information on the display means indicating that a frame image that is not the subject of analysis has been detected. program. 19. On one or more computers, We obtained video footage of the road, Based on the results of comparing at least one frame image constituting the video with a reference image, and the first criterion, it is determined whether or not the at least one frame image is the subject of analysis. This is intended to cause the information based on the results of the aforementioned judgment to be displayed on the display means. To display the above means, This includes displaying the analysis results of at least one frame image that has been determined to be the subject of analysis on the display means, and displaying detection information on the display means indicating that a frame image that is not the subject of analysis has been detected. A recording medium on which a program is stored. [Explanation of Symbols]

[0284] 100,400 Road Monitoring Systems 101 Imaging device 102 First Information Processing Device 103 Second Information Processing Device 121 Video Acquisition Unit 122 Target Determination Unit 122a Reference image storage unit 122b Comparison section 122c First Judgment Division 123 Analysis Department 124 Transmitter 124a First Transmitter 124b Second Transmitter 131 Receiving Unit 132 Display section 133 Display Control Unit 133a Content Judgment Section 133b First control unit 133c Second Control Unit 222,322 Target determination unit 222a Candidate storage unit 222b Selection Department 222c,322c 1st Judgment Department 322d Second Judgment Department

Claims

1. A means of acquiring video footage of a road, A target determination means for determining whether at least one frame image is the target of analysis based on whether the similarity between at least one frame image constituting the video and a reference image is above a threshold, The system includes a display control means for displaying information based on the result of the aforementioned determination on a display means, The display control means is A first control means that causes the display means to display the analysis results of the at least one frame image that has been determined to be the subject of analysis, The system includes a second control means that causes the display means to display detection information indicating that a frame image not subject to analysis has been detected. Road monitoring system.

2. The target determination means determines whether the at least one frame image is the target of analysis based on whether the similarity between the image features of the at least one frame image and the image features of the reference image is equal to or greater than a threshold. The road monitoring system according to claim 1.

3. The system further comprises a selection means for selecting the aforementioned reference image from among a plurality of candidate images in accordance with selection conditions that define the conditions for selecting the aforementioned reference image, The selection criteria include at least one of the following: (1) conditions relating to the shooting environment, and (2) conditions relating to the timing of switching the reference image. The road monitoring system according to claim 1 or 2.

4. The system further comprises a selection means for selecting the aforementioned reference image from among a plurality of candidate images in accordance with selection conditions that define the conditions for selecting the aforementioned reference image, The selection criteria include conditions relating to the timing of switching the reference image. The selection means selects the reference image from among the multiple candidate images when it is time to switch the reference image. The road monitoring system according to claim 1 or 2.

5. The aforementioned video consists of a time-series of frame images, including at least one frame image. The means for determining the target is, A first determination means for determining whether the similarity between each of the frame images in the time series and the reference image is equal to or greater than the threshold, The system includes a second determination means for determining whether each of the frame images is the subject of analysis, based on at least one of the number, frequency, or percentage of times the frame images in the time series are determined not to satisfy the condition that the similarity is equal to or greater than the threshold. The road monitoring system according to claim 1 or 2.

6. The system further comprises multiple shooting means for photographing the aforementioned road and generating video footage. The video acquisition means acquires a plurality of videos of the road, each of the plurality of shooting means. The target determination means determines whether each of the multiple frame images constituting each of the multiple videos is the target of analysis based on whether the similarity between each of the multiple frame images constituting each of the multiple videos and the reference image is above a threshold. The road monitoring system according to claim 1 or 2.

7. A means of acquiring video footage of a road, A target determination means for determining whether at least one frame image is the target of analysis based on whether the similarity between at least one frame image constituting the video and a reference image is above a threshold, The system includes a transmission means for transmitting information based on the results of the aforementioned determination, The aforementioned transmission means is A first transmission means for transmitting the analysis results of the at least one frame image that has been determined to be the subject of analysis, Includes a second transmission means that transmits detection information indicating that a frame image not subject to analysis has been detected. Information processing device.

8. One or more computers, We obtained video footage of the road, Based on whether the similarity between at least one frame image constituting the video and a reference image is above a threshold, it is determined whether or not the at least one frame image is the subject of analysis. This includes displaying information based on the results of the aforementioned determination on a display means, The display includes displaying the analysis results of at least one frame image that has been determined to be the subject of analysis on the display means, and displaying detection information on the display means indicating that a frame image that is not the subject of analysis has been detected. Road monitoring method.

9. On one or more computers, We obtained video footage of the road, Based on whether the similarity between at least one frame image constituting the video and a reference image is above a threshold, it is determined whether or not the at least one frame image is the subject of analysis. This is intended to cause the information based on the results of the aforementioned judgment to be displayed on the display means. To display the above means, This includes displaying the analysis results of at least one frame image that has been determined to be the subject of analysis on the display means, and displaying detection information on the display means indicating that a frame image that is not the subject of analysis has been detected. program.

Images