Information processor, in-railroad-crossing risk detection system, neglected object detection system, train side monitoring system, road monitoring system, and information processing method

Abstract

To detect an unknown object using a normal image generated while excluding a known object.SOLUTION: An information processor is configured using a computer including an arithmetic device for executing predetermined arithmetic processing and a storage device capable of being accessed by the arithmetic device. The information processor comprises: a known object detection section where the arithmetic device outputs a known object detection result obtained by detecting a known object from data acquired by a sensor; a normal data generation section where the arithmetic device synthesizes areas other than the known object of data obtained at a plurality of time points by the sensor using the known object detection result to generate normal data; and a first difference detection section where the arithmetic device compares the normal data with data obtained by the sensor and outputs a difference detection result obtained by detecting a difference between the normal data and the data obtained by the sensor as an unknown object.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device and an information processing method. [Background technology]

[0002] There are systems that detect abnormalities by comparing images taken by cameras with images taken under normal conditions. For example, a railroad crossing hazard detection system, which aims to support safe railway operation, detects objects such as pedestrians or fallen objects remaining on the railroad crossing after the barrier has been lowered, and notifies the train or control center of any abnormalities within the crossing. A useful method for a railroad crossing hazard detection system is to use an analysis device installed near the crossing or in a remote location to detect objects from images taken by a camera installed near the crossing.

[0003] Background art in this technical field includes Patent Document 1 (Patent Publication No. 2021-183429), Patent Document 2 (Patent Publication No. 2023-174362), and Patent Document 3 (Patent Publication No. 2021-33407).

[0004] Patent document 1 describes an abnormality notification system that has an information processing device arranged in or near a monitoring area of ​​a railroad crossing through which railway vehicles traveling on railroad tracks or equivalent tracks pass, and a monitoring center device that monitors the railway vehicles, and that detects abnormalities in the monitoring area and notifies the railway vehicles, wherein the information processing device has a detection means that detects abnormal conditions that may cause abnormalities when the railway vehicle passes through the crossing based on images of the railroad crossing, and a first notification means that notifies the monitoring center device of abnormality information corresponding to the abnormal condition detected by the detection means, and the monitoring center device has a determination means that determines the railway vehicle to notify of the abnormality information based on the abnormality information notified from the information processing device and location information notified from the railway vehicle, and a second notification means that notifies the railway vehicle determined by the determination means of the abnormality information.

[0005] Furthermore, Patent Document 2 describes an image anomaly detection method including a feature amount map generating step of generating a feature amount map from an input image, a differential feature amount map generating step of generating a differential feature amount map based on the difference between a feature amount map generated using a set of feature amount maps generated from images in a normal state and the feature amount map of the input image, and an anomaly map output step of outputting an anomaly map using a plurality of the differential feature amount maps generated at a plurality of feature amount levels.

[0006] Furthermore, Patent Document 3 describes an object extraction device that extracts objects from each frame of a video by performing background difference calculations using a background model obtained by statistically processing past frames as a background, the object extraction device comprising: means for detecting an object area from each frame; means for acquiring displacement information of the object area between frames; means for determining a background difference threshold to be used in the current frame by moving the background difference threshold used in the previous frame to a corresponding pixel area of ​​the current frame based on the displacement information; means for classifying each pixel of the current frame as either background or foreground by performing background difference calculations that compare the difference between the background model and the current frame with the updated background difference threshold; and means for updating the background model based on the current frame. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2021-183429 [Patent Document 2] Japanese Patent Application Publication No. 2023-174362 [Patent Document 3] Patent Publication No. 2021-33407 Summary of the Invention [Problem to be solved by the invention]

[0008] The technology described in Patent Document 1 mentioned above detects vehicles or people that have entered a railroad crossing or its surrounding area, but since people and vehicles are not the only objects that pose a danger within a railroad crossing, it is necessary to detect unknown objects.

[0009] An object of the present invention is to provide an information processing apparatus and method for generating a high-quality normal image from which known objects are excluded, and detecting unknown objects using the generated normal image. [Means for solving the problem]

[0010] A representative example of the invention disclosed in the present application is as follows: That is, the invention is configured by a computer having an arithmetic unit that executes predetermined arithmetic processing and a storage device accessible by the arithmetic unit, wherein the arithmetic unit comprises a known object detection unit that outputs a known object detection result in which a known object is detected from data acquired by a sensor, a normal data generation unit that uses the known object detection result to synthesize areas other than known objects in data acquired by the sensor at multiple times to generate the normal data, and a first difference detection unit that compares the normal data with data acquired by the sensor and outputs a difference detection result in which a difference between the normal data and the data acquired by the sensor is detected as an unknown object.

[0011] Furthermore, an example information processing device of the present invention is characterized in that it includes a detection result integration unit that integrates the known object detection result and the difference detection result and outputs at least one of an object detection result that distinguishes between the detected known object and the detected unknown object, or an object detection result that does not distinguish between the detected known object and the detected unknown object.

[0012] In addition, in one example of the information processing device of the present invention, the normal data generation unit outputs a normal data generation completion flag indicating the generation of normal data, and the first difference detection unit detects a difference after the normal data generation completion flag indicates the generation of normal data.

[0013] In addition, in one example of the information processing device of the present invention, the first difference detection unit has a feature extractor that extracts features from the normal data and the data acquired by the sensor, and the feature extractor is trained with images of multiple types of objects different from the data acquired by the sensor.

[0014] In addition, in one example of the information processing device of the present invention, the feature extractor has a neural network that extracts features from the normal data and the data acquired by the sensor, and the first difference detection unit generates a difference map indicating an object region using an output from an intermediate layer of the neural network, applies an averaging filter with a kernel size of 1 or more to the difference map, and then performs threshold processing to detect the difference.

[0015] In addition, in one example of an information processing device of the present invention, the normal data generation unit excludes data areas with differences indicated by the difference detection results, or sets a lower contribution to synthesis for data areas with higher difference detection scores output by the first difference detection unit, and synthesizes areas other than known objects in data acquired by the sensor at multiple times to generate normal data in which areas other than the detected known objects and unknown objects match.

[0016] Furthermore, an example of an information processing device of the present invention is characterized in that it includes a normal data control unit that instructs the normal data generation unit to regenerate normal data in response to at least one of a global change in the data frame and information input from the outside.

[0017] Furthermore, an example of an information processing device of the present invention includes a normal data buffer that stores the normal data generated by the normal data generation unit and sends normal data suitable for instructions from the normal data generation unit to the normal data generation unit, and the normal data generation unit updates the normal data stored in the normal data buffer.

[0018] Furthermore, an example of an information processing device of the present invention includes a normal data buffer that stores the normal data generated by the normal data generation unit, and the calculation device includes a second difference detection unit that detects differences between multiple pieces of the normal data, and the second difference detection unit compares normal data at different times and outputs an aging detection result that detects the differences between the normal data at the different times as aging changes.

[0019] In addition, in the information processing device according to one example of the present invention, the data is image data or point cloud data.

[0020] Furthermore, a railroad crossing danger detection system according to one example of the present invention is characterized in that it detects objects within the railroad crossing using the information processing device described above.

[0021] Furthermore, an abandoned object detection system according to one example of the present invention is characterized in that it detects abandoned objects in public spaces using the information processing device described above.

[0022] Furthermore, a train side monitoring system according to an embodiment of the present invention is characterized in that the above-mentioned information processing device monitors the train and the situation around the train when it arrives or departs from a station.

[0023] Furthermore, a road monitoring system according to an embodiment of the present invention is characterized in that an obstacle on the road is detected by the information processing device described above. [Effects of the Invention]

[0024] According to one aspect of the present invention, it is possible to generate high-quality normal data and detect unknown objects with high accuracy. Problems, configurations, and effects other than those described above will become apparent from the following description of the preferred embodiment of the present invention. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a diagram showing the overall configuration of a railroad crossing hazard detection system according to a first embodiment. [Figure 2]FIG. 10 is a diagram showing the overall configuration of an abandoned object detection system according to a second embodiment. [Figure 3] FIG. 10 is a diagram showing the overall configuration of a train side monitoring system according to a third embodiment. [Figure 4] FIG. 10 is a diagram showing the overall configuration of an abandoned object detection system according to a fourth embodiment. [Figure 5] FIG. 10 is a diagram showing the overall configuration of a road monitoring system according to a fifth embodiment. [Figure 6] FIG. 13 is a diagram showing the overall configuration of a road monitoring system according to a sixth embodiment. [Figure 7] FIG. 4 is a diagram illustrating an example of a process executed by a normal image generating unit according to the first embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of a process executed by a difference detection unit according to the first embodiment. [Figure 9] FIG. 10 is a diagram illustrating an example of a process in which the detection result integration unit in the first embodiment outputs an object detection result in which a known object and an unknown object are distinguished from each other. [Figure 10] FIG. 10 is a diagram illustrating an example of a process in which the abandoned object detection system of the second embodiment monitors a room under a plurality of conditions. [Figure 11] FIG. 11 is a diagram illustrating an example of a process executed by the train side monitoring system according to the third embodiment. [Figure 12] FIG. 10 is a diagram illustrating an example of a process executed by the abandoned object detection system according to the fourth embodiment. [Figure 13] FIG. 10 is a diagram showing an example of an aging detection process executed by the road monitoring system of the fifth embodiment. [Figure 14] FIG. 20 is a diagram illustrating an example of a process executed by the road monitoring system according to the sixth embodiment. [Figure 15] 1 is a flowchart of a process executed by an information processing device (railroad crossing danger detection system) according to a first embodiment. [Figure 16] 10 is a detailed flowchart of a normal image generating step according to the first embodiment. [Figure 17] 10 is a detailed flowchart of another process of the normal image generating step in the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0026] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0027] Example 1 In Example 1, a railroad crossing hazard detection system, which is one application of the information processing device and method of the present invention, will be described. The railroad crossing hazard detection system detects pedestrians, fallen objects, and other objects remaining on the railroad crossing after the barrier has been lowered from images captured by a camera installed near the crossing, and notifies trains passing through the crossing and the control center of the crossing status.

[0028] The railroad crossing hazard detection system of this embodiment is a railroad crossing hazard detection system that can detect unknown objects by detecting known objects from acquired video frames, generating a high-quality normal image using the detection results of the known objects, comparing the normal image with the current video frame, and detecting the difference between them.

[0029] [System Configuration] The configuration and operation of the railroad crossing hazard detection system of this embodiment will be described below. Fig. 1 is a diagram showing the overall configuration of the railroad crossing hazard detection system of this embodiment.

[0030] As shown in FIG. 1, the crossing hazard detection system of this embodiment includes a camera 1, a known object detection unit 2, a normal image generation unit 3, a difference detection unit 4, and a detection result integration unit 5.

[0031] The operation of the railroad crossing hazard detection system will be described with reference to Figure 1. Camera 1 is an imaging sensor installed near the railroad crossing that outputs video frames capturing images of the situation inside the railroad crossing. Known object detection unit 2 detects known objects from the video frames captured by camera 1 and outputs a known object detection result. Based on the known object detection result output from known object detection unit 2, normal image generation unit 3 synthesizes image areas other than known objects in video frames from multiple past times to generate a normal image in which the image areas other than known objects match. Difference detection unit 4 compares the feature amounts extracted from the normal image generated by normal image generation unit 3 with the feature amounts extracted from the current video frame, detects the difference in feature levels between the normal image and the current video frame, and outputs a difference detection result that is the difference between the current video frame and the normal image. The detection result integration unit 5 integrates the known object detection results output from the known object detection unit 2 and the difference detection results output from the difference detection unit 4, and outputs either an object detection result that distinguishes between known objects and unknown objects, or an object detection result that does not distinguish between known objects and unknown objects.

[0032] In the railroad crossing hazard detection system, the image area in which known objects and unknown objects are detected may be the entire area of ​​the video frame captured by the camera 1, or may be a predetermined partial area.

[0033] In the railroad crossing hazard detection system, known objects detected by the known object detection unit 2 include, for example, barriers, people, cars, motorcycles, and trains, while unknown objects detected by the difference detection unit 4 include, for example, objects that have fallen on the railroad crossing. While the barrier is raised, the railroad crossing hazard detection system does not notify the train or control center of the detection of any of the above-mentioned objects. On the other hand, after the barrier is lowered, if the system detects known objects other than barriers and trains, or unknown objects, it will notify the train and control center.

[0034] The known object detection unit 2 may detect known objects using, for example, a convolutional neural network (CNN) trained using images of known objects to be detected as training data. Alternatively, the known object detection unit 2 may detect known objects by pattern matching with images of many known objects. The task to be solved for detecting known objects may be, for example, a semantic segmentation task that labels object types on a pixel-by-pixel basis in an image. Alternatively, it may be, for example, an object detection task that outputs information on the position and size of an object. Using these CNNs, information indicating the image area of ​​the known object (for example, a bounding box) is output as the known object detection result. A known object detection score indicating the accuracy of the CNN's detection may be added to the known object detection result.

[0035] FIG. 7 is a diagram showing an example of processing executed by the normal image generating unit 3. As shown in FIG.

[0036] A video frame F101 acquired by camera 1 captures a person F102, a car F103, and a barrier F104. The known object detection unit 2 detects known objects using, for example, a semantic segmentation CNN trained to detect these known objects, and generates a known object detection result indicating the name of the known object to which each pixel belongs or whether the pixel does not belong to any known object. The normal image generation unit 3 generates a mask image F105 based on the known object detection result. The mask image F105 is an image in which known object regions are assigned a value of 0 and image regions other than known objects are assigned a value of 1, and is used to exclude a person region F106, a car region F107, and a barrier region F108. The mask image F105 may also be an image in which the region of a bounding box containing a detected object is assigned a value of 0. Similarly, mask images for excluding known object regions are generated from video frames at multiple times. A mask image is then applied to each video frame, and the pixel values ​​of each video frame are multiplied by the pixel values ​​of the mask image to generate multiple images with the mask image applied, i.e., images with known objects removed. The pixel values ​​of the multiple images are then averaged by the number of frames in which the pixel was not a known object, and image areas other than the known object are synthesized to generate a normal image F109. The normal image F109 is a high-quality normal image in which known objects that hinder the detection of unknown objects are removed, and the image areas other than the object match the current video frame. The normal image generator 3 generates a normal image by removing the known object detection area indicated by the difference detection result, but the higher the difference detection score output by the difference detector, the lower the contribution to synthesis may be set for a data area, so that data areas with a higher contribution may be synthesized as the normal image.

[0037] The first method for combining image regions other than the known object from multiple video frames to generate a normal image is pixel averaging. For each pixel, the pixel values ​​of the video frames other than the known object are summed and divided by the number of video frames used to generate a good normal image. The first method using pixel averaging is described below with reference to FIG. 16.

[0038] The second method for generating a normal image by synthesizing image regions other than those containing known objects from multiple video frames is to filter pixel values. For example, a Kalman filter with pixel values ​​as state variables is provided for each pixel, and the state variables and variance of the Kalman filter are updated each time a video frame containing no known objects is observed. After each pixel observes a predetermined number of video frames, the state variables are extracted, allowing a high-quality normal image to be generated. The second method using a Kalman filter will be described later with reference to FIG. 17.

[0039] The normal image generating unit 3 may output, as a normal image generation result, a normal image generation completion flag indicating that normal data has been generated, in addition to the normal image. In the first method described above, the normal image generation completion flag may be set to "true," indicating that normal data has been generated, for example, when the number of video frames used exceeds a predetermined threshold. In the second method described above, the normal image generation completion flag may be set to "true," indicating that normal data has been generated, for example, when the variance values ​​of the Kalman filter for all pixels fall below a predetermined threshold (see FIGS. 16 and 17). When the difference detecting unit 4 detects a difference after the normal image generation completion flag becomes true, the difference can be suitably detected using a high-quality normal image.

[0040] FIG. 8 is a diagram showing an example of processing executed by the difference detection unit 4. As shown in FIG.

[0041] In the current video frame F101, a fallen person F110 and a bag F111 are present at the railroad crossing. The difference detection unit 4 includes a feature extractor F112 for extracting features from images, and extracts features from both the normal image and the current video frame. The intermediate layer of the feature extractor F112a for the normal image is compared with the intermediate layer of the feature extractor F112b for the current video frame to generate intermediate layer difference maps F113a-F113c. Because the intermediate layer difference maps F113a-F113c have different receptive fields depending on the depth of the feature extractors F112a and F112b from the input, for example, a difference map F114 that appropriately indicates the object region can be generated by averaging multiple intermediate layer difference maps F113a-F113c. Alternatively, one of the intermediate layer difference maps F113a-F113c may be used as the difference map F114.

[0042] The feature extractors F112a and F112b use, for example, trained CNNs. The CNNs may be trained using various training image data that are different from the video frames acquired by the railroad crossing hazard detection system. By using a trained CNN trained using various training image data, it is possible to appropriately extract features of unknown objects and generate a difference map. For example, a trained CNN for classification may be used as the CNN. Alternatively, a trained CNN for semantic segmentation or object detection may be used. In this case, in addition to the various training image data, training image data related to known objects to be detected by the known object detection unit 2 may be used to train the CNN of the difference detection unit 4, thereby enabling the CNN used by the known object detection unit 2 to be shared, thereby reducing memory consumption.

[0043] To generate intermediate layer difference maps F113a-F113c from the intermediate layer of the normal image feature extractor F112a and the intermediate layer of the current video frame feature extractor F112b, it is necessary to calculate the similarity between each node. Because the intermediate layer of a CNN has a large number of dimensions, the similarity between each node may be calculated using, for example, cosine similarity, which can suitably calculate the similarity between high-dimensional vectors. By subtracting the calculated cosine similarity from, for example, 1, a difference map can be generated in which the greater the difference in feature values, the greater the value.

[0044] When the difference detection unit 4 detects a significant difference from the difference map F114, it may apply an averaging filter with a predetermined kernel size of 1×1 or more to the difference map F114, and then detect an area exceeding a predetermined threshold. This reduces the influence of minute noise on the difference map, and allows significant differences to be properly detected even when the object area is small compared to the image size. The difference detection unit 4 outputs information indicating the image area with a significant difference as the difference detection result. The difference detection result may be accompanied by the difference map as, for example, a difference detection score.

[0045] FIG. 9 is a diagram showing an example of processing in which the detection result integration unit 5 outputs an object detection result in which known objects and unknown objects are distinguished.

[0046] The known object detection result F115, in which the known object detection unit 2 detects a known object from the current video frame F101, includes a barrier detection result F116 and a fallen person detection result F117. Meanwhile, the difference detection result F118, in which the difference detection unit 4 detects the difference between the current video frame F101 and a normal image, includes a difference detection result F119 related to the barrier, a difference detection result F120 related to the fallen person, and a difference detection result F121 related to the bag. For example, the mask image F105 generated from the known object detection result F115 is an image in which the barrier region F108 and the fallen person region F122 are set to 0 and the surrounding image regions are set to 1. Therefore, by multiplying the mask image F105 by the difference detection result F118, only the difference detection result F121 for the bag can be extracted, and an unknown object detection result F123 including the bag detection result F124 can be output. Then, the known object detection result F115 and the unknown object detection result F123 are output together as the object detection result.

[0047] When outputting object detection results that do not distinguish between known objects and unknown objects, for example, an area that appears in at least one of the known object detection result F115 and the difference detection result F118 may be output as the object detection result. Although differences indicating known objects are also detected in the difference detection result F118, the known object detection result F115 obtained using a trained CNN using training data for known objects is considered to have detected known objects with a higher degree of accuracy than the difference detection result F118. Therefore, a method of outputting object detection results using both the known object detection result F115 and the difference detection result F118 is useful.

[0048] The detection result integration unit 5 can include the dwell time in the detection criteria, for example, by outputting the object detection result after a predetermined time has elapsed since the first detection. In the railroad crossing hazard detection system of this embodiment, it is preferable to output the object detection result immediately after object detection, from the perspective of accident prevention and damage minimization. Furthermore, if an object is detected in M ​​frames out of N consecutive frames, the object detection result may be output, excluding sporadic erroneous detections and non-detections.

[0049] [Operation sequence] FIG. 15 is a flowchart of the process executed by the information processing device (railroad crossing danger detection system) of this embodiment.

[0050] After startup, the information processing device starts a processing loop. First, in a video frame acquisition step S1, the information processing device acquires a video frame captured by a camera 1. Then, in a known object detection step S2, a known object detection unit 2 detects a known object from the acquired video frame and outputs a known object detection result. Then, in a normal image generation step S3, a normal image generation unit 3 synthesizes image areas other than the known object in video frames from multiple previous time points based on the output known object detection result, to generate a normal image in which the image areas other than the object match those in the current video frame. The processing of the normal image generation step S3 will be described later with reference to Figures 16 and 17. Then, in a normal image generation completion determination step S4, the normal image generation unit 3 determines whether generation of a normal image has been completed. If generation of a normal image has been completed, the normal image generation unit 3 sends the normal image generation result to a difference detection unit 4. In difference detection step S5, difference detection unit 4 compares the feature amounts extracted from the generated normal image with the feature amounts extracted from the current video frame to detect differences in feature levels between the normal image and the current video frame, and outputs a difference detection result. Then, in detection result integration step S6, detection result integration unit 5 integrates the known object detection result and the difference detection result, and outputs at least one of an object detection result that distinguishes between known objects and unknown objects, or an object detection result that does not distinguish between known objects and unknown objects.

[0051] Fig. 16 is a detailed flowchart of the normal image generation step S3. In the formula shown in Fig. 16, m is the mask value, nt is the number of video frames used, xt is the pixel value of the normal image at time t, and yt is the pixel value of the input image at time t.

[0052] First, the normal image generation unit 3 generates a mask value m for each pixel. For example, it sets 0 for a known object detection area and 1 if the area is not a known object detection area (S11). Then, the normal image generation unit 3 calculates nt by adding m to the number of most recently used video frames nt-1, and updates the number of video frames (S12). Then, the normal image generation unit 3 updates the pixel value xt of the normal image by excluding the known object area (S13). The normal image generation unit 3 repeatedly executes the above procedure for each pixel, and when processing is completed for all pixels, updating of the normal image using one video frame is completed.

[0053] Fig. 17 is a detailed flowchart of another process of the normal image generation step S3. In the formula shown in Fig. 17, m is the mask value, xt is the pixel value of the normal image at time t, Pt is the variance of the state variable at time t, Q is the variance of the process noise, et is the prediction error at time t, Kt is the Kalman gain at time t, R is the variance of the observation noise, and the symbols with hats are the estimated values ​​of the variables.

[0054] First, the normal image generation unit 3 generates a mask value m for each pixel. For example, it sets 0 in a known object detection area and 1 if not (S21). Then, the normal image generation unit 3 predicts the state (S22). For example, it predicts the pixel value xt of the normal image and the variance Pt of the state variables. Then, the normal image generation unit 3 calculates the prediction error e at time t by subtracting the predicted value of the pixel value xt of the normal image at time t from the pixel value yt of the input image at time t (S23). Then, the normal image generation unit 3 calculates the Kalman gain (S24). For example, it calculates the Kalman gain Kt at time t by dividing the estimated value of the variance Pt of the state variables at time t by the sum of the predicted value of the variance Pt of the state variables at time t and the variance R of the observation noise. Then, the normal image generation unit 3 updates the state (pixel value xt of the normal image and variance Pt of the state variables) excluding the known object area using the formula shown in the figure (S25). The normal image generating unit 3 repeatedly executes the above procedure for each pixel, and when the process is completed for all pixels, updating of the normal image using one video frame is completed.

[0055] As described above, the railroad crossing hazard detection system of this embodiment detects known objects from acquired video frames, generates a high-quality normal image using the detection results of the known objects, and detects unknown objects by detecting the difference between the features extracted from the normal image and the current video frame.

[0056] <Example 2> In Example 2, an abandoned object detection system, which is one application of the information processing device and method of the present invention, will be described. The abandoned object detection system detects abandoned objects such as suitcases in transportation facilities and public facilities such as airports and train stations, and notifies a monitoring center.

[0057] The configuration and operation of the abandoned object detection system of this embodiment will be described. Figure 2 is a diagram showing the overall configuration of the abandoned object detection system of this embodiment. The abandoned object detection system of this embodiment is capable of robustly detecting unknown objects against changes in lighting conditions, etc., by regenerating a normal image using a global change in the video frame as a trigger.

[0058] [System Configuration] As shown in Figure 2, the abandoned object detection system of this embodiment adds a normal image control unit 6 to the configuration of the railroad crossing hazard detection system of Example 1 (Figure 1), and the difference detection result output by the difference detection unit 4 is also input to the normal image generation unit 3.

[0059] The operation of the abandoned object detection system of this embodiment will be described with reference to FIG. 2. Description of operations overlapping with those of the first embodiment will be omitted. The difference detection results output by the difference detection unit 4 are input to the detection result integration unit 5, the normal image generation unit 3, and the normal image control unit 6. The normal image generation unit 3 generates mask images of the detected known and unknown objects based on the known object detection results and the difference detection results, and synthesizes multiple images that do not include known and unknown objects, generated by multiplying the mask images generated from video frames at multiple times before the current time with multiple video frames, to generate a normal image in which the image areas other than the known and unknown objects match the current video frame. The difference detection results input to the normal image generation unit 3 include the known object detection results and the unknown object detection results. Unknown objects detected for a short period of time, less than a predetermined time, are not background and should therefore be included in the mask image. The normal image control unit 6 uses one or both of the video frame and the difference detection result to detect global changes in the video frame (for example, changes in brightness of the entire image, or the area of ​​the region where a difference is detected), and instructs the normal image generation unit 3 to regenerate a normal image if the global change is large. The detection result integration unit 5 compares the known object detection result with the difference detection result, and outputs the object detection result if the difference detection result contains a detection result that does not exist in the known object detection result, i.e., a detection result of an unknown object.

[0060] In the abandoned object detection system, the image area in which known objects and unknown objects are detected may be the entire area of ​​the video frame captured by the camera 1, or may be a predetermined partial area.

[0061] In the abandoned object detection system, a known object detected by the known object detection unit 2 is, for example, a person, and an unknown object detected by the difference detection unit 4 is, for example, an abandoned object such as a carry case. The abandoned object detection system is installed in transportation facilities and public facilities such as airports and train stations.

[0062] FIG. 10 is a diagram showing an example of a process in which the abandoned object detection system of this embodiment monitors a room under a plurality of conditions.

[0063] First, when a normal image F202a generated when the lights were on is applied to a video frame F201a with the lights on, the difference detection result F203a shows that differences between people are properly detected. Then, when the lights are turned off, when a normal image F202b generated when the lights were on is applied to a video frame F201b, the difference detection result F203b is output, in which differences exist throughout the image, making subsequent object detection difficult. Therefore, global changes in the video frame when the lights were turned off are detected, and a normal image with the lights off is regenerated. As a result, when a normal image F202c regenerated after the lights were turned off is applied to a video frame F201c, differences between people and abandoned objects can be properly detected, as shown in the difference detection result F203c.

[0064] As shown in FIG. 2, the normal image control unit 6 uses a video frame or a difference detection result as a trigger to instruct the normal image generation unit 3 to regenerate a normal image. When a video frame is used as a trigger, the normal image generation unit 3 may instruct to regenerate a normal image when, for example, a change in average luminance exceeding a predetermined threshold is detected in the entire image or a predetermined partial region. When a difference detection result is used as a trigger, the normal image generation unit 3 may instruct to regenerate a normal image when, for example, a difference in area exceeding a predetermined threshold is detected in the difference detection result in the entire image or a predetermined partial region. Either the video frame or the difference detection result may be used, or both may be used together. This makes it possible to use an appropriate normal image depending on changes in the shooting conditions (illumination) and to suitably detect unknown objects.

[0065] The detection result integration unit 5 can include the dwell time in the detection criteria, for example, by outputting the object detection result after a predetermined time has elapsed since the initial detection. For example, as shown in FIG. 10, when an elevator F204 appears in a video frame F201 in the abandoned object detection system of this embodiment, the opening and closing of the elevator F204's doors is detected as a difference. However, since the abandoned object detection system should not detect the opening and closing of the elevator F204's doors, it is desirable not to output the difference resulting from the opening and closing of the elevator F204's doors as the object detection result. Therefore, by including the dwell time in the detection criteria, abandoned objects F205 that have remained in the same location for a long time can be suitably detected.

[0066] In particular, when the detection result integration unit 5 includes dwell time in its detection criteria, object detection continues for a long time, and the normal image is also updated for a long time. In this case, it is desirable to ensure that the unknown object being detected is not included in the normal image. Therefore, the normal image generation unit 3 can generate a normal image by combining image regions other than known objects and unknown objects using the known object detection results and difference detection results. A method for incorporating the difference detection results into the generation of the normal image may be to generate the normal image by excluding image regions with significant differences indicated by the difference detection results. Alternatively, if the difference detection results include a difference detection score such as a difference map, the contribution to the combination can be determined according to the difference detection score. For example, the contribution to the combination of image regions where the difference is not significant but the presence of an object is suspected can be set low.

[0067] As described above, the abandoned object detection system of this embodiment regenerates a normal image using global changes in the acquired video frame as a trigger, and can robustly detect unknown objects even if lighting conditions change.

[0068] Example 3 In Example 3, a train side monitoring system, which is one application of the information processing device and method of the present invention, will be described. The train side monitoring system monitors the train and the situation around the train when it departs from or arrives at a station, detects danger such as an object getting caught in the door, and notifies the crew and a monitoring center.

[0069] The configuration and operation of the train side monitoring system of this embodiment will be described. Figure 3 is a diagram showing the overall configuration of the train side monitoring system of this embodiment. The train side monitoring system of this embodiment starts generating normal images when the train stops at a station, and can detect unknown objects such as an object caught in the doors when they are closed.

[0070] [System Configuration] As shown in Fig. 3, the train side monitoring system of this embodiment adds a normal image control unit 6 to the configuration of the railroad crossing hazard detection system of embodiment 1 (Fig. 1), and the difference detection results output by the difference detection unit 4 are also input to the normal image generation unit 3. Also, unlike the abandoned object detection system configuration of embodiment 2 (Fig. 2), external information is input to the normal image control unit 6.

[0071] The operation of the train side monitoring system of this embodiment will be described with reference to FIG. 3. Description of operations overlapping with those of the first embodiment will be omitted. The difference detection result output by the difference detection unit 4 is input to the detection result integration unit 5, the normal image generation unit 3, and the normal image control unit 6. When normal image generation is stopped during unknown object detection, the normal image generation unit 3 generates a mask image of the detected known object based on the known object detection result, and synthesizes multiple images not including the known object, which are generated by multiplying the mask image generated from video frames at multiple times before the current time with the multiple video frames, to generate a normal image in which the image area other than the known object matches the current video frame. When normal image generation is continued during unknown object detection, the normal image generation unit 3 may generate mask images of the detected known object and unknown object based on the known object detection result and the difference detection result, and synthesizes multiple images not including the known object and unknown object, which are generated by multiplying the mask image generated from video frames at multiple times before the current time with the multiple video frames, to generate a normal image in which the image area other than the known object and unknown object matches the current video frame. The normal image control unit 6 uses external information taken in from outside the train side monitoring system to instruct the normal image generation unit 3 to regenerate a normal image. The detection result integration unit 5 compares the known object detection result with the difference detection result, and outputs an object detection result if the difference detection result contains a detection result that does not exist in the known object detection result, i.e., a detection result of an unknown object.

[0072] In the train side monitoring system, a known object detected by the known object detection unit 2 is, for example, a person, and an unknown object detected by the difference detection unit 4 is, for example, an object such as a bag that gets caught in a door.

[0073] The external information taken in by the normal image control unit 6 is, for example, information relating to the photographing date and time, linked system signals, etc. The linked system signals are, for example, speed signals and door open signals obtained from the train control system.

[0074] FIG. 11 is a diagram showing an example of processing executed by the train side monitoring system of this embodiment.

[0075] When the train stops at a station, the normal image control unit 6 instructs the normal image generation unit 3 to regenerate a normal image based on external information acquired from the train control system, for example, when the speed signal reaches a value indicating the train is stopping or a door-open signal is input. A person F302 waiting to board the train is captured in the video frame F301a, and a person F302 boarding or disembarking is also captured in the video frame F301b after the doors open. Therefore, since no object caught in the door is captured in the period immediately before and after the train stops, the normal image generation unit 3 generates a normal image F303 using the image area other than the person F302 detected by the known object detection unit 2. This makes it possible to use an appropriate normal image according to changes in the shooting conditions (station) and to appropriately detect unknown objects.

[0076] After the door is closed, when an object F304 caught in the door appears in the video frame F301c, the difference detection unit 4 compares the feature amount extracted from the normal image F303 with the feature amount extracted from the video frame F301c, and outputs a difference detection result F305 having a difference detection result F306 for the object caught in the door.

[0077] In the train side monitoring system of this embodiment, it is preferable that the detection result integration unit 5 outputs the object detection result immediately after detecting an object, from the viewpoint of preventing accidents and minimizing damage.

[0078] As described above, the train side monitoring system of this embodiment starts generating normal images when the train stops at a station, and can detect unknown objects such as an object caught in the doors when the doors are closed.

[0079] Example 4 In Example 4, an abandoned object detection system, which is one application of the information processing device and method of the present invention, will be described. The abandoned object detection system detects abandoned objects such as carry cases in transportation facilities and public facilities such as airports and train stations, and notifies a monitoring center.

[0080] The configuration and operation of the abandoned object detection system of this embodiment will be described. Fig. 4 is a diagram showing the overall configuration of the abandoned object detection system of this embodiment. The abandoned object detection system of this embodiment generates and saves multiple normal images according to variations in lighting conditions, etc., and selects the most suitable normal image according to the environment to detect the object, thereby making it possible to detect unknown objects robustly against changes in lighting conditions, etc.

[0081] [System Configuration] As shown in FIG. 4, the abandoned object detection system of this embodiment has a normal image buffer 7 added to the configuration of the abandoned object detection system of the second embodiment (FIG. 2).

[0082] The operation of the abandoned object detection system of this embodiment will be described with reference to FIG. 4. Description of operations overlapping with those of the first embodiment will be omitted. The difference detection results output by the difference detection unit 4 are input to the detection result integration unit 5, the normal image generation unit 3, and the normal image control unit 6. The normal image generation unit 3 generates mask images of the detected known and unknown objects based on the known object detection results and the difference detection results, and synthesizes multiple images that do not include known and unknown objects, generated by multiplying the mask images generated from video frames at multiple times before the current time with the multiple video frames, to generate a normal image in which the image areas other than the known and unknown objects match the current video frame. The difference detection results input to the normal image generation unit 3 include the known object detection results and the unknown object detection results. Unknown objects detected for a short period of time, less than a predetermined time, are not background and should therefore be included in the mask image. The normal image control unit 6 uses one or both of the video frame and the difference detection result to detect global changes in the video frame (for example, changes in brightness of the entire image, or the area of ​​the region where a difference is detected), and instructs the normal image generation unit 3 to regenerate a normal image if the global change is large. The detection result integration unit 5 compares the known object detection result with the difference detection result, and outputs the object detection result if the difference detection result contains a detection result that does not exist in the known object detection result, i.e., a detection result of an unknown object.

[0083] The normal image buffer 7 stores the normal images generated by the normal image generation unit 3, and in accordance with instructions from the normal image control unit 6, reads out a normal image that matches the instruction and sends it to the normal image generation unit 3. The normal image generation unit 3 sends the normal image obtained from the normal image buffer 7 to the difference detection unit 4, and updates the normal image using the normal image as the initial value. This allows the difference detection unit 4 to be activated quickly.

[0084] The normal image control unit 6 instructs the normal image buffer 7 to read out a normal image having image features similar to those of the video frame. The similarity of image features may be determined, for example, using the average luminance of the entire image or a predetermined region. Alternatively, the similarity may be determined using features extracted by a predetermined CNN or the like.

[0085] FIG. 12 is a diagram illustrating an example of a process executed by the abandoned object detection system of this embodiment.

[0086] The normal image buffer 7 stores normal images generated when the lights are ON and normal images generated when the lights are OFF. When the lights are ON, the normal image generation unit 3 reads out from the normal image buffer 7 a normal image F402a generated when the lights are ON, which has image features similar to those of the video frame F401a. This allows the difference detection unit 4 to output a difference detection result F403a that indicates a significant difference in the abandoned object area. Similarly, when the lights are OFF, the normal image generation unit 3 reads out from the normal image buffer 7 a normal image F402b generated when the lights are OFF, which has image features similar to those of the video frame F401b. This allows the difference detection unit 4 to output a difference detection result F403b that indicates a significant difference in the abandoned object area.

[0087] The normal image buffer 7 may store normal images under different conditions such as weather and time in addition to the above-mentioned lighting conditions.

[0088] The timing for saving the normal image in the normal image buffer 7 may be synchronized with the timing when the normal image control unit 6 instructs the normal image generation unit 3 to regenerate a normal image, or may be at a predetermined time interval, or may be at the timing of an external manual operation, for example.

[0089] The number of normal images stored in the normal image buffer 7 may be a predetermined set value, or may be automatically determined using a predetermined clustering algorithm based on the image features of the generated normal images so as to reduce the number of similar images stored.

[0090] As described above, the abandoned object detection system of this embodiment generates and saves multiple normal images according to variations in lighting conditions, etc., and selects the appropriate normal image when detecting an object, thereby enabling robust detection of unknown objects even when lighting conditions change.

[0091] <Example 5> In Example 5, a road monitoring system, which is one application of the information processing device and method of the present invention, will be described. The road monitoring system detects obstacles such as fallen objects and flying objects on the road and notifies a monitoring center, and is suitable for monitoring expressways.

[0092] The configuration and operation of the road monitoring system of this embodiment will be described. Figure 5 is a diagram showing the overall configuration of the road monitoring system of this embodiment. The road monitoring system of this embodiment can detect unknown objects on the road and further compare past and present normal images to detect changes in the road over time.

[0093] [System Configuration] As shown in Fig. 5, the road monitoring system of this embodiment adds a normal image buffer 7 and a difference detection unit 4b to the configuration of the train side monitoring system of embodiment 3 (Fig. 3). Similar to the difference detection unit 4 of embodiment 1, the difference detection unit 4a compares the feature amount extracted from the normal image generated by the normal image generation unit 3 with the feature amount extracted from the current video frame, detects the difference in feature amount level between the normal image and the current video frame, and outputs the difference detection result which is the difference between the current video frame and the normal image.

[0094] The operation of the road monitoring system will be described with reference to FIG. 5. Descriptions of operations overlapping with those in Examples 1 and 3 will be omitted. The normal image generation unit 3 generates a mask image of the detected known object based on the known object detection result, synthesizes multiple images that do not include the known object, and generates a normal image in which the image area other than the known object matches the current video frame, based on external information, for example, regarding the shooting date and time, to the normal image generation unit 3 at predetermined time intervals. The normal image control unit 6 instructs the normal image generation unit 3 to generate normal images at predetermined time intervals, based on external information regarding the shooting date and time, and stores the generated normal images in the normal image buffer 7. The difference detection unit 4b compares the feature values ​​extracted from the current normal image generated by the normal image generation unit 3 with those extracted from previous normal images read from the normal image buffer 7 to detect differences in feature value levels between the current normal image and the previous normal images, and outputs an aging detection result, which is the difference in feature value levels between the current normal image and the previous normal images. The normal image buffer 7 may compare normal images at different times, and may compare the current normal image with a past normal image or normal images at any other time. The difference detection unit 4b may compare the generated current normal image with a normal image from a predetermined time ago when the current normal image is generated, or may compare the normal image at a time specified by the user.

[0095] In the road monitoring system, known objects detected by the known object detection unit 2 are, for example, automobiles, motorcycles, trucks, buses, etc., and unknown objects detected by the difference detection unit 4 are, for example, fallen objects such as timber, and flying objects such as vinyl sheets.

[0096] In the road monitoring system, the difference detection unit 4b detects changes over time such as potholes and cracks in the asphalt, and overgrowth of trees along the road or in the median strip.

[0097] FIG. 13 is a diagram showing an example of the aging detection process executed by the road monitoring system.

[0098] The normal image generator 3 synthesizes image areas other than the vehicle F502 from multiple video frames F501a-F501b to generate a normal image F503a, and synthesizes image areas other than the vehicle F502 from multiple video frames F501c-F501d to generate a normal image F503a. A pothole F504 that occurred between the generation timing of the past (e.g., one month ago) normal image F503a (i.e., the capture timing of video frame F501b) and the generation timing of the current normal image F503b (i.e., the capture timing of video frame F501d) is captured in both the current video frames F501c and F501d, so a current normal image F503b capturing the pothole F504 is generated. Therefore, the difference detector 4b outputs a difference detection result F506 including the pothole as the aging detection result F505.

[0099] Unknown objects such as falling objects and flying objects suddenly appear in an image frame, so they can be detected by the difference detection unit 4a, which compares the current image frame with a normal image. However, changes over time such as potholes appear as changes over a long period of time, making them difficult to detect by the difference detection unit 4a. Therefore, the difference detection unit 4b can effectively detect changes over time by comparing a past normal image with a current normal image.

[0100] As described above, the road monitoring system of this embodiment can detect unknown objects on the road by comparing past and present normal images, and detect changes in the road over time.

[0101] Example 6 In Example 6, a road monitoring system, which is one application of the information processing device and method of the present invention, will be described. The road monitoring system detects obstacles such as fallen objects or flying objects on the road and notifies a monitoring center, and is suitable for monitoring expressways.

[0102] The configuration and operation of the road monitoring system of this embodiment will be described. Fig. 6 is a diagram showing the overall configuration of the road monitoring system of this embodiment. The road monitoring system of this embodiment detects known objects from a point cloud frame acquired by the LiDAR 11, generates a high-quality normal point cloud using the detection results of the known objects, and detects the difference between the feature amounts extracted from the normal point cloud and the feature amounts extracted from the current point cloud frame, thereby being able to detect unknown objects.

[0103] [System Configuration] 6, the road monitoring system of this embodiment is provided with a LiDAR 11 instead of the camera 1 of the configuration of the railroad crossing hazard detection system of the first embodiment (FIG. 1). The processing unit, which is the other configuration, is configured in the same way as the railroad crossing hazard detection system of the first embodiment.

[0104] The operation of the road monitoring system of this embodiment will be described with reference to FIG. 6. The LiDAR 11 is a sensor installed in a position near the road from which the road can be seen and acquires a point cloud frame obtained by scanning the road conditions. The known object detection unit 2 detects known objects from the point cloud frame acquired by the LiDAR 11 and outputs a known object detection result. The normal point cloud generation unit 13 generates mask data based on the known object detection result. The mask data is data in which known object areas are assigned a value of 0 and point cloud areas other than known objects are assigned a value of 1. The mask data generated from point cloud frames at multiple times before the current time is multiplied by multiple point cloud frames to generate a point cloud that does not contain known objects, thereby generating a normal point cloud in which the point cloud areas other than known objects match the current point cloud frame. The difference detection unit 4 compares feature values ​​extracted from the normal point cloud with feature values ​​extracted from the current point cloud frame to detect differences in feature levels between the normal point cloud and the current point cloud frame and outputs the difference detection result. The detection result integration unit 5 integrates the known object detection result and the difference detection result, and outputs at least one of an object detection result that distinguishes between known objects and unknown objects, or an object detection result that does not distinguish between known objects and unknown objects.

[0105] FIG. 14 is a diagram showing an example of processing executed by the road monitoring system of this embodiment.

[0106] The known object detection unit 2 detects the vehicle F602 from the point cloud frames F601a and F601b in which the vehicle F602 appears. The normal point cloud generation unit 13 generates a normal point cloud F603 using the point cloud area other than the vehicle F602. After that, when a falling object occurs and a falling object F604 appears in the point cloud frame F601c, the difference detection unit 4 compares the feature amount extracted from the normal point cloud F603 with the feature amount extracted from the point cloud frame F601c, and outputs a difference detection result F605 including a difference detection result F606 indicating the fallen object.

[0107] In the road monitoring system of this embodiment, by using point cloud data instead of images, the system is less susceptible to changes in brightness due to, for example, sunlight conditions, and can reliably detect obstacles such as fallen or flying objects on the road.

[0108] The sensor for acquiring point cloud data may be, for example, a millimeter wave radar instead of a LiDAR.

[0109] As described above, the road monitoring system of this embodiment detects known objects from the point cloud frame acquired by the LiDAR 11, generates a high-quality normal point cloud using the detection results of the known objects, and detects the difference between the normal point cloud and the features extracted from the current point cloud frame, thereby detecting unknown objects.

[0110] The image data handled in the above-described embodiments is generally a color image, but any image that allows an object to be recognized may be used, such as a grayscale image, a near-infrared image, a far-infrared image, a depth image, etc. Also, while the above-described embodiments use image data and point cloud data, the handled data is not limited to these.

[0111] The present invention is not limited to the above-described embodiments, but includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to configurations including all of the described configurations. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, part of the configuration of each embodiment may be added, deleted, or replaced with other configurations.

[0112] Furthermore, the aforementioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole in hardware, for example by designing them as integrated circuits, or may be realized in software by having a processor interpret and execute a program that realizes each function.

[0113] Information such as programs, tables, and files that realize each function can be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a recording medium such as an IC card, an SD card, or a DVD.

[0114] In addition, the control lines and information lines shown are those that are considered necessary for explanation, and do not necessarily represent all the control lines and information lines that are necessary for implementation. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0115] 1 camera 2 Known object detection unit 3. Normal image generation section 4, 4a, 4b Difference detection section 5. Detection result integration unit 6 Normal image control section 7 Normal Image Buffer 11 LiDAR 13 Normal point cloud generator F112, F112a, F112b feature extractors

Claims

1. An information processing device, It is composed of a computer having an arithmetic unit that performs predetermined arithmetic processing and a storage device that the arithmetic unit can access, The aforementioned computing device includes a known object detection unit that outputs a known object detection result in which a known object has been detected from the data acquired by the sensor, The aforementioned computing device includes a normal data generation unit that uses the known object detection result to synthesize regions other than the known object in the data acquired by the sensor at multiple time points to generate normal data, The information processing device is characterized in that it can detect unknown objects with high accuracy by comprising a first difference detection unit that compares the normal data with the data acquired by the sensor and outputs a difference detection result in which the difference between the normal data and the data acquired by the sensor is detected as an unknown object.

2. An information processing apparatus according to claim 1, An information processing device comprising a detection result integration unit that integrates the known object detection result and the difference detection result and outputs at least one of the following: an object detection result that distinguishes between the detected known object and the detected unknown object, or an object detection result that does not distinguish between the detected known object and the detected unknown object.

3. An information processing apparatus according to claim 1, The normal data generation unit outputs a normal data generation completion flag indicating that normal data has been generated. The first difference detection unit is characterized in that it can suitably detect differences using high-quality normal data by detecting differences after the normal data generation completion flag indicates the generation of normal data.

4. An information processing apparatus according to claim 1, The first difference detection unit is, The system includes a feature extractor that extracts features from the normal data and the data acquired by the sensor. The information processing device is characterized in that the feature extractor is trained on images of multiple types of objects different from the data acquired by the sensor, thereby enabling it to appropriately extract features of unknown objects and generate a difference map.

5. An information processing apparatus according to claim 4, The feature extractor has a neural network that extracts features from the normal data and the data acquired by the sensor. The first difference detection unit is, A difference map indicating the object region is generated using the output from the intermediate layer of the aforementioned neural network. An information processing device characterized by applying an averaging filter with a kernel size of 1 or more to the difference map, followed by thresholding to detect differences, thereby reducing the influence of minute noise on the difference map and enabling the appropriate detection of significant differences even when the object region is small relative to the data size.

6. An information processing apparatus according to claim 1, The information processing apparatus is characterized in that the normal data generation unit excludes the data region of the difference indicated by the difference detection result, or sets the contribution to the synthesis lower for data regions with a higher difference detection score output by the first difference detection unit, and synthesizes regions other than known and unknown objects in the data acquired by the sensor at multiple time points to generate normal data in which the detected regions other than known and unknown objects match.

7. An information processing apparatus according to claim 1, An information processing device characterized by having a normal data control unit that instructs the normal data generation unit to regenerate normal data, triggered by a global change in the area of ​​at least one region in which a change or difference in the brightness of the entire image is detected, or by at least one piece of information input from an external source, thereby enabling the use of an appropriate normal image in response to changes in shooting conditions and enabling the suitability detection of unknown objects.

8. An information processing apparatus according to claim 7, The system includes a normal data buffer that stores the normal data generated by the normal data generation unit and sends normal data suitable for instructions from the normal data generation unit to the normal data generation unit. The information processing device is characterized in that the normal data generation unit can quickly activate the difference detection unit by updating the normal data using the normal data sent from the normal data buffer as the initial value.

9. An information processing apparatus according to claim 7, A normal data buffer for storing the normal data generated by the normal data generation unit, The calculation device includes a second difference detection unit that detects the difference between a plurality of normal data, The information processing apparatus is characterized in that the second difference detection unit outputs an aging change detection result by comparing normal data at different times and detecting the difference in the normal data at different times as an aging change.

10. An information processing apparatus according to claim 1, An information processing device characterized in that the aforementioned data is image data or point cloud data.

11. A level crossing hazard detection system, A level crossing hazard detection system characterized by detecting an object within the level crossing using an information processing device according to any one of claims 1 to 10.

12. A system for detecting abandoned objects, An abandoned object detection system characterized by detecting abandoned objects in a public space using an information processing device according to any one of claims 1 to 10.

13. A train side monitoring system, A train side monitoring system characterized by monitoring the conditions of a train and its vicinity at the time of arrival and departure from a station using an information processing device according to any one of claims 1 to 10.

14. It is a road monitoring system, A road monitoring system characterized by detecting obstacles on a road using an information processing device according to any one of claims 1 to 10.

15. An information processing method in which an information processing device detects differences between multiple data sets, The information processing device comprises a arithmetic unit that performs predetermined arithmetic processing and a storage device that the arithmetic unit can access. The aforementioned information processing method is The aforementioned computing device outputs a known object detection result in which a known object has been detected from the data acquired by the sensor, The calculation device uses the known object detection result to synthesize regions other than the known object in the data acquired by the sensor at multiple time points to generate normal data; this is a normal data generation procedure. An information processing method characterized in that it can detect unknown objects with high accuracy, comprising a first difference detection procedure in which the computing device compares the normal data with the data acquired by the sensor and outputs a difference detection result in which the difference between the normal data and the data acquired by the sensor is detected as an unknown object.

Images