Image processing apparatus, image processing method, and image processing program

Abstract

To more easily determine a parameter suitable for detecting a predetermined detection object.SOLUTION: The image processing apparatus includes a processor that executes a process of acquiring a plurality of pieces of image data including a predetermined detection target, a process of generating a plurality of pieces of filtered image data by applying an image filter in which each of a plurality of parameters is set to each of the plurality of pieces of image data, a process of detecting the predetermined detection target from the plurality of pieces of filtered image data, a process of storing the parameter of the image filter set to the filtered image data having the highest detection accuracy in the detecting process in a storage unit, and a process of detecting the predetermined detection target by setting the image filter in which the parameter stored in the storage unit is set to a frame image acquired from a camera that images a monitoring area.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an image processing device, an image processing method, and an image processing program. [Background technology]

[0002] A monitoring system has been proposed that uses a camera to monitor fallen objects on the road (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-144600 Summary of the Invention [Problem to be solved by the invention]

[0004] When detecting falling objects using image data from video captured by a camera, preprocessing such as erosion and expansion is applied to the image data to make the falling object more apparent in the image data. In the preprocessing, parameters are set, exemplified by the number of times erosion, expansion, etc. are performed on the image data. The preprocessing parameters preferred for detecting falling objects vary depending on the influence of the shooting environment, such as the camera installation angle, weather, time of day, and season. Conventionally, workers have responded by adjusting the parameters as the shooting environment changes, which has resulted in a workload for the workers every time the shooting environment changes.

[0005] An object of one aspect of the disclosed technology is to provide an image processing device, an image processing method, and an image processing program that can more easily determine parameters suitable for detecting a predetermined detection target. [Means for solving the problem]

[0006] One aspect of the disclosed technology is exemplified by the following image processing device: The image processing device includes a processor that executes the following processes: acquiring a plurality of image data including a predetermined detection target; applying an image filter, to each of the plurality of image data, an image filter having a plurality of parameters set thereto to generate a plurality of filtered image data; detecting the predetermined detection target from the plurality of filtered image data; storing, in a storage unit, the parameters of the image filter set for the filtered image data with the highest detection accuracy in the detection process; and detecting the predetermined detection target by setting the image filter, to which the parameters stored in the storage unit are set, for image data acquired from a camera that captures an image of a monitoring area.

[0007] According to the image processing device, an image filter, each having a plurality of parameters set thereto, is applied to each of the plurality of image data to generate a plurality of filtered image data, and the selected parameters are stored in the storage unit based on the detection accuracy of the predetermined detection target using the generated plurality of filtered image data. Therefore, according to the image processing device, the parameters can be easily determined. Here, the image filter may include at least one of a dilation process, an erosion process, a smoothing process, and an expansion / erosion process. Furthermore, the parameters may include at least one of the number of times the image filter is applied to the image data and the filter size of the image filter.

[0008] The image processing device may further include the following features: the image data is image data obtained by cutting out an area of ​​the predetermined object to be detected from a captured frame image. The data size of the image data extracted from the frame image is smaller than that of the room image. With an image processing device having such a feature, the entire frame image is not acquired, so the storage capacity used to save the image data can be reduced.

[0009] The image processing device may further include the following feature: the plurality of image data including the predetermined detection object include a first image data group including the predetermined detection object and a second image data group excluding the predetermined detection object, and the process of storing the parameters in the storage unit determines the parameters to be stored in the storage unit based on a first frequency with which the predetermined detection object is detected from the first image data group in the detection process and a second frequency with which the predetermined detection object is detected from the second image data group in the detection process. By selecting the parameters in this manner, false detections in which something other than the predetermined detection object is detected as the predetermined detection object are suppressed.

[0010] The image processing device may further include the following feature: the processor accepts specification of a domain of the parameters, and the plurality of parameters are parameters within the domain. An image processing device having such a feature can prevent the processing time for parameter determination from becoming excessively long.

[0011] The disclosed technology can also be understood from the aspects of an image processing method and an image processing program. [Effects of the Invention]

[0012] According to the disclosed technology, it is possible to more easily determine parameters suitable for detecting a predetermined detection target. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating an example of a falling object detection system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of the detection device according to the embodiment. [Figure 3] FIG. 3 is a diagram illustrating an example of a processing block of the detection device according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of foreground extraction by the foreground extraction unit. [Figure 5]FIG. 5 is a diagram showing an example of a tracking extraction image generated by a tracking unit in the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of a tracking management table stored in the data storage unit in the embodiment. [Figure 7] 7A to 7C are diagrams illustrating examples of the detection frequencies of noise and falling objects when contraction and expansion processing is performed as preprocessing in the embodiment. [Figure 8] FIG. 8 is a diagram showing an example of a correct / incorrect information management table stored in the auxiliary storage unit by the correct / incorrect receiving unit. [Figure 9] FIG. 9 is a diagram illustrating an example of a correctness information management table to which a determination result by the abnormality determination unit is added in the embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of a confusion matrix generated by the evaluation unit in the embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of an evaluation result table that the evaluation unit stores in the auxiliary storage unit in the embodiment. [Figure 12] FIG. 12 is a diagram illustrating an example of a processing flow of the detection device according to the embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of processing by the candidate extraction unit in the embodiment. [Figure 14] FIG. 14 is a diagram illustrating an example of processing by the abnormality determination unit and the evaluation unit in the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] <Application example> An application example of the present invention will be described below. This application example is, for example, a detection device 2 illustrated in Fig. 1. The detection device 2 is an information processing device that monitors a road 500 and notifies a traffic warden M1 when it detects a fallen object.

[0015] The detection device 2 detects fallen objects on the road 500 based on frame images of the video captured by the camera 1. The detection device 2 performs predetermined pre-processing on the frame images before detecting fallen objects. For the pre-processing, parameters are set, exemplified by the number of times a filter is applied. The pre-processing parameters suitable for detecting fallen objects differ depending on the shooting conditions of the camera 1. If an operator were to perform such parameter setting every time the shooting conditions changed, the workload would be enormous.

[0016] The detection device 2 acquires image data including a falling object, and performs preprocessing on the image data while changing parameters set for the preprocessing, thereby detecting a falling object. Then, the detection device 2 determines the parameters to be stored in the auxiliary storage unit 23 based on the detection accuracy of the falling object detection.

[0017] When the detection device 2 is in operation, preprocessing is performed using the parameters stored in the auxiliary storage unit 23 (see FIG. 2). Therefore, according to this application example, parameters suitable for detecting a falling object can be easily determined. Note that, if the detection accuracy decreases during operation of the detection device 2, the detection device 2 can be made to execute processing to determine the parameters again.

[0018] <Embodiment> Hereinafter, an embodiment will be described with reference to the drawings. Fig. 1 is a diagram showing an example of a falling object detection system 100 according to an embodiment. The falling object detection system 100 is a system that detects an object that has fallen onto a road 500. The falling object detection system 100 includes a camera 1, a detection device 2, and a display device 3. The camera 1 and the detection device 2 are connected by a computer network N1. The detection device 2 and the display device 3 are connected by a connection cable L1.

[0019] The road 500 is, for example, a motorway or a national expressway, but may also be a general national road, a prefectural road, or a municipal road.

[0020] Camera 1 is a video camera that captures images of road 500 to be monitored. Camera 1 may be, for example, a network camera that transmits captured images via computer network N1. Camera 1 is, for example, a digital video camera that uses a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) as an imaging element. Video data captured by camera 1 is transmitted to detection device 2 via computer network N1. The frame rate of the video data captured by camera 1 is, for example, 1 fps.

[0021] The detection device 2 is an information processing device that detects a fallen object on the road 500 based on video data acquired from the camera 1 via the computer network N1. When the detection device 2 detects a fallen object on the road 500, it outputs video data of the detected fallen object to the display device 3 via the connection cable L1.

[0022] The display device 3 is a display monitored by the traffic warden M1. For example, a liquid crystal display (LCD), a plasma display panel (PDP), an inorganic electroluminescence (EL) panel, or an organic EL panel. The display device 3 displays the information output by the detection device 2.

[0023] 2 is a diagram illustrating an example of the hardware configuration of the detection device 2 according to the embodiment. The detection device 2 includes a Central Processing Unit (CPU) 21, a main memory unit 22, an auxiliary memory unit 23, a communication unit 24, a connection terminal 25, and a bus B1. The CPU 21, the main memory unit 22, the auxiliary memory unit 23, the communication unit 24, and the connection terminal 25 are interconnected by the bus B1.

[0024] The CPU 21 is also referred to as a microprocessor unit (MPU) or processor. The CPU 21 is not limited to a single processor and may have a multiprocessor configuration. Furthermore, a single CPU 21 connected via a single socket may have a multi-core configuration. At least a portion of the processing performed by the CPU 21 may be performed by a processor other than the CPU 21, such as a dedicated processor, such as a digital signal processor (DSP), a graphics processing unit (GPU), a numerical calculation processor, a vector processor, or an image processing processor. Furthermore, at least a portion of the processing performed by the CPU 21 may be performed by an integrated circuit (IC) or other digital circuit. Furthermore, at least a portion of the CPU 21 may include an analog circuit. Examples of integrated circuits include large-scale integrated circuits (LSIs), application-specific integrated circuits (ASICs), and programmable logic devices (PLDs). Examples of PLDs include field-programmable gate arrays (FPGAs). The CPU 21 may be a combination of a processor and an integrated circuit. This combination is called, for example, a microcontroller unit (MCU), a system-on-a-chip (SoC), a system LSI, or a chipset. In the detection device 2, the CPU 21 loads a program stored in the auxiliary memory 23 into a working area in the main memory 22 and controls peripheral devices through the execution of the program. This allows the detection device 2 to execute processing that meets a predetermined purpose. The main memory 22 and the auxiliary memory 23 are recording media that the detection device 2 can read.

[0025] The main storage unit 22 is exemplified as a storage unit that is directly accessed by the CPU 21. The main storage unit 22 includes a random access memory (RAM) and a read only memory (ROM).

[0026] The auxiliary storage unit 23 stores various programs and various data on a readable and writable recording medium. The auxiliary storage unit 23 is also called an external storage device. The auxiliary storage unit 23 stores an operating system (OS), various programs, various tables, etc. The OS includes a communication interface program that exchanges data with external devices connected via the communication unit 24. The external devices include, for example, other information processing devices and external storage devices connected via a computer network, etc. The auxiliary storage unit 23 may be, for example, part of a cloud system, which is a group of computers on a network.

[0027] The auxiliary storage unit 23 is, for example, an erasable programmable ROM (EPROM), a solid state drive (SSD), a hard disk drive (HDD), etc. The auxiliary storage unit 23 is, for example, a compact disc (CD) drive, a digital versatile disc (DVD) drive, a Blu-ray (registered trademark) Disc (BD) drive device, etc. The auxiliary storage unit 23 may be provided by a network attached storage (NAS) or a storage area network (SAN).

[0028] The communication unit 24 is, for example, an interface with the computer network N1, and communicates with external devices such as the camera 1 via the computer network N1.

[0029] The connection terminal 25 is a connection terminal to which the connection cable L1 is connected. When the detection device 2 detects a falling object, it outputs an alarm to the display device 3 via the connection cable L1.

[0030] <Detection device 2 processing block> 3 is a diagram illustrating an example of processing blocks of the detection device 2 according to the embodiment. The detection device 2 includes a foreground extraction unit 201, a preprocessing unit 202, a tracking unit 203, an anomaly detection unit 204, a parameter adjustment unit 205, a correct / incorrect acceptance unit 206, a candidate extraction unit 207, an anomaly determination unit 208, an evaluation unit 209, a data accumulation unit 210, and a parameter management unit 211. The detection device 2 executes the processes of each unit of the detection device 2, such as the foreground extraction unit 201, the preprocessing unit 202, the tracking unit 203, the anomaly detection unit 204, the parameter adjustment unit 205, the correct / incorrect acceptance unit 206, the candidate extraction unit 207, the anomaly determination unit 208, the evaluation unit 209, the data accumulation unit 210, and the parameter management unit 211, by the CPU 21 executing a computer program executable by the main storage unit 22.

[0031] The foreground extraction unit 201 , the preprocessing unit 202 , the tracking unit 203 and the anomaly detection unit 204 are processing blocks that are used when the falling object detection system 100 is operated mainly to detect falling objects on the road 500 .

[0032] The foreground extraction unit 201 generates an extracted image by extracting the foreground from a frame image in the video data received from the camera 1. The foreground is extracted, for example, by a background subtraction method. FIG. 4 is a diagram showing an example of foreground extraction by the foreground extraction unit 201. In the example of FIG. 4, a frame image I1 at time T and a frame image I2 at time T+1, which is the time immediately following time T, are input to the foreground extraction unit 201. Frame image I1 shows only a road 500, while frame image I2 shows a box O1 and a car O2 on the road 500. The foreground extraction unit 201 extracts the box O1 as the foreground based on the difference between frame image I1 and frame image I2, and generates an extracted image D1 including the box O1 and the car O2. The extracted image D1 is, for example, an image in which the box O1 and the car O2 extracted as the foreground are shown in white, and the road 500 in the background is removed and shown in black. That is, the extracted image D1 is a black and white binarized image.

[0033] Returning to FIG. 3, the tracking unit 203 tracks the foreground (e.g., a box O1 and a passenger car O2) extracted by the foreground extraction unit 201. The tracking unit 203, for example, sets a rectangular tracking frame that surrounds the foreground and tracks the foreground using the tracking frame. The tracking unit 203 also generates a tracking extraction image by extracting the area surrounded by the tracking frame from the extraction image D1, and stores the image in the data accumulation unit 210. The extraction image D1 may contain noise due to, for example, dust on the lens of the camera 1 or the influence of the image sensor of the camera 1.

[0034] FIG. 5 is a diagram showing an example of tracking extraction images G1 and G2 generated by the tracking unit 203 in the embodiment. The tracking unit 203 sets a tracking frame W1 that surrounds the periphery of a box O1 in the extraction image D1, and also sets a tracking frame W2 that surrounds the periphery of a passenger car O2. The tracking unit 203 sets a tracking frame W1 that surrounds the periphery of a box O1 in the extraction image D1, and also sets a tracking frame W2 that surrounds the periphery of a passenger car O2. The tracking unit 203 extracts an area surrounded by a tracking frame W1 to generate a tracking extraction image G1. The tracking unit 203 also extracts an area surrounded by a tracking frame W2 from the extraction image D1 to generate a tracking extraction image G2. Because the extraction image D1 is a binarized image, the tracking extraction images G1 and G2 extracted from the extraction image D1 are also binarized images. Because the extraction image D1 may contain noise, the tracking extraction images G1 and G2 may also contain noise.

[0035] The tracking unit 203 stores information related to each of the tracking extraction images G1 and G2 in the data storage unit 210. The data storage unit 210 is, for example, a storage area (storage unit) constructed in the auxiliary storage unit 23. FIG. 6 is a diagram showing an example of a tracking management table 2101 stored in the data storage unit 210 in this embodiment. The tracking management table 2101 has the following fields: "No.", "Start," "End," "Image Series," and "Coordinate Series." "No." stores a tracking number uniquely assigned to each tracking frame set by the tracking unit 203. "Start" stores information indicating the frame image at which setting of the tracking frame began. Examples of information indicating the frame image include a frame number uniquely assigned to each frame image. "End" stores information indicating the frame image at which setting of the tracking frame ended (the frame image at which the tracking frame was last set). "Image Series" stores information indicating the tracking extraction image. Examples of information indicating the tracking extraction image include a number uniquely assigned to each tracking extraction image, and the path name of the tracking extraction image stored in the auxiliary storage unit 23. The "coordinate series" stores information indicating the position of the tracking frame. Examples of information indicating the position of the tracking frame include a combination of the coordinates (starting point coordinates) of one of the four corners of the rectangular tracking frame, the width of the tracking frame, and the height of the tracking frame.

[0036] 3, the preprocessing unit 202 performs predetermined preprocessing on the tracking extraction images G1 and G2 extracted by the tracking unit 203. Examples of preprocessing include erosion processing of the tracking extraction images G1 and G2, median filtering, gamma correction, and acquisition of Histograms of Oriented Gradients (HOG) feature values. Examples of erosion processing include dilation processing, erosion processing, erosion-expansion processing, and dilation-erosion processing.

[0037] The expansion process, for example, changes pixels surrounding white pixels in the tracking extraction images G1 and G2 to white pixels. The erosion process, for example, changes pixels surrounding black pixels in the tracking extraction images G1 and G2 to black pixels. The erosion / expansion process is a process in which an erosion process is performed a predetermined number of times, followed by an expansion process a predetermined number of times. The expansion / erosion process is a process in which an erosion process is performed a predetermined number of times, followed by an erosion process a predetermined number of times. The median filter is a process in which the density of a pixel in the tracking extraction images G1 and G2 is replaced with the median density of the surrounding pixels. The process of applying a median filter is an example of a smoothing process. The gamma correction corrects the pixel values ​​(pixel brightness) of the tracking extraction images G1 and G2. By correcting the pixel values, it is possible to emphasize or suppress differences between colors. By performing at least one of the following pre-processing steps in the preprocessing: erosion processing, expansion processing, erosion / expansion processing, expansion / erosion processing, median filter, and gamma correction, it is possible to reduce noise in the tracking extraction images G1 and G2, and to make the box O1 in the tracking extraction image G1 and the automobile O2 in the tracking extraction image G2 clearer.

[0038] In acquiring HOG features, the tracking extracted images G1 and G2 are converted into gradient images. In the gradient images, the tracking extracted images G1 and G2 are divided into cells. Each cell contains multiple pixels of the tracking extracted images G1 and G2. In the gradient images, the gradient of the pixels in each cell (e.g., the gradient of brightness in each cell) is represented by, for example, a histogram. Preprocessing Unit The preprocessing unit 202 acquires gradient-related features for each cell as HOG features. The HOG features allow, for example, the presence or absence of changes in texture between the tracking-extracted images G1 and G2 to be grasped. The preprocessing unit 202 can clarify the presence or absence of changes in texture by using the HOG features. For example, there is almost no change in texture for a falling object, but a change in texture is detected for objects that move easily in the wind, such as leaves or plastic bags.

[0039] The parameter management unit 211, which will be described later, stores parameters related to preprocessing. Examples of the parameters related to preprocessing include the number of times each of the dilation process, erosion process, and dilation / erosion process is executed, and the filter size indicating the size of the filter used in each of the dilation process, erosion process, and dilation / erosion process. Examples of the parameters related to preprocessing include the gamma value in gamma correction. Examples of the parameters related to preprocessing include the number of pixels included in a cell when acquiring HOG features.

[0040] The pre-processing unit 202 executes pre-processing in accordance with the parameters stored in the parameter management unit 211. For example, the pre-processing unit 202 may execute an erosion process twice and then an expansion process twice on the tracking extraction images G1 and G2 in accordance with the parameters stored in the parameter management unit 211. In this embodiment, it is assumed that an erosion and expansion process is executed as pre-processing. A method for determining the parameters used in pre-processing will be described later.

[0041] The anomaly detection unit 204 detects fallen objects on the road 500. Unlike a passenger car O2, a fallen object such as a box O1 on the road 500 is considered to be immobile. Furthermore, unlike noise contained in the tracking extraction images G1 and G2, a fallen object such as the box O1 is considered to be often detected as a single mass. Therefore, the anomaly detection unit 204 detects, for example, an object with no (little) positional fluctuation and an area on the tracking extraction images G1 and G2 that is equal to or greater than a predetermined threshold as a fallen object. Upon detecting a fallen object, the anomaly detection unit 204 notifies the traffic monitor M1 of the occurrence of the fallen object by outputting a message on the display device 3 notifying the detection of a fallen object.

[0042] Here, an example of detection and false detection of falling objects due to differences in parameters will be described. FIGS. 7A to 7C are diagrams illustrating the detection frequency of noise and falling objects when contraction and expansion processing is performed as preprocessing in an embodiment. In FIGS. 7A to 7C, image data including noise and image data including falling objects are used as test data, and the anomaly detection unit 204 performs detection of falling objects. Compared to noise, falling objects are often detected as a single mass. Therefore, in the examples of FIGS. 7A to 7C, the anomaly detection unit 204 illustrates the detection frequency of falling objects and noise based on area.

[0043] 7A to 7C, the vertical axis illustrates frequency, and the horizontal axis illustrates area. In Figures 7A to 7C, graph R1 illustrates the frequency with which anomaly detection unit 204 detected image data containing falling objects as falling objects when the image data was input to anomaly detection unit 204. Graph H1 illustrates the frequency with which anomaly detection unit 204 detected image data containing noise as falling objects when the image data was input to anomaly detection unit 204.

[0044] Fig. 7A illustrates an example in which the erosion / expansion process is performed once, Fig. 7B illustrates an example in which the erosion / expansion process is performed twice, and Fig. 7C illustrates an example in which the erosion / expansion process is performed three times.

[0045] In Fig. 7A and Fig. 7C, there is an overlapping region K1 where the graph R1 and the graph H1 overlap. In the overlapping region, noise is detected as a falling object, or the falling object is overlooked as noise. In Fig. 7B, there is no overlapping region K1. Therefore, by setting the threshold value TH1, the anomaly detection unit 204 can detect a falling object separately from noise. Note that the threshold value TH1 also 7A to 7C are examples of parameters for contraction / expansion processing. Although contraction / expansion processing has been described as an example in FIGS. 7A to 7C, the accuracy of falling object detection varies depending on the parameters set in other preprocessing processes as well. Therefore, in this embodiment, the following configuration is adopted to set parameters suitable for falling object detection.

[0046] The parameter adjustment unit 205, the correctness / error acceptance unit 206, the candidate extraction unit 207, the abnormality determination unit 208, the evaluation unit 209, and the parameter management unit 211 are processing blocks mainly used for parameter adjustment used in preprocessing by the preprocessing unit 202.

[0047] The parameter adjustment unit 205 receives designation of parameters for each pre-processing operation executed by the pre-processing unit 202. The parameter adjustment unit 205 may receive, for example, designation of minimum and maximum values ​​for each parameter (i.e., designation of a parameter range).

[0048] The correctness / error receiving unit 206 receives a designation of whether or not each tracking extraction image managed by the tracking management table 2101 is a falling object. The correctness / error receiving unit 206, for example, associates information on whether or not each tracking extraction image is a falling object with each tracking extraction image and stores the information in the auxiliary storage unit 23. FIG. 8 is a diagram showing an example of a correctness / error information management table 2061 stored in the auxiliary storage unit 23 by the correctness / error receiving unit 206. The correctness / error information management table 2061 has the following fields: "No.", "Start," "End," "Image Series," "Correctness / Error," and "Judgment." The correctness / error information management table 2061 is, for example, a tracking management table 2101 with the addition of the fields "Correctness / Error" and "Judgment."

[0049] The "No.", "Start", "End", and "Image Series" in the correct / incorrect information management table 2061 are the same as those in the tracking management table 2101, and therefore a description thereof will be omitted. "Correct / incorrect" stores information indicating whether or not the image is a falling object that has been accepted by the correct / incorrect accepting unit 206. In the example of FIG. 8, "correct / incorrect" stores "1" if the image is a falling object, and "0" if the image is not a falling object. "Determination" stores the determination result by the abnormality determining unit 208. The example of FIG. 8 illustrates a state in which the abnormality determining unit 208 has not yet made a determination. Therefore, in the example of FIG. 8, the "Determination" of the abnormality determining unit 208 is left blank.

[0050] The candidate extraction unit 207 extracts tracking numbers used to track fallen object candidates from the tracking numbers managed by the accuracy information management table 2061. Examples of tracking numbers used to track fallen object candidates include tracking numbers that have been tracked continuously for a predetermined period of time or more. The candidate extraction unit 207 stores the extracted tracking numbers in the auxiliary storage unit 23.

[0051] The anomaly determination unit 208 determines whether or not each of the tracking numbers extracted by the candidate extraction unit 207 is the result of tracking a fallen object. For example, the anomaly determination unit 208 causes the preprocessing unit 202 to perform preprocessing using parameters accepted by the parameter adjustment unit 205 on the tracking extraction images associated with each of the tracking numbers extracted by the candidate extraction unit 207. The anomaly determination unit 208 causes the anomaly detection unit 204 to perform falling object detection on each of the preprocessed tracking images. The anomaly determination unit 208 stores the determination result in the accuracy information management table 2061.

[0052] 9 is a diagram showing an example of the correctness information management table 2061 to which the determination result by the abnormality determination unit 208 has been added in the embodiment. The determination result by the abnormality determination unit 208 is stored in the "determination" field of the correctness information management table 2061, and in the example of FIG. 9, "1" is stored if a falling object is detected, and "0" is stored if a falling object is not detected.

[0053] The evaluation unit 209 compares the detection result of the tracking number by the abnormality determination unit 208 with the normal The abnormality determination unit 208 evaluates the result of the falling object detection based on information indicating whether or not each of the tracking extraction images stored in the auxiliary storage unit 23 by the error acceptance unit 206 is a falling object. The evaluation unit 209 generates a confusion matrix that associates, for example, the number of tracking numbers corresponding to the presence or absence of a falling object accepted by the error acceptance unit 206 with the number of tracking numbers corresponding to the presence or absence of a falling object detected by the abnormality determination unit 208.

[0054] FIG. 10 is a diagram illustrating an example of a confusion matrix 2091 generated by the evaluation unit 209 in the embodiment. The confusion matrix 2091 has entries of "absence" and "presence" associated with "correct answer" and entries of "absence" and "presence" associated with "determination." "absence" associated with "correct answer" stores the number of tracking numbers associated with tracking extract images designated by the correct / incorrect accepting unit 206 as not falling objects. "presence" associated with "correct answer" stores the number of tracking numbers associated with tracking extract images designated by the correct / incorrect accepting unit 206 as falling objects. "absence" associated with "determination" stores the number of tracking numbers associated with tracking extract images in which no falling objects were detected by the anomaly determining unit 208. "presence" associated with "determination" stores the number of tracking numbers associated with tracking extract images in which a falling object was detected by the anomaly determining unit 208.

[0055] The evaluation unit 209 evaluates the determination result by the anomaly determination unit 208 with reference to the confusion matrix 2091. The evaluation by the evaluation unit 209 can use, for example, various evaluation indices for binary classification. Examples of evaluation indices for binary classification used by the evaluation unit 209 include Accuracy, Precision, Recall, Fβ score, and Matthews correlation coefficient. The evaluation unit 209 stores the evaluation results based on the evaluation indices obtained by erosion and expansion processing using various parameters in association with the parameters in the auxiliary storage unit 23.

[0056] FIG. 11 is a diagram showing an example of an evaluation result table 2092 that the evaluation unit 209 stores in the auxiliary storage unit 23 in the embodiment. The evaluation result table 2092 has items of "parameter" and "evaluation result." "Parameter" stores information indicating a parameter specified by the parameter adjustment unit 205. The information indicating a parameter may be the parameter itself or an ID associated with the parameter. "Evaluation result" stores evaluation results based on various evaluation indices. The evaluation unit 209, for example, refers to the evaluation result table 2092 to identify the parameter with the highest evaluation result, and stores the identified parameter in the parameter management unit 211.

[0057] The parameter management unit 211 stores parameters used in various preprocessing operations executed by the preprocessing unit 202. The parameter management unit 211 is, for example, a storage area constructed in the auxiliary storage unit 23. The parameter management unit 211 stores parameters in association with the types of preprocessing operations, for example.

[0058] (Processing flow) Fig. 12 is a diagram showing an example of a processing flow of the detection device 2 according to the embodiment. Fig. 12 illustrates an overall processing flow for determining parameters suitable for preprocessing by the detection device 2. Hereinafter, an example of the processing flow of the detection device 2 will be described with reference to Fig. 12.

[0059] In step S1, the tracking extraction images tracked by the tracking unit 203 are stored in the tracking management table 2101. Furthermore, the correct / incorrect receiving unit 206 receives information on whether or not each of the tracking extraction images is a falling object. The correct / incorrect receiving unit 206 generates the correct / incorrect information management table 2061 by associating the tracking extraction images with the information on whether or not each of the tracking extraction images is a falling object.

[0060] In step S2, parameter adjustment section 205 accepts parameter designation, which is assumed to be a parameter range designation.

[0061] In step S3, the candidate extractor 207 extracts candidates for tracking numbers that include images of fallen objects. Details of the processing in step S3 will be described later with reference to FIG.

[0062] In step S4, the evaluation unit 209 evaluates the result of the detection of the fallen object by the abnormality determination unit 208, thereby searching for suitable parameters for the pre-processing. Details of the processing in step S4 will be described later with reference to FIG.

[0063] Fig. 13 is a diagram showing an example of processing by the candidate extraction unit 207 in the embodiment. Fig. 13 illustrates an example of details of the processing of step S3 in Fig. 12. Hereinafter, with reference to Fig. 13, an example of processing by the candidate extraction unit 207 will be described.

[0064] The processing from step S31 to step S32 is executed for all tracking numbers stored in the tracking management table 2101. In step S31, the candidate extraction unit 207 refers to the tracking management table 2101 and acquires a tracking number.

[0065] In step S32, the candidate extraction unit 207 determines whether the tracking number acquired in step S31 is a tracking number for tracking a fallen object candidate. The candidate extraction unit 207 extracts, for example, a tracking number that has been tracked continuously for a predetermined period of time or more as a tracking number for tracking a fallen object candidate.

[0066] In step S33, the candidate extraction unit 207 stores the tracking number extracted in step S32 in the auxiliary storage unit 23 as the tracking number used to track the fallen object candidate.

[0067] Fig. 14 is a diagram showing an example of processing by the abnormality determination unit 208 and the evaluation unit 209 in the embodiment. Fig. 14 illustrates an example of details of the processing of step S4 in Fig. 12. Hereinafter, an example of processing by the abnormality determination unit 208 and the evaluation unit 209 will be described with reference to Fig. 14.

[0068] In step S41, the preprocessing unit 202 performs an erosion / expansion process on the tracking extraction image associated with the tracking number stored in the auxiliary storage unit 23 in step S33 of Fig. 13, using parameters within the range specified in step S2 of Fig. 12. The abnormality determination unit 208 determines whether the tracking extraction image that has been subjected to the erosion / expansion process is an image of a fallen object.

[0069] In step S42, the evaluation unit 209 evaluates the falling object detection result by the abnormality determination unit 208 and calculates an evaluation result based on the detection result in step S41 and information indicating whether each of the tracking extraction images is a falling object or not, which information has been stored in the auxiliary storage unit 23 by the true / false acceptance unit 206. The evaluation unit 209 stores the calculated evaluation result in an evaluation result table 2092 in association with the parameters used in the preprocessing in step S41.

[0070] In step S43, evaluation unit 209 determines whether the evaluation result in step S42 is the best evaluation result among the evaluation results stored in evaluation result table 2092. If it is the best evaluation result (YES in step S43), the process proceeds to step S44. If it is not the best evaluation result (NO in step S43), the process proceeds to step S45.

[0071] In step S44, the evaluation unit 209 causes the parameter management unit 211 to store the parameters used in the contraction / expansion processing in step S41.

[0072] In step S45, the preprocessing unit 202 changes the parameters used in the erosion / expansion processing within the range specified in step S2 of FIG.

[0073] <Effects of the embodiment> In this embodiment, the parameters used in the preprocessing by the preprocessing unit 202 are adjusted according to the processing flow illustrated in FIG. 14. That is, the detection device 2 stores in the parameter management unit 211 the parameters set for the tracking extraction image with the highest detection accuracy among the tracking extraction images that have undergone erosion and expansion processing with various parameters set. Therefore, according to this embodiment, parameters with high detection accuracy can be easily determined. Note that when the detection device 2 is in operation, the erosion and expansion processing is performed using the parameters stored in the parameter management unit 211.

[0074] In this embodiment, the tracking extraction image, rather than the frame image itself, is stored in the auxiliary storage unit 23. Therefore, the data size of the tracking extraction image is smaller than that of the frame image. Therefore, according to this embodiment, it is possible to reduce the storage capacity of the image data used for parameter determination.

[0075] In this embodiment, the true / false acceptance unit 206 assigns information on whether or not each tracking extraction image is a falling object. The evaluation unit 209 can then determine parameters based on the frequency with which a falling object is detected as a falling object and the frequency with which an object other than a falling object is detected as a falling object. Therefore, this embodiment prevents noise and the like other than a falling object from being erroneously detected as a falling object.

[0076] In this embodiment, the parameter range is specified by the parameter adjustment unit 205. Therefore, according to this embodiment, it is possible to prevent the processing time required for determining the parameters from becoming enormous.

[0077] <Modification> In this embodiment, the true / false acceptance unit 206 assigns information to each tracking extraction image as to whether it is a falling object or not, but the candidate extraction unit 207 and the abnormality determination unit 208 may perform falling object detection using a tracking extraction image containing a falling object, and the evaluation unit 209 may evaluate the detection accuracy to determine the parameters.

[0078] In this embodiment, fallen objects on the road 500 are the detection target, but areas other than the road 500 (railroad tracks, parks, etc.) may be the monitoring area, and objects other than fallen objects (such as signs installed without permission) may be the detection target.

[0079] In this embodiment, an object that does not move for a predetermined time is detected as a falling object, but a predetermined detection target such as a falling object may be detected under other conditions. As other conditions, for example, an image of the predetermined detection target may be stored in the auxiliary storage unit 23 in advance, and an object whose shape matches the stored image may be detected as the predetermined detection target.

[0080] When determining parameters for multiple preprocessing processes (for example, erosion processing and gamma correction), the processing time required to determine the parameters may be long. In such cases, the processing time required to determine the parameters can be reduced by optimizing the parameter search using, for example, Bayesian optimization. An example of Bayesian optimization that can be applied to this embodiment is the Tree-Structured Parzen Estimator (TPE). do.

[0081] The embodiments and modifications disclosed above can be combined with each other.

[0082] <Computer-readable recording medium> An information processing program that causes a computer or other machine or device (hereinafter referred to as a computer, etc.) to realize any of the above functions can be recorded on a computer-readable recording medium. Then, by having the computer, etc. read and execute the program from this recording medium, the function can be provided.

[0083] Here, a computer-readable recording medium refers to a recording medium that stores information such as data and programs electrically, magnetically, optically, mechanically, or chemically and that can be read by a computer. Among such recording media, those that are removable from a computer include, for example, flexible disks, magneto-optical disks, Compact Disc Read Only Memory (CD-ROM), Compact Disc-Recordable (CD-R), Compact Disc-Rewriteable (CD-RW), Digital Versatile Disc (DVD), Blu-ray Disc (BD), Digital Audio Tape (DAT), 8mm tape, flash memory, external hard disk drives, and solid state drives (SSDs). Furthermore, recording media that are fixed to a computer include built-in hard disk drives, SSDs, ROMs, etc.

[0084] <Appendix 1> A process of acquiring a plurality of image data (G1, G2, I1, I2) including a predetermined detection object (O1); A process of applying an image filter, each having a plurality of parameters set, to each of the plurality of image data (G1, G2, I1, I2) to generate a plurality of filtered image data; A process of detecting the predetermined detection object (O1) from the plurality of filtered image data; a process of storing, in a storage unit, the parameters of the image filter set for the filtered image data having the highest detection accuracy in the detection process; a processor (21) that executes a process of detecting the predetermined detection object (O1) by setting the image filter to which the parameters stored in the storage unit (211) are set for image data (G1, G2, I1, I2) acquired from a camera (1) that photographs a monitoring area (500), Image processing device (2). <Appendix 2> The image data (G1, G2, I1, I2) are image data (G1, G2) obtained by cutting out the region (W1) of the predetermined detection object (O1) from the captured frame image (I1, I2). 2. The image processing device according to claim 1. <Appendix 3> The plurality of image data including the predetermined detection object (O1) include a first image data group (image group of graph R1) including the predetermined detection object (O1) and a second image data group (image group of graph H1) from which the predetermined detection object is excluded, The process of storing the parameters in the storage unit (211) determines the parameters to be stored in the storage unit based on a first frequency at which the predetermined detection object (O1) is detected from the first image data group (image group of graph R1) in the detection process and a second frequency at which the predetermined detection object (O1) is detected from the second image data group (image group of graph H1) in the detection process. An image processing device (2) according to appendix 1 or 2. <Appendix 4> the image filter includes at least one of an expansion process, an erosion process, a smoothing process, and an expansion / erosion process; An image processing device (2) according to any one of appendices 1 to 3. <Appendix 5> The processor (21) accepts a specification of a range of the parameter; the plurality of parameters are parameters within the domain; An image processing device (2) according to any one of appendices 1 to 4. <Appendix 6> the parameters include at least one of the number of times the image filter is set to the image data and the filter size of the image filter. An image processing device (2) according to any one of appendices 1 to 5. <Appendix 7> A process of acquiring a plurality of image data (G1, G2, I1, I2) including a predetermined detection object (O1); A process of applying an image filter, each having a plurality of parameters set, to each of the plurality of image data (G1, G2, I1, I2) to generate a plurality of filtered image data; A process of detecting the predetermined detection object (O1) from the plurality of filtered image data; a process of storing, in a storage unit, the parameters of the image filter set for the filtered image data having the highest detection accuracy in the detection process; A computer (2) executes a process of detecting the predetermined detection object by setting the image filter to which the parameters stored in the storage unit are set for image data (G1, G2, I1, I2) acquired from a camera (500) that photographs a monitoring area (500). Image processing methods. <Appendix 8> A process of acquiring a plurality of image data (G1, G2, I1, I2) including a predetermined detection object (O1); A process of applying an image filter, each having a plurality of parameters set, to each of the plurality of image data (G1, G2, I1, I2) to generate a plurality of filtered image data; A process of detecting the predetermined detection object (O1) from the plurality of filtered image data; a process of storing, in a storage unit, the parameters of the image filter set for the filtered image data having the highest detection accuracy in the detection process; setting the image filter, to which the parameters stored in the storage unit (211) are set, for image data (G1, G2, I1, I2) acquired from a camera (1) that photographs a monitoring area (500), and detecting the predetermined detection object (O1); Image processing program. [Explanation of symbols]

[0085] 1. Camera 2. Detection device 3...Display device 21 CPU 22...Main memory 23...Auxiliary storage section 24. Communications Department 25 Connection terminal 100··Falling Object Detection System 201...Foreground extraction part 202··Pretreatment section 203 Tracking section 204 Abnormality detection unit 205··Parameter adjustment section 206··Corrections Department 207··Candidate Extraction Unit 208...Abnormality determination section 209··Evaluation Section 210 Data storage unit 211··Parameter Management Unit 500...road 2101 Tracking Management Table 2061 Errata Management Table 2091...confusion matrix 2092··Evaluation result table B1 Bus D1 Extracted image G1 Tracking extracted image G2 Tracking extracted image I1··Frame image I2··Frame Image L1 Connection cable M1...Traffic warden N1: Computer Network O1··Box O2··Passenger car W1 Tracking Frame W2 Tracking Frame

Claims

1. A process of acquiring a plurality of image data including a predetermined detection target; A process of applying an image filter, each having a plurality of parameters set, to each of the plurality of image data to generate a plurality of filtered image data; a process of detecting the predetermined detection target from the plurality of filtered image data; a process of storing, in a storage unit, the parameters of the image filter set for the filtered image data having the highest detection accuracy in the detection process; a processor that executes a process of detecting the predetermined detection target by setting the image filter to which the parameters stored in the storage unit are set for image data acquired from a camera that captures an image of a monitoring area, Image processing device.

2. The image data is image data obtained by cutting out an area of ​​the predetermined detection object from a captured frame image. The image processing device according to claim 1 .

3. the plurality of image data including the predetermined detection object include a first image data group including the predetermined detection object and a second image data group from which the predetermined detection object is excluded; The process of storing the parameters in the storage unit determines the parameters to be stored in the storage unit based on a first frequency at which the predetermined detection object is detected from the first image data group in the detection process and a second frequency at which the predetermined detection object is detected from the second image data group in the detection process. The image processing device according to claim 1 .

4. the image filter includes at least one of an expansion process, an erosion process, a smoothing process, and an expansion / erosion process; The image processing device according to claim 1 .

5. The processor accepts a specification of a domain of the parameter; the plurality of parameters are parameters within the domain; The image processing device according to claim 1 .

6. the parameters include at least one of the number of times the image filter is set to the image data and the filter size of the image filter. The image processing device according to claim 1 .

7. A process of acquiring a plurality of image data including a predetermined detection target; A process of applying an image filter, each having a plurality of parameters set, to each of the plurality of image data to generate a plurality of filtered image data; a process of detecting the predetermined detection target from the plurality of filtered image data; a process of storing, in a storage unit, the parameters of the image filter set for the filtered image data having the highest detection accuracy in the detection process; a process of detecting the predetermined detection target by setting the image filter, to which the parameters stored in the storage unit are set, for image data acquired from a camera that captures an image of a monitoring area, by a computer; Image processing methods.

8. A process of acquiring a plurality of image data including a predetermined detection target; Applying an image filter having a plurality of parameters set to each of the plurality of image data. generating a plurality of filtered image data; a process of detecting the predetermined detection target from the plurality of filtered image data; a process of storing, in a storage unit, the parameters of the image filter set for the filtered image data having the highest detection accuracy in the detection process; setting the image filter, to which the parameters stored in the storage unit are set, for image data acquired from a camera that captures an image of a monitoring area, and detecting the predetermined detection target; Image processing program.

Images