OBJECT DETECTION APPARATUS, OBJECT DETECTION METHOD AND PROGRAM

Abstract

To provide an object detection device that efficiently creates high-quality teacher data.SOLUTION: An object detection device 300 includes: a detection unit 301 that detects an object from an input image using a dictionary; a reception unit 302 that displays the input image on a display device with a display that emphasizes a subregion of the detected object, and receives selection of the subregion and input of a class for the selected subregion by one operation of an input device; a generation device 303 that generates teacher data from an image of the selected subregion and the input class; and a learning unit 304 that learns a dictionary from the teacher data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an object detection device that detects an object from an image using a dictionary. [Background technology]

[0002] Apparatuses have been proposed that analyze images captured by an imaging device to detect objects. Patent Document 1 discloses a technique for analyzing images captured by an imaging device during the manufacturing process of a board such as a printed wiring board, and detecting objects such as scratches on the board. Patent Document 2 discloses a technique for analyzing monitor images of a road captured by a surveillance camera to detect objects such as vehicles.

[0003] In the techniques disclosed in Patent Documents 1 and 2, an object detection device is trained to perform the above-mentioned detection. For this training, teacher data (also called training data) is required. Teacher data is examples of pairs of input and output. There are two types of teacher data: positive examples and negative examples. To train an object detection device correctly, both positive examples and negative examples are necessary. However, creating appropriate teacher data requires a lot of time and effort.

[0004] Patent Document 1 discloses a technique for creating training data required for detecting objects such as scratches on a board. In this technique, regions having brightness values ​​different from those of non-defective products are extracted from an image of a printed wiring board and displayed on a display, and the selection of the region and input of its class (also called category) are received from the user using a keyboard and mouse. Specifically, the user selects one specific region from among multiple existing regions by clicking the mouse, and then selects the desired class from a pull-down menu that is displayed at the time of selection by clicking the mouse.

[0005] Patent Document 2 discloses a technique for creating training data required for detecting objects such as vehicles traveling on a road. In this technique, a dictionary is used to cut out an object area from an arbitrary captured image, a predetermined feature amount is extracted from the area, and the dictionary is trained based on the extracted feature amount. This series of operations is performed automatically. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2003-317082 A [Patent Document 2] JP 2014-59729 A Summary of the Invention [Problem to be solved by the invention]

[0007] However, in the technology disclosed in Patent Document 1, the user must select the area to be used for creating the training data and input the class for the selected area by separately operating an input device, which makes the technology inefficient.

[0008] On the other hand, the technology disclosed in Patent Document 2 automatically extracts object regions, extracts features from the regions, and learns a dictionary using the extracted features, so that training data can be created efficiently. However, the regions and classes of objects extracted using the dictionary are not always correct, which deteriorates the quality of the training data.

[0009] An object of the present invention is to solve the above-mentioned problems, that is, to provide an object detection device and the like that efficiently creates high-quality training data. [Means for solving the problem]

[0010] In order to solve the above-mentioned problems, a first feature of the present invention is to a detection unit for detecting an object from an input image using the dictionary; a reception unit that displays an input image on a display device with a display that highlights a partial region of a detected object, and receives a selection of a partial region and an input of a class for the selected partial region by a single operation of the input device; A generation unit that generates training data from an image of the selected partial region and an input class; A learning unit that learns a dictionary using training data; The object detection apparatus has the following features.

[0011] The second feature of the present invention is Detect objects from the input image using the dictionary, Displaying the input image on a display device with a display that highlights a partial region of the detected object; accepting a selection of a subregion and an input of a class for the selected subregion through one operation of an input device; Generate training data from the image of the selected subregion and the input class; This is an object detection method that learns a dictionary using training data.

[0012] The third feature of the present invention is Computer, a detection unit for detecting an object from an input image using the dictionary; a reception unit that displays an input image on a display device with a display that highlights a partial region of a detected object, and receives a selection of a partial region and an input of a class for the selected partial region by a single operation of the input device; A generation unit that generates training data from an image of the selected partial region and an input class; A learning unit that learns a dictionary using training data; It is a recording medium that stores a program that causes the device to function as described above. Effect of the Invention

[0013] Since the present invention has the above-mentioned configuration, it is possible to efficiently create high-quality teacher data. [Brief description of the drawings]

[0014] [Figure 1] 1 is a block diagram of an object detection device according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is an explanatory diagram of the concept of detecting an object from an image in the first embodiment of the present invention. [Diagram 3] FIG. 4 is a diagram showing an example of an object detection result in the first embodiment of the present invention. [Figure 4] FIG. 2 is a diagram showing an example of a reception screen in the first embodiment of the present invention. [Diagram 5] 5 is a flowchart showing an example of the operation of the object detection device according to the first embodiment of the present invention. [Figure 6] 5 is a flowchart showing details of the operation of a reception unit of the object detection device according to the first embodiment of the present invention. [Figure 7] FIG. 11 is a diagram showing an example of a combination of gesture types in a modified example of the first embodiment of the present invention. [Figure 8] FIG. 11 is a diagram showing an example of a combination of gesture types in a modified example of the first embodiment of the present invention. [Figure 9] FIG. 11 is a diagram showing an example of a reception screen in the second embodiment of the present invention. [Figure 10A] 13A and 13B are diagrams illustrating the meaning of drag-and-drop in the lower right direction used in the second embodiment of the present invention and an example of calculating a partial region. [Figure 10B] 13A and 13B are diagrams illustrating the meaning of drag-and-drop in the lower left direction used in the second embodiment of the present invention and an example of calculating a partial region. [Figure 11] 10 is a flowchart showing details of the operation of a reception unit of an object detection device according to a second embodiment of the present invention. [Figure 12A] FIG. 11 is a diagram showing an example of a detection result before being updated by a reception unit of an object detection device according to a second embodiment of the present invention. [Figure 12B] FIG. 11 is a diagram showing an example of a detection result after being updated by a receiving unit of an object detection device according to a second embodiment of the present invention. [Figure 13A] FIG. 11 is a diagram showing an example of an image window before it is updated by a reception unit of an object detection device according to a second embodiment of the present invention. [Figure 13B] FIG. 11 is a diagram showing an example of an image window after being updated by a reception unit of an object detection device according to a second embodiment of the present invention. [Figure 14] FIG. 13 is a diagram showing an example of a combination of gesture types in a modified example of the second embodiment of the present invention. [Figure 15] FIG. 13 is an explanatory diagram of a method for calculating a partial area of ​​class 2 from the position of a right double click performed on an image in a modified example of the second embodiment of the present invention. [Figure 16A] 13A and 13B are diagrams illustrating the meaning of a swipe in the lower right direction and an example of calculating a partial region used in a modified example of the second embodiment of the present invention. [Figure 16B] 13A and 13B are diagrams illustrating an example of calculation of a partial region and the meaning of a swipe in the lower left direction used in a modified example of the second embodiment of the present invention. [Figure 17A] 13A and 13B are diagrams illustrating an example of calculation of a partial region and the meaning of a downward swipe used in a modified example of the second embodiment of the present invention. [Figure 17B] 13A and 13B are diagrams illustrating an example of calculation of a partial region and the meaning of an upward swipe used in a modified example of the second embodiment of the present invention. [Figure 18] FIG. 13 is a diagram showing an example of a combination of gesture types in a modified example of the second embodiment of the present invention. [Figure 19] 13 is a flowchart showing details of the operation of a reception unit of an object detection device according to a third embodiment of the present invention. [Figure 20] FIG. 13 is a diagram showing an example of a reception screen used in the fourth embodiment of the present invention. [Figure 21]FIG. 13 is a diagram showing an example in which a receiving unit of an object detection device according to a fourth embodiment of the present invention receives an input of class 0 for an existing partial region that partially overlaps with a newly calculated partial region. [Figure 22] 13 is a flowchart showing details of the operation of a reception unit of an object detection device according to a fourth embodiment of the present invention. [Figure 23A] FIG. 13 is a diagram showing an example of a detection result before being updated by a reception unit of an object detection device according to a fourth embodiment of the present invention. [Figure 23B] FIG. 13 is a diagram showing an example of a detection result after being updated by a receiving unit of an object detection device according to a fourth embodiment of the present invention. [Figure 24A] FIG. 13 is a diagram showing an example of an image window before it is updated by a reception unit of an object detection device according to a fourth embodiment of the present invention. [Figure 24B] FIG. 13 is a diagram showing an example of an image window after being updated by a reception unit of an object detection device according to a fourth embodiment of the present invention. [Diagram 25] FIG. 13 is a block diagram of an object detection device according to a fifth embodiment of the present invention. [Figure 26] 1 is a diagram illustrating an example of a hardware configuration of an object detection device according to a first embodiment of the present invention. [Figure 27] FIG. 13 is a block diagram of an object detection device according to a sixth embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Next, an embodiment of the present invention will be described in detail with reference to the drawings. In the following description of the drawings, the same or similar parts are given the same or similar reference numerals. However, the drawings are schematic representations of the configuration of the embodiment of the present invention. Furthermore, the embodiment of the present invention described below is merely an example, and can be modified as appropriate within the scope of the same essence. [First embodiment] 1, an object detection device 100 according to a first embodiment of the present invention has a function of analyzing a monitor image of a road captured by a surveillance camera 110 and detecting an object. In this embodiment, there are two types of objects to be detected: a two-wheeled vehicle and a four-wheeled vehicle. That is, the object detection device 100 is a type of multi-class classifier, and detects two specific types of objects from an image.

[0016] The object detection device 100 has, as main functional units, a detection unit 101, a dictionary 102, a detection result DB (detection result database) 103, a reception unit 104, a generation unit 105, a teacher data memory 106, and a learning unit 107. A display device 120 and an input device 130 are connected to the object detection device 100. For example, as shown in FIG. 26, the object detection device 100 may be configured by an information processing device 230 and a storage medium for recording a program 240. The information processing device 230 has an arithmetic processing unit 210 such as one or more microprocessors, and a storage unit 220 such as a semiconductor memory or a hard disk. The storage unit 220 stores the dictionary 102, the detection result DB 103, the teacher data memory 106, and the like. The program 240 is loaded into memory from an external computer-readable recording medium when the object detection device 100 is started up, etc., and by controlling the operation of the calculation processing unit 210, it realizes functional means such as a detection unit 101, a reception unit 104, a generation unit 105, and a learning unit 107 on the calculation processing unit 210.

[0017] The display device 120 is a screen display device such as an LCD (Liquid Crystal Display) or a PDP (Plasma Display Panel), and has a function of displaying various information such as detection results on the screen in response to instructions from the object detection apparatus 100.

[0018] The input device 130 is an operation input device such as a keyboard or a mouse. The input device 130 detects an operator's operation as an input and outputs it to the object detection device 100. In this embodiment, a mouse is used as the input device 130. The mouse used is capable of performing two types of mouse gestures, a left click and a right click.

[0019] The detection unit 101 inputs images captured by the surveillance camera 110 one by one in chronological order, and detects objects from each input image using the dictionary 102. The detection unit 101 has a function of saving the detection results in the detection result DB 103. One image refers to one frame of an image captured by the surveillance camera 110. Consecutive frame numbers are assigned to the series of frames output from the surveillance camera 110 for identification purposes.

[0020] FIG. 2 is an explanatory diagram of the concept of detecting an object from an image. The detection unit 101 sets a search window 112 for object detection in the image 111, and extracts a feature amount of the image in the search window 112. Furthermore, the detection unit 101 calculates a likelihood indicating a similarity between the extracted feature amount and the dictionary 102, and detects the object by determining whether the image in the search window 112 is an object or not based on the calculated likelihood. When the detection unit 101 detects an object, the detection result having information of the partial region in which the object is detected and a class described later is stored in the detection result DB 103 in association with the identification information of the image 111. As information of the partial region of the object, position information on the image 111 of the search window 112 determined to be an object is used. Here, as the position information, for example, when the search window 112 is a rectangle, the coordinate values ​​of the upper left and lower right vertices are used. In addition, there are three classes related to objects: class 1 representing a class of two-wheeled vehicles, class 2 representing a class of four-wheeled vehicles, and class 0 representing a class that is neither of the above. Furthermore, the identification information of the image 111 is, for example, the frame number of the image.

[0021] The detection unit 101 changes the position and size of the search window 112 and repeats the same operations as described above to thoroughly search for objects of different positions and sizes that exist in the image 111. In the example shown in FIG. 2, three objects are detected from the image 111. The detection unit 101 detects a first object using a search window with a position and size indicated by reference numeral 113, detects a second object using a search window with a position and size indicated by reference numeral 114, and detects a third object using a search window with a position and size indicated by reference numeral 115. In FIG. 2, the horizontal direction of the image is the X-axis and the vertical direction is the Y-axis. After completing the detection process for one image 111, the detection unit 101 repeats the same process for the next image.

[0022] FIG. 3 is an example of the detection result DB 103. The detection result DB 103 in this example records the detection results corresponding to the frame number. The detection results are composed of information on partial regions of objects and classes. For example, the detection result DB 103 records pairs of information on partial regions of three objects and classes corresponding to frame number 001. One of them is a pair of a partial region whose upper left vertex coordinates are (x1, y1) and whose lower right vertex coordinates are (x2, y2) and class 2, another is a pair of a partial region whose upper left vertex coordinates are (x3, y3) and whose lower right vertex coordinates are (x4, y4) and class 1, and the remaining one is a pair of a partial region whose upper left vertex coordinates are (x5, y5) and whose lower right vertex coordinates are (x6, y6) and class 1.

[0023] 3 has a column for recording a correction class for each detection result. The correction class corresponds to the class input by the reception unit 104. When the detection unit 101 outputs the detection result to the detection result DB 103, all of the columns for the correction class are NULL (indicated by - in FIG. 3).

[0024] The reception unit 104 visualizes the detection results of the images stored in the detection result DB 103, displays them on the display device 120, and receives input of corrections from the operator.

[0025] 4 shows an example of a reception screen 121 that the reception unit 104 displays on the display device 120. In this example, the reception screen 121 displays a graph 122 at the bottom of the screen, a slider 123 at the top thereof, and an image window 124 at the top thereof.

[0026] Graph 122 is a graph showing the relationship between a plurality of images including image 111 and the number of partial regions in which an object is detected in each screen. When the horizontal direction of reception screen 121 is the X axis and the vertical direction is the Y axis, the X axis of graph 122 shows the frame number of the image, and the Y axis shows the number of partial regions in which an object is detected, that is, the number of object detections. That is, reception unit 104 sets the X axis of graph 122 to a column in which input images corresponding to the detection results stored in detection result DB 103 are arranged in ascending or descending order of frame numbers. However, the X axis may be other than the column in which input images are arranged in ascending or descending order of frame numbers. For example, the X axis may be a column in which input images are arranged by the number of partial regions of objects (that is, the number of object detections). In this case, reception unit 104 sorts the input images of detection results 103 stored in detection result DB 103 in ascending or descending order by the number of object detections, and sets the column of sorted input images to the X axis of graph 122. Furthermore, although graph 122 is a bar graph in FIG. 4, the type of graph 122 is not limited to a bar graph, and may be another type, such as a line graph.

[0027] The slider 123 is a GUI (Graphical User Interface) for selecting a position on the X-axis of the graph 122, i.e., an image. The slider 123 includes a slide bar 123a, and the position on the X-axis of the graph 122 is selected by operating the slide bar 123a left and right with the input device 130.

[0028] The image window 124 displays, frame by frame, one image 111 selected from the multiple images by operating the slider 123. A display for emphasizing a partial region of a detected object is added to the image 111 displayed in the image window 124. In the example of Fig. 4, a rectangular frame 125 indicating the periphery of the partial region of the object is displayed. However, the display for emphasizing the partial region of the object is not limited to the rectangular frame 125, and any display form may be used, such as a display form in which the brightness of the entire partial region is made higher or lower than other regions, or a display form in which the entire partial region is shaded.

[0029] Furthermore, the display highlighting the partial region of the detected object is displayed in a different display form depending on the class of the detected object. In Fig. 4, the partial region of an object of class 1 is displayed with a dashed frame 125, and the partial region of an object of class 2 is displayed with a solid frame 125. In addition to varying the line type of frame 125, the display color of frame 125 may also be varied. Also, a numerical value indicating the class may be displayed near the partial image.

[0030] Generally, a GUI slider displays a scale and a label as a guide for the operator's operation. Instead of the scale and the label, a reception screen 121 shown in Fig. 4 displays a graph 122 indicating the number of object detections for each image. By displaying such a graph 122, the operator can easily select one image from multiple images based on the number of object detections.

[0031] The reception unit 104 also has a function of receiving a selection of a partial region and an input of a class for the selected partial region by an operation of the input device 130 on the image 111 displayed on the display device 120. The reception unit 104 receives the selection of a partial region according to the position of the image 111 where the operation of the input device 130 is performed, and receives an input of a class according to the type of the operation. In this embodiment, the input device 130 is a mouse, and the types of operations are left click and right click. A left click means an operation of single clicking the left button of the mouse. A right click means an operation of single clicking the right button of the mouse. The reception unit 104 receives the selection of a partial region having within its region the position of the image 111 that is left clicked or right clicked. The reception unit 104 also receives an input of class 0 for the selected partial region when the left click is performed, and receives an input of a class incremented from the current class when the right click is performed. Here, the incremented class from the current class means class 2 when the current class is class 1, class 0 when the current class is class 2, and class 1 when the current class is class 0. The receiving unit 104 records the received class for the selected partial region in a column of corrected class in the detection result DB 103.

[0032] The generating unit 105 generates training data from the image of the partial region selected by the receiving unit 104 and the input class, and stores the training data in the training data memory 106. The generating unit 105 generates training data for each partial region in which any of class 0, class 1, and class 2 is recorded in the column for corrected class in the detection result DB 103 shown in Fig. 3. The training data is an input / output pair consisting of the image of the partial region or a feature amount extracted from the image of the partial region and the class recorded in the column for corrected class.

[0033] The learning unit 107 learns the dictionary 102 using the teacher data stored in the teacher data memory 106. The method of learning the dictionary 102 using the teacher data is widely known, so a description thereof will be omitted. The dictionary 102 is upgraded by learning using the teacher data. The detection unit 101 uses this upgraded dictionary 102 to detect objects again from each image input from the surveillance camera 110.

[0034] 5 is a flowchart showing the operation of this embodiment. Hereinafter, the operation of this embodiment will be described with reference to FIG.

[0035] The detection unit 101 of the object detection device 100 inputs images captured by the surveillance camera 110 one by one in chronological order (step S101). Next, the detection unit 101 detects an object from the input image using the dictionary 102 by the method described with reference to FIG. 2, and stores the detection result in the detection result DB 103 (step S102). If there is a next image captured by the surveillance camera 110 (step S103), the detection unit 101 returns to step S100 and repeats the same process as described above. As a result, the detection results of a plurality of images as shown in FIG. 3 are accumulated in the detection result DB 103. When the processing of the detection unit 101 for the series of images captured by the surveillance camera 110 is completed, the processing by the reception unit 104 is started.

[0036] The reception unit 104 displays a reception screen 121 as shown in Fig. 4 on the display device 120, and receives an input operation of the input device (step S104). The details of step S104 will be described later.

[0037] When the reception unit 104 finishes receiving the data, the generation unit 105 generates training data from the image of the partial region selected by the reception unit 104 and the input class (step S105). Next, the learning unit 107 learns the dictionary 102 using the training data stored in the training data memory 106 (step S106).

[0038] 6 is a flowchart showing the details of the operation of the reception unit 104 performed in step S104. The operation of the reception unit 104 will be described in detail below with reference to FIG.

[0039] The receiving unit 104 calculates the number of detected objects for each image from the detection result DB 103 (step S111). With reference to Fig. 3, three partial regions are detected from the image with frame number 001, so the receiving unit 104 sets the number of detected objects for frame number 001 to 3. In the same manner, the receiving unit 104 calculates the number of detected objects for images with other frame numbers.

[0040] Next, the reception unit 104 displays an initial reception screen 121 on the display device 120 (step S112). As shown in FIG. 4, the initial reception screen 121 displays a graph 122, a slider 123, and an image window 124. The graph 122 displays the number of detected objects for each frame number calculated in step S111. The slider 123 is a slide bar 123a placed at a predetermined position. The image window 124 displays an image of the object number on the X-axis of the graph 122 pointed to by the slide bar 123a. In addition, a partial area of ​​an object detected on the image displayed in the image window 124 is highlighted by a frame 125 of a line type according to the class of the partial area.

[0041] Next, the accepting unit 104 judges whether the slide bar 123a of the slider 123 has been operated (step S113), whether an image on the image window has been clicked (step S114), and whether an acceptance end condition has been established (step S115). The acceptance end condition may be, for example, that an acceptance end command has been input by the operator, or that no input operation has been performed for a certain period of time or more.

[0042] When the reception unit 104 detects a left click or right click on the image in the image window 124 (YES in step S114), it determines whether or not a partial area including the clicked position exists in the image (step S116). This determination is made by checking whether the coordinate values ​​of the clicked position are inside any partial area of ​​the image displayed in the image window 124. If there is no partial area including the clicked position (NO in step S116), the reception unit 104 ignores the click. On the other hand, if there is a partial area including the clicked position (YES in step S116), the reception unit 104 determines that the partial area including the clicked position has been selected, and determines whether the type of click is a left click or a right click to determine the input of a class (step S117).

[0043] If the click is a left click, the accepting unit 104 accepts input of class 0 for the selected partial region (step S118). Next, the accepting unit 104 updates the correction class corresponding to the selected partial region of the detection result in the detection result DB 103 to the accepted class 0 (step S119). Next, the accepting unit 104 makes the frame 125 displayed in the selected partial region of the image displayed in the image window 124 invisible (step S120).

[0044] Furthermore, in the case of right-clicking, the reception unit 104 receives an input of a class obtained by incrementing the current class of the selected partial region (step S121). The current class of the selected partial region is the class of the detection result when the corrected class corresponding to the selected partial region of the detection result in the detection result DB 103 is NULL, and is the class described in the corrected class when the corrected class is not NULL. Next, the reception unit 104 updates the corrected class corresponding to the selected partial region of the detection result in the detection result DB 103 to the incremented class (step S122). Next, the reception unit 104 updates the display of the frame 125 displayed in the selected partial region of the image displayed in the image window 124 (step S123). Specifically, if the incremented class is class 2, the reception unit 104 displays the frame 125 with a solid line. If the incremented class is class 0, the reception unit 104 hides the frame 125. Furthermore, if the class after the increment is class 1, the receiving unit 104 displays the frame 125 in a dashed line.

[0045] On the other hand, when the reception unit 104 detects that the slide bar 123a has been operated (step S113), it updates the image of the slide bar 123a on the reception screen 121 and the image window 124 (step 124). Specifically, the reception unit 104 moves the display position of the slide bar 123a in response to the input operation. The reception unit 104 also displays in the image window 124 an image of the object number on the X-axis of the graph 122 pointed to by the moved slide bar 123a.

[0046] Furthermore, the receiving unit 104 judges whether the partial region on the image displayed in the image window 124 has the same position and the same pre-input class as the partial region of the image in which the class was previously input. If it is judged that they have the same position and the same pre-input class, the receiving unit 104 hides the frame 125 that highlights the partial region. For example, the positions of the third partial images in the frame numbers 001, 002, and 003 in FIG. 3 are all rectangles with the upper left vertex and the lower right vertex at (x5, y5) and (x6, y6), respectively, and the pre-input class is also the same class 1. Therefore, for example, after the class of the third partial image in the image with frame number 001 is corrected from class 1 to class 0, when the images with frame numbers 002 and 003 are displayed in the image window 124, the frame 125 is not displayed on the third partial image in the images with frame numbers 002 and 003. However, class 1 is still recorded as the class of those detection results.

[0047] As described above, according to this embodiment, high-quality teacher data can be efficiently created because the receiving unit 104 displays the input image on the display device 120 with a display that highlights a partial region of the detected object, and receives the selection of a partial region and the input of a class for the selected partial region by a click, which is one operation of the input device 130.

[0048] According to this embodiment, one image can be easily selected from a plurality of images based on the number of detected objects. This is because the reception unit 104 displays a graph 122 showing the relationship between the input image and the number of partial regions in which objects are detected on the display device 120, and selects an input image for which the detection result is displayed on the display device 120 according to the position on the graph 122 received by the operation of the slider 123. In this embodiment, an image to be processed can be selected based on the number of detected objects. Therefore, depending on the level of skill and preference of the operator, images in which many objects are detected can be processed preferentially, or conversely, images in which the number of detected objects is low can be processed preferentially. Alternatively, the operator can easily process images in which the number of detected objects is intermediate between the above. Generally, the total work time is shortened by prioritizing processing images in which many objects are detected, but in the case of images in which there is a lot of overlap between partial regions, it is difficult for an operator who is not accustomed to the process to confirm the overlap.

[0049] In addition, when a surveillance camera 110 with a fixed field of view captures images of two-wheeled or four-wheeled vehicles traveling on a road at a constant interval, the stationary object commonly appears at the same position in a number of consecutive frame images. For this reason, for example, if a puddle on the road is mistakenly detected as a two-wheeled vehicle, the same erroneous recognition result appears at the same position in a number of consecutive frame images. It is sufficient to correct one of the same erroneous recognition results to generate training data, and it is rather annoying if the same erroneous recognition result that has already been corrected is highlighted across a number of frame images. In this embodiment, the reception unit 104 hides the frame 125 that highlights the same position as a partial area of ​​an image in which a class was previously input and the partial area having the same pre-input class. This can eliminate the annoyingness.

[0050] [Modification of the first embodiment] Next, various modifications of the configuration of the first embodiment will be described.

[0051] In the first embodiment, the receiving unit 104 receives an input of class 0 by a left click of the mouse gesture, and receives an input of an increment of the class by a right click. However, various combinations of gesture types that receive an input of class 0 and gesture types that receive an input of an increment of the class are possible, as exemplified in the list of FIG.

[0052] 7, the accepting unit 104 accepts an input of class 0 by a left click of the mouse gesture, and accepts an input of an increment of the class by a double click. A double click means an operation of single clicking the left mouse button twice in succession.

[0053] 7, the reception unit 104 receives an input of class 0 by a left click of the mouse gesture, and receives an input of an increment of the class by drag-and-drop. Drag-and-drop means an operation of moving (dragging) the mouse while pressing the left button of the mouse, and releasing (dropping) the left button at another location. When drag-and-drop is used, it is determined in step S114 of FIG. 6 whether drag-and-drop has been performed on the image window, and in step S116 it is determined, for example, whether there is a partial area that includes the start point of the drag and the drop point.

[0054] The numbers 1-1 and 1-2 use mouse gestures as in the first embodiment. On the other hand, the numbers 1-3 to 1-6 shown in FIG. 7 use touch gestures. A touch gesture means a touch operation performed on a touch panel using a part of the human body such as a fingertip. There are many types of touch gestures, such as tap (light tapping with a finger, etc.), double tap (tapping twice with a finger, etc.), flick (flicking or sweeping with a finger, etc.), swipe (tracing with a finger, etc.), pinch-in (pinching or narrowing with multiple fingers, etc.), and pinch-out (widening with multiple fingers, etc.). When using touch gestures, a touch panel is used as the input device 130. The use of a touch panel is also useful when the present invention is implemented in a mobile device such as a smartphone or tablet terminal.

[0055] In number 1-3 in FIG. 7, the reception unit 104 receives an input of class 0 by a flick, and receives an input of an increment of the class by a swipe. In number 1-4 in FIG. 7, the reception unit 104 receives an input of class 0 by a flick to the left, and receives an input of an increment of the class by a flick to the right. In number 1-5 in FIG. 7, the reception unit 104 receives an input of class 0 by a pinch-in, and receives an input of an increment of the class by a pinch-out. In number 1-6 in FIG. 7, the reception unit 104 receives an input of class 0 by a tap, and receives an input of an increment of the class by a double tap. These are examples, and any other combination of gestures may be used.

[0056] In the above first embodiment and numbers 1-1 to 1-6 in Fig. 7, the reception unit 104 receives an input of class 0 by a first type of gesture, and receives an input of an increment of the class by a second type of gesture. However, the reception unit 104 may receive an input of a decrement instead of an increment. That is, in the above first embodiment and numbers 1-1 to 1-6 in Fig. 7, the reception unit 104 may receive an input of class 0 by a first type of gesture, and receives an input of a decrement of the class by a second type of gesture.

[0057] In the first embodiment, the left click of the mouse gesture accepts input of class 0, and the right click accepts input of an increment of the class. However, classes and operations may be associated one-to-one, and inputs of class 0, class 1, and class 2 may each be accepted by one specific operation. As shown in the list of FIG. 8, various combinations of gesture types that accept inputs of each class are possible.

[0058] For example, in the case of number 1-11 shown in FIG. 8, the reception unit 104 receives an input of class 0 by a left click of the mouse gesture, receives an input of class 1 by a right click, and receives an input of class 2 by a double click. In addition, in the case of number 1-12 shown in FIG. 8, the reception unit 104 receives inputs of class 0 and class 1 by a mouse gesture similar to that of number 1-11, and receives an input of class 2 by drag and drop. The above is an example using mouse gestures. In contrast, in the cases of numbers 1-13 to 1-15 shown in FIG. 8, touch gestures are used. For example, in the case of number 1-13 shown in FIG. 8, the reception unit 104 receives an input of class 0 by a flick, receives an input of class 1 by a tap, and receives an input of class 2 by a double tap. In addition, in the case of number 1-14 shown in FIG. 8, the reception unit 104 receives an input of class 0 by a left flick, receives an input of class 1 by an upward flick, and receives an input of class 2 by a downward flick. In addition, in the case of number 1-15 shown in FIG. 8, the accepting unit 104 accepts an input of class 0 by tapping, an input of class 1 by pinching in, and an input of class 2 by pinching out.

[0059] In the above first embodiment, the number of classes was three (class 0, class 1, class 2), but the present invention can also be applied to a two-class object detection device with two classes (class 0, class 1), and a multi-class object detection device with four or more classes.

[0060] [Second embodiment] In the second embodiment of the present invention, an object detection device is described that allows selection of a partial region of an object that cannot be mechanically detected from an input image and input of a class for that partial region by operating an input device on the input image. This embodiment is different from the first embodiment in the function of the reception unit 104, but is otherwise basically the same as the first embodiment. Below, the function of the reception unit 104 suitable for the second embodiment will be described in detail with reference to FIG. 1, which is a block diagram of the first embodiment.

[0061] The reception unit 104 has a function of visualizing the image detection result DB 103, displaying it on the display device 120, and receiving input of corrections from the operator.

[0062] FIG. 9 shows an example of a reception screen 126 that the reception unit 104 displays on the display device 120. As shown in FIG. 4, reception screen 126 in this example differs only in that a rectangular frame indicating the periphery of a partial region is not displayed at the location of the two-wheeled vehicle in image 111 displayed in image window 124. The rectangular frame is not displayed because when detection unit 101 detected an object from image 111 using dictionary 102, it failed to detect the two-wheeled vehicle at that location. To improve such missed detections, the object detection device needs to create training data that is an input / output pair between the partial region of the two-wheeled vehicle in image 111 and its class, and learn dictionary 102.

[0063] The reception unit 104 has a function of receiving a selection of a partial region and an input of a class for the selected partial region by an operation of the input device 130 on the image 111 displayed on the display device 120. Here, in this embodiment, the selection of a partial region means both the selection of an existing partial region and the creation of a new partial region. The reception unit 104 receives the selection of a partial region based on the position on the image 111 where the operation of the input device 130 is performed, and receives the input of a class based on the type of the operation. In this embodiment, the input device 130 is a mouse, and the types of operations are left click, right click, drag and drop in the lower right direction, and drag and drop in the lower left direction.

[0064] Drag-and-drop in the lower-right direction means, as shown by the arrow in Fig. 10A, moving (dragging) the mouse from a certain location (start point) a on image 111 while pressing the left button, and then releasing (dropping) the left button at another location (end point) b located to the lower-right of start point a. Drag-and-drop in the lower-left direction means, as shown by the arrow in Fig. 10B, moving (dragging) the mouse from a certain location (start point) a on image 111 while pressing the left button, and then releasing (dropping) the left button at another location (end point) b located to the lower-left of start point a.

[0065] The reception unit 104 receives a selection of a partial region having within it the position of the image 111 that has been left-clicked or right-clicked. Furthermore, the reception unit 104 receives an input of class 0 for the selected partial region when the user left-clicks, and receives an input of a class obtained by incrementing the current class when the user right-clicks. Here, the class obtained by incrementing the current class means class 2 when the current class is class 1, class 0 when the current class is class 2, and class 1 when the current class is class 0. The reception unit 104 records the received class for the selected partial region in the column of corrected class of the detection result 103. The above is the same as when the user left-clicks or right-clicks in the first embodiment.

[0066] 10A, and accepts input of class 1 for the partial region. Furthermore, when the reception unit 104 detects a drag-and-drop in the lower-left direction, it calculates a rectangular region with the start point a as the upper-left vertex and the end point b as the lower-right vertex as shown by the dashed line in Fig. 10A, and accepts input of class 2 for the partial region. Furthermore, when the reception unit 104 detects a drag-and-drop in the lower-left direction, it calculates a rectangular region with the start point a as the upper-right vertex and the end point b as the lower-left vertex as shown by the dashed line in Fig. 10B, and accepts input of class 2 for the partial region. Furthermore, the reception unit 104 records information on the newly calculated partial region and its class in the detection result DB 103.

[0067] Fig. 11 is a flowchart showing the details of the operation of the reception unit 104. In Fig. 11, steps S211 to S224 are the same as steps S111 to S124 in Fig. 6. Hereinafter, the operation of the reception unit 104 will be described in detail with reference to Fig. 11, focusing on the differences from Fig. 6.

[0068] When the reception unit 104 detects a drag-and-drop on the image of the image window 124 (YES in step S225), it determines the direction of the drag-and-drop (step S226). If the drag-and-drop is in the lower right direction, the reception unit 104 calculates a rectangular partial area with the upper left vertex as the starting point and the lower right vertex as the ending point, as described with reference to FIG. 10A, and receives the input of class 1 (step S227). Next, the reception unit 104 records the pair of the calculated partial area and class 1 as one detection result in the detection result DB 103 (step S228). Next, the reception unit 104 updates the image window 124 of the reception screen 126 (step S229). Then, the reception unit 104 returns to the process of step S213.

[0069] If the drag-and-drop is in the lower left direction (step S226), the accepting unit 104 calculates a rectangular partial area with the start point being the upper right vertex and the end point being the lower left vertex as described with reference to FIG. 10B, and accepts the input of class 2 (step S230). Next, the accepting unit 104 records the pair of the calculated partial area and class 2 as one detection result in the detection result DB 103 (step S231). Next, the accepting unit 104 updates the image window 124 of the accepting screen 126 (step S232). Then, the accepting unit 104 returns to the process of step S213.

[0070] 12A and 12B are explanatory diagrams of step S228 by the receiving unit 104. FIG. 12A shows the detection result DB 103 generated by the detection unit 101, and FIG. 12B shows the detection result DB 103 after the receiving unit 104 adds a record of a pair of a newly calculated partial region and class 1. The newly added partial region is a rectangle with an upper left vertex (x3, y3) and a lower right vertex (x4, y4), and its class is class 1. In this way, the class of the added partial region is recorded in the column of the corrected class. In this way, the generating unit 105 generates teacher data from the added partial region and class. In the case of step S231 by the receiving unit 104, it is different in that class 2 is recorded in the corrected class.

[0071] 13A and 13B are explanatory diagrams of step S229 by the reception unit 104. Fig. 13A shows the image window 124 before updating, and Fig. 13B shows the image window 124 after updating. A rectangular frame 125 drawn with dashed lines to highlight the newly added partial region is drawn in the updated image window 124. In the case of step S232 by the reception unit 104, a rectangular frame drawn with solid lines to highlight the newly added partial region is drawn in the updated image window 124, which is different.

[0072] As described above, according to this embodiment, good quality teacher data can be efficiently created. This is because the receiving unit 104 displays the input image on the display device 120 with a display that highlights the partial region of the detected object, and receives the selection of the partial region and the input of the class for the selected partial region by a click, which is one operation of the input device 130. Also, the receiving unit 104 detects one operation of dragging and dropping in the lower right or lower left direction on the image, and receives the calculation of a new partial region and the input of class 1 or class 2.

[0073] [Modification of the second embodiment] Next, various modifications of the configuration of the second embodiment will be described.

[0074] In the above second embodiment, the receiving unit 104 receives input of class 0 by left-clicking the mouse gesture, receives input of an increment of the class by right-clicking, calculates a partial area of ​​class 1 by dragging and dropping in the lower right direction, and calculates a partial area of ​​class 2 by dragging and dropping in the lower left direction. However, various combinations of gesture types for receiving input of class 0, gesture types for receiving input of an increment of the class, gesture types for receiving calculation of a partial area of ​​class 1, and gesture types for receiving calculation of a partial area of ​​class 2 are possible, as exemplified in the list of FIG.

[0075] 14, the receiving unit 104 receives an input of class 0 by a left click of the mouse gesture, and receives an input of an increment of the class by a double click. Furthermore, the receiving unit 104 receives a calculation of a partial area of ​​class 1 by dragging and dropping in the lower right direction, and receives a calculation of a partial area of ​​class 2 by dragging and dropping in the lower left direction.

[0076] 14, the receiving unit 104 receives input of class 0 by left-clicking the mouse gesture, receives input of class increment by right-clicking, receives calculation of a partial area of ​​class 1 by dragging and dropping in the upper left direction, and receives calculation of a partial area of ​​class 2 by dragging and dropping in the upper right direction. Here, the upper left dragging and dropping is a dragging and dropping in the opposite direction to the arrow in FIG. 10A, and means an operation of moving (dragging) the mouse from a certain place (starting point) b on the image 111 while pressing the left button of the mouse, and releasing (dropping) the left button at another place (ending point) a located in the upper left direction of the starting point b. The upper right dragging and dropping is a dragging and dropping in the opposite direction to the arrow in FIG. 10B, and means an operation of moving (dragging) the mouse from a certain place (starting point) b on the image 111 while pressing the left button of the mouse, and releasing (dropping) the left button at another place (ending point) c located in the upper right direction of the starting point b.

[0077] 14, the operation for accepting the calculation of the partial area of ​​class 1 and class 2 is different from that for number 2-2. In number 2-3, the accepting unit 104 accepts the calculation of the partial area of ​​class 1 by a left double click, and accepts the calculation of the partial area of ​​class 2 by a right double click. Here, a left double click means clicking the left button of the mouse twice in succession. Also, a right double click means clicking the right button of the mouse twice in succession.

[0078] FIG. 15 is an explanatory diagram of a method in which the reception unit 104 calculates a partial area of ​​class 2 from the position of a right double click performed on the image 111. When the reception unit 104 detects that a right double click has been performed at a point c on the image 111, the reception unit 104 assumes that the center of gravity of a typical class 2 object (a four-wheeled vehicle in this embodiment) 131 is present at a position that coincides with point c, and calculates a circumscribing rectangle 132 of the object as a partial area. When an object is captured by a surveillance camera 110 with a fixed imaging field of view, there is a correlation between each point of the object and each point on the image, so that the circumscribing rectangle of an object of a typical size having a center of gravity at a certain point on the image is almost uniquely determined. Furthermore, the circumscribing rectangle of a typical class 1 object (a two-wheeled vehicle in this embodiment) having a center of gravity at a certain point on the image is also almost uniquely determined. In this way, the reception unit 104 calculates a partial area of ​​class 1 from the position of a left double click performed on the image 111 by a similar method.

[0079] In the example of FIG. 15, the operator is configured to right-double-click or left-double-click the center of gravity of the object via the input device 130, but the center of gravity is not limited to the center of gravity, and any predetermined position may be used. For example, the center of the front wheel of a two-wheeled vehicle, or the middle between the front and rear wheels of a four-wheeled vehicle may be right-double-clicked or left-double-clicked. Furthermore, in the case of an object detection device that detects people, the operator may be configured to left-double-click the person's head via the input device 130. In this case, the receiving unit 104 estimates a person area from the head position, and calculates a circumscribing rectangle of the estimated person area as a partial area.

[0080] The above numbers 2-1 to 2-3 use mouse gestures as in the second embodiment, whereas numbers 2-4 to 2-7 shown in Fig. 14 use touch gestures as follows.

[0081] In number 2-4 of Figure 14, the receiving unit 104 accepts input of class 0 by a flick, accepts input of a class increment by a tap, accepts calculation of a partial area of ​​class 1 by a swipe in the lower right direction, and accepts calculation of a partial area of ​​class 2 by a swipe in the lower left direction.

[0082] A swipe in the lower right direction means an operation of tracing with a fingertip from a certain place (starting point) a on the image 111 to another place (ending point) b located in the lower right direction of the starting point a, as shown by an arrow in FIG. 16A. A swipe in the lower left direction means an operation of tracing with a fingertip from a certain place (starting point) a on the image 111 to another place (ending point) b located in the lower left direction of the starting point a, as shown by an arrow in FIG. 16B. When detecting a swipe in the lower right direction, the reception unit 104 calculates a rectangular area with the starting point a as the upper left vertex and the ending point b as the lower right vertex as a partial area, as shown by a dashed line in FIG. 16A, and accepts an input of class 1 for the partial area. When detecting a swipe in the lower left direction, the reception unit 104 calculates a rectangular area with the starting point a as the upper right vertex and the ending point b as the lower left vertex as a partial area, as shown by a dashed line in FIG. 16B, and accepts an input of class 2 for the partial area.

[0083] In number 2-5 of Figure 14, the reception unit 104 accepts input of class 0 with a left flick, accepts input of a class increment with a right flick, accepts calculation of a partial area of ​​class 1 with a downward swipe, and accepts calculation of a partial area of ​​class 2 with an upward swipe.

[0084] A downward swipe means an operation of tracing with a fingertip from a certain place (start point) a on the image 111 to another place (end point) b located below the start point a, as shown by an arrow in FIG. 17A. An upward swipe means an operation of tracing with a fingertip from a certain place (start point) b on the image 111 to another place (end point) a located above the start point b, as shown by an arrow in FIG. 17B. When the reception unit 104 detects a downward swipe, it calculates a rectangle as shown by a dashed line in FIG. 17A as a partial area of ​​class 1. The rectangle is line-symmetrical with respect to the swipe line, and the vertical length is equal to the swipe length. The horizontal length is calculated by multiplying the swipe length by a predetermined ratio. For example, the ratio of the horizontal length to the vertical length of a circumscribing rectangle of a class 1 object of a typical shape (a two-wheeled vehicle in this embodiment) can be used as the predetermined ratio. Furthermore, when the reception unit 104 detects an upward swipe, it calculates a rectangle as shown by the dashed line in FIG. 17B as a partial area of ​​class 2. The rectangle is symmetrical about the swipe line, and the vertical length is equal to the length of the swipe. The horizontal length is calculated by multiplying the length of the swipe by a predetermined ratio. As the predetermined ratio, the ratio of the horizontal length to the vertical length of the circumscribing rectangle of a class 2 object of a typical shape (a four-wheeled automobile in this embodiment) can be used.

[0085] In number 2-6 of Figure 14, the receiving unit 104 accepts input of class 0 by pinching in, accepts input of a class increment by pinching out, accepts calculation of a partial area of ​​class 1 by swiping in the lower right direction, and accepts calculation of a partial area of ​​class 2 by swiping in the lower left direction.

[0086] 14, the accepting unit 104 accepts an input of class 0 by tapping, an input of an increment of the class by double tapping, a calculation of a partial area of ​​class 1 by swiping in the lower right direction, and a calculation of a partial area of ​​class 2 by swiping in the lower left direction. These are merely examples, and any other combination of gestures may be used.

[0087] In the above second embodiment and in numbers 2-1 to 2-7 in Fig. 14, the reception unit 104 receives an input of class 0 by a first type of gesture, and receives an input of an increment of the class by a second type of gesture. However, the reception unit 104 may receive an input of a decrement instead of an increment. That is, in the above second embodiment and in numbers 2-1 to 2-7 in Fig. 14, the reception unit 104 may receive an input of a decrement of the class by a second type of gesture.

[0088] In the above second embodiment, the receiving unit 104 receives input of class 0 by left-clicking the mouse gesture, and receives input of incrementing the class by right-clicking. However, classes and operations may be associated one-to-one with each other, and inputs of class 0, class 1, and class 2 may be received by one specific operation. Various combinations of gesture types for receiving inputs of each class and gesture types for receiving calculations of partial areas of each class are possible, as exemplified in the list of FIG. 18.

[0089] For example, in number 2-11 shown in FIG. 18, the receiving unit 104 accepts input of class 0 by a left-click mouse gesture, accepts input of class 1 by a right-click, accepts input of class 2 by a double-click, accepts calculation of a partial area of ​​class 1 by drag-and-drop in the lower right direction, and accepts calculation of a partial area of ​​class 2 by drag-and-drop in the lower left direction.

[0090] 18 differs from number 2-11 in the gesture types for accepting calculation of partial areas of class 1 and class 2. In number 2-12, the accepting unit 104 accepts calculation of a partial area of ​​class 1 by drag-and-drop in the upper left direction, and accepts calculation of a partial area of ​​class 2 by drag-and-drop in the upper right direction.

[0091] Also, in number 2-13 shown in FIG. 18, the reception unit 104 accepts input of class 0 by a touch gesture flick, accepts input of class 1 by a tap, accepts input of class 2 by a double tap, accepts calculation of a partial area of ​​class 1 by a swipe in the lower right direction, and accepts calculation of a partial area of ​​class 2 by a swipe in the lower left direction.

[0092] 18 differs from number 2-13 in the gesture types that accept inputs of classes 0, 1, and 2. In number 2-14, the accepting unit 104 accepts an input of class 0 by a leftward flick, an input of class 1 by an upward flick, and an input of class 2 by a downward flick.

[0093] Also, in number 2-15 shown in FIG. 18, the receiving unit 104 accepts input of class 0 by tapping, accepts input of class 1 by pinching in, accepts input of class 2 by pinching out, accepts calculation of a partial area of ​​class 1 by swiping downward, and accepts calculation of a partial area of ​​class 2 by swiping upward.

[0094] In the above second embodiment, the number of classes was three (class 0, class 1, class 2), but the present invention can also be applied to a two-class object detection device with two classes (class 0, class 1), and a multi-class object detection device with four or more classes.

[0095] [Third embodiment] The third embodiment of the present invention describes an object detection device that accepts inputs of different classes even for the same operation depending on whether or not there is an existing partial area that has the position of the image where the operation was performed within the area. This embodiment differs from the second embodiment in the function of the reception unit 104, but is otherwise basically the same as the second embodiment. Below, the function of the reception unit 104 suitable for the third embodiment will be described in detail with reference to FIG. 1, which is a block diagram of the first embodiment.

[0096] The reception unit 104 has a function of visualizing the image detection result DB 103, displaying it on the display device 120, and receiving input of corrections from the operator.

[0097] The reception unit 104 has a function of receiving a selection of a partial region and an input of a class for the selected partial region by an operation of the input device 130 on the image 111 displayed on the display device 120. Here, in this embodiment, the selection of a partial region means both the selection of an existing partial region and the selection of a new partial region. The reception unit 104 receives the selection of a partial region based on the position of the image 111 where the operation of the input device 130 is performed, and receives an input of a class based on the type of the operation. Also, in this embodiment, the reception unit 104 receives input of a different class even for the same operation depending on whether or not there is an existing partial region that has the position of the image where the operation is performed within the region. In this embodiment, the input device 130 is a mouse, and the types of one operation are a left click and a right click.

[0098] The reception unit 104 receives a selection of an existing partial region having within it the position of the image 111 that has been left-clicked or right-clicked. Furthermore, the reception unit 104 receives an input of class 0 for the selected partial region when the user left-clicks, and receives an input of a class obtained by incrementing the current class when the user right-clicks. Here, the class obtained by incrementing the current class means class 2 when the current class is class 1, class 0 when the current class is class 2, and class 1 when the current class is class 0. The reception unit 104 records the received class for the selected partial region in the column of modified class in the detection result DB 103. The above is the same as when the user left-clicks or right-clicks in the first embodiment.

[0099] Furthermore, if there is no existing partial area that has the position of the left-clicked or right-clicked image 111 within its area, the receiving unit 104 calculates a new rectangular partial area based on the clicked position, and receives an input of class 1 for the partial area if it is a left click, or class 2 for the partial area if it is a right click. The method of calculating a new partial area based on the clicked position can be the same as the method of calculating a partial area of ​​an object based on the double-clicked position described with reference to Fig. 15.

[0100] FIG. 19 is a flowchart showing the details of the operation of the reception unit 104. In FIG. 19, steps S211 to S224 and S226 to S232 are almost the same as steps S211 to S224 and S226 to S232 in FIG. 11. FIG. 19 differs from FIG. 11 in that if it is determined in step S216 that there is no existing partial area that includes the clicked position, the process proceeds to step S226. Note that step S226 in FIG. 11 determines the direction (type) of drag and drop, but step S226 in FIG. 19 determines the type of click. Hereinafter, the operation of the reception unit 104 will be described in detail with reference to FIG. 19, focusing on the differences from FIG. 11.

[0101] The receiving unit 104 detects a left click or a right click on the image in the image window 124 (YES in step S214). At this time, if the receiving unit 104 determines that a partial area including the click position does not exist in the image (NO in step S216), it determines whether the type of click is a left click or a right click in order to determine the input of a class (step S226).

[0102] If the click is a left click, the accepting unit 104 calculates a new rectangular partial area and accepts the input of class 1 (step S227). Next, the accepting unit 104 records the pair of the calculated partial area and class 1 as one detection result in the detection result DB 103 (step S228). Next, the accepting unit 104 updates the image window 124 of the acceptance screen 126 (step S229). Then, the accepting unit 104 returns to the process of step S213.

[0103] Furthermore, if the click is a right click (step S226), the accepting unit 104 calculates a new rectangular partial area and accepts input of class 2 (step S230). Next, the accepting unit 104 records the pair of the calculated partial area and class 2 as one detection result in the detection result DB 103 (step S231). Next, the accepting unit 104 updates the image window 124 of the accepting screen 126 (step S232). Then, the accepting unit 104 returns to the process of step S213.

[0104] As described above, according to this embodiment, good quality teacher data can be efficiently created. This is because the receiving unit 104 displays the input image on the display device 120 with a display that highlights the partial region of the detected object, and receives the selection of a partial region and the input of a class for the selected partial region by a single operation of the input device 130, that is, a click. Also, the receiving unit 104 detects a single operation of a left click or right click at a position that does not overlap with an existing partial region, and receives the calculation of a new partial region and the input of class 1 or class 2.

[0105] [Modification of the third embodiment] In the above third embodiment, two types of gestures, left click and right click, are used, but other types of mouse gesture combinations and touch gesture combinations can be used.

[0106] [Fourth embodiment] In the fourth embodiment of the present invention, an object detection device is described that allows input of a correct class for an erroneously recognized partial region and selection of a correct partial region of an object by operating an input device on an input image. This embodiment is different from the second embodiment in the function of the reception unit 104, but is otherwise basically the same as the second embodiment. Below, the function of the reception unit 104 suitable for the fourth embodiment will be described in detail with reference to FIG. 1, which is a block diagram of the first embodiment.

[0107] The reception unit 104 has a function of visualizing the image detection result DB 103, displaying it on the display device 120, and receiving input of corrections from the operator.

[0108] FIG. 20 shows an example of a reception screen 127 that the reception unit 104 displays on the display device 120. The reception screen 127 in this example differs from the reception screen 121 shown in FIG. 4 only in the position of a rectangular frame indicating the periphery of the partial region of the four-wheeled vehicle in the image 111 displayed in the image window 124. The reason for the difference in the position of the rectangular frame is that when the detection unit 101 detects an object from the image 111 using the dictionary 102, it erroneously detects a puddle on the road as a part of the partial region of the object. In this example, the partial region is erroneously detected, but the class is correctly determined. However, there are also cases where both the partial region and the class are erroneously detected. In order to improve such erroneous detection, it is necessary to create teacher data that is an input / output pair between the erroneously detected partial region in the image 111 and its class 0, and teacher data that is an input / output pair between the correct partial region of the four-wheeled vehicle in the image 111 and its class 2, and to learn the dictionary 102.

[0109] The reception unit 104 has a function of receiving a selection of a partial region and an input of a class for the selected partial region by an operation of the input device 130 on the image 111 displayed on the display device 120. Here, in this embodiment, the selection of a partial region means both the selection of an existing partial region and the selection of a new partial region. The reception unit 104 receives the selection of a partial region based on the position of the image 111 where the operation of the input device 130 is performed, and receives an input of a class based on the type of the operation. Also, in this embodiment, when a selection operation of a new partial region that partially overlaps with an existing partial region is performed, the reception unit 104 receives an input of class 0 for the existing partial region and receives an input of a class according to the type of operation for the new partial region. In this embodiment, the input device 130 is a mouse, and the types of one operation are a left click, a right click, a drag and drop in the lower right direction, and a drag and drop in the lower left direction.

[0110] The reception unit 104 receives a selection of a partial region having within it the position of the image 111 that has been left-clicked or right-clicked. Furthermore, the reception unit 104 receives an input of class 0 for the selected partial region when the user left-clicks, and receives an input of a class obtained by incrementing the current class when the user right-clicks. Here, the class obtained by incrementing the current class means class 2 when the current class is class 1, class 0 when the current class is class 2, and class 1 when the current class is class 0. The reception unit 104 records the received class for the selected partial region in the column of corrected class in the detection result DB 103. The above is the same as when the user left-clicks or right-clicks in the first embodiment.

[0111] Furthermore, when the reception unit 104 detects a drag-and-drop in the lower right direction, it calculates a rectangular region with the start point a as the upper left vertex and the end point b as the lower right vertex as a partial region, as shown by the dashed line in FIG. 10A, and accepts an input of class 1 for the partial region. Furthermore, when the reception unit 104 detects a drag-and-drop in the lower left direction, it calculates a rectangular region with the start point a as the upper right vertex and the end point b as the lower left vertex as a partial region, as shown by the dashed line in FIG. 10B, and accepts an input of class 2 for the partial region. Furthermore, the reception unit 104 records the newly calculated partial region and information on its class in the detection result DB 103 as one detection result. The above is the same as when drag-and-drop is performed in the lower right direction and the lower left direction in the second embodiment.

[0112] Furthermore, the receiving unit 104 has a function of receiving an input of class 0 for an existing partial region that partially overlaps with a newly calculated partial region. For example, as shown in Fig. 21, when a drag-and-drop is performed in the lower left direction (or lower right direction) on an image 111 in which a frame 125 highlighting a partial region of class 2 is displayed, the receiving unit 104 calculates a partial region 125a that partially overlaps with the frame 125, and receives an input of class 0 for the existing partial region related to the frame 125.

[0113] Fig. 22 is a flowchart showing the details of the operation of the reception unit 104. In Fig. 22, steps S311 to S327 and S330 are the same as steps S211 to S327 and S230 in Fig. 11. Hereinafter, the operation of the reception unit 104 will be described in detail with reference to Fig. 22, focusing on the differences from Fig. 11.

[0114] When the reception unit 104 receives an input of class 1 for the newly calculated partial region (step S327), it judges whether or not an existing partial region partially overlapping the new partial region exists on the image 111 (step S341). This judgment is performed by investigating whether or not a detection result of an existing partial region partially overlapping the new partial region exists in the detection result DB 103 related to the image 111. If such an existing partial region exists, the reception unit 104 receives an input of class 0 for the partial region (step S342) and proceeds to step S343. If such an existing partial region does not exist, the reception unit 104 skips the process of step S342 and proceeds to step S343. In step S343, the reception unit 104 updates the detection result DB 103 according to the reception results of steps S327 and S342. That is, the reception unit 104 records a pair of the newly calculated partial region and class 1 as one detection result in the detection result DB 103 based on the reception result of step S327. Furthermore, the receiving unit 104 records class 0 as the correction class of the existing partial region based on the reception result of step S342. Next, the receiving unit 104 updates the image 111 on the image window 124 in accordance with the update of the detection result DB 103 in step S343 (step S344). Then, the receiving unit 104 returns to the process of step S313.

[0115] Furthermore, when the reception unit 104 receives an input of class 2 for the newly calculated partial region (step S330), it determines whether or not an existing partial region that partially overlaps the new partial region exists on the image 111 (step S345). If such an existing partial region exists, the reception unit 104 receives an input of class 0 for the partial region (step S346) and proceeds to step S347. If such an existing partial region does not exist, the reception unit 104 skips the process of step S346 and proceeds to step S347. In step S347, the reception unit 104 updates the detection result DB 103 according to the reception results of steps S330 and S346. That is, the reception unit 104 records a pair of the newly calculated partial region and class 2 as one detection result in the detection result DB 103 based on the reception result of step S330. Furthermore, the reception unit 104 records class 0 in the correction class of the existing partial region based on the reception result of step S346. Next, the reception unit 104 updates the image 111 on the image window 124 in accordance with the update of the detection result DB 103 in step S347 (step S348). Then, the reception unit 104 returns to the process of step S313.

[0116] 23A and 23B are explanatory diagrams of step S347 by the receiving unit 104. FIG. 23A shows the detection result DB 103 generated by the detection unit 101, and FIG. 23B shows the detection result DB 103 after it has been updated by the receiving unit 104. In FIG. 23A and FIG. 23B, a rectangular partial area with an upper left vertex (x7, y7) and a lower right vertex (x8, y8) is newly added as class 2. Also, the class of an existing rectangular partial area with an upper left vertex (x1, y1) and a lower right vertex (x2, y2) is corrected from class 2 to class 0. In the case of step S343 by the receiving unit 104, the difference is that the corrected class of the newly added partial area becomes class 1.

[0117] 24A and 24B are explanatory diagrams of step S348 by the reception unit 104. Fig. 24A shows the image window 124 before updating, and Fig. 24B shows the image window 124 after updating. In the updated image window 124, a rectangular frame 125 drawn with solid lines to highlight the newly added partial region is drawn, and a frame that was displayed in the image window 124 before updating and that partially overlaps with the rectangular frame is not displayed. In the case of step S344 by the reception unit 104, a difference is that in the updated image window 124, a rectangular frame that highlights the newly added partial region is drawn with dashed lines.

[0118] As described above, according to the present embodiment, good quality teacher data can be efficiently created. This is because the receiving unit 104 displays the input image on the display device 120 with a display that highlights the partial region of the detected object, and receives the selection of the partial region and the input of the class for the selected partial region by a click, which is one operation of the input device 130. Also, the receiving unit 104 detects one operation of dragging and dropping in the lower right or lower left direction on the image, and receives the calculation of a new partial region and the input of class 1 or class 2. Furthermore, the receiving unit 104 receives the input of class 0 for an existing partial region that partially overlaps with the newly calculated partial region.

[0119] [Modification of the fourth embodiment] The configuration of the fourth embodiment can be modified in the same manner as the example described in the modified example of the second embodiment.

[0120] [Fifth embodiment] A fifth embodiment of the present invention will be described. With reference to Fig. 25, an object detection device 1 according to this embodiment has a function of detecting an object from an input image 2. The object detection device 1 has a dictionary 10, and also has, as main functional units, a detection unit 11, a reception unit 12, a generation unit 13, and a learning unit 14. Furthermore, the object detection device 1 is connected to a display device 3 and an input device 4.

[0121] The detection unit 11 has a function of detecting an object from the input image 2 using the dictionary 10. The reception unit 12 has a function of displaying the input image 2 on the display device 3 with a display emphasizing a partial region of the object detected by the detection unit 11, and has a function of receiving, with a single operation of the input device 4, the selection of a partial region and an input of a class for the selected partial region. The generation unit 13 has a function of generating teacher data from the image of the selected partial region and the input class. The learning unit 14 has a function of learning the dictionary 10 with the teacher data generated by the generation unit 13, and upgrading the dictionary 10.

[0122] Next, the operation of the object detection device 1 according to this embodiment will be described. The detection unit 11 of the object detection device 1 detects an object from the input image 2 using the dictionary 10, and notifies the reception unit 12 of the detection result. The reception unit 12 displays the input image 2 on the display device 3 with a display that highlights a partial region of the object detected by the detection unit 11. The reception unit 12 then receives a selection of a partial region and an input of a class for the selected partial region through a single operation of the input device 4, and notifies the generation unit 13 of the result. The generation unit 13 generates teacher data from the image of the selected partial region and the input class, and notifies the learning unit 14 of the result. The learning unit 14 learns the dictionary 10 using the teacher data generated by the generation unit 13, and upgrades the dictionary 10.

[0123] As described above, according to this embodiment, high-quality teacher data can be efficiently created. This is because the receiving unit 12 displays the input image 2 on the display device 3 with a display that highlights the partial region of the detected object. Furthermore, the receiving unit 12 receives the selection of a partial region and the input of a class for the selected partial region by a click, which is one operation of the input device 4.

[0124] This embodiment is based on the above-mentioned configuration, but various additions and modifications can be made as described below.

[0125] The receiving unit 12 may be configured to receive a selection of a partial region based on the position of the input image 2 where one operation has been performed.

[0126] The receiving unit 12 may also be configured to receive a selection of a partial area having within its area the position of the input image 2 on which the one operation was performed, and if there is no partial area having within its area the position of the input image 2 on which the one operation was performed, calculate a partial area of ​​the object based on the position of the input image 2 on which the one operation was performed.

[0127] The receiving unit 12 may be configured to receive an input of a class according to the type of one operation.

[0128] The receiving unit 12 may also be configured to receive input of a first class in advance if the type of one operation is a first type, and to receive input of a class that is an increment or decrement of the current class of the partial region if the type of one operation is a second type.

[0129] Furthermore, the receiving unit 12 may be configured to receive inputs of different classes depending on whether or not there is a partial area that includes the position of the input image 2 where one operation has been performed.

[0130] The receiving unit 12 may also be configured to accept input of the first class if a partial area exists within the area that includes the position of the input image 2 where one operation was performed, and to accept input of the second class if such an area does not exist.

[0131] In addition, when there is a partial area having within it the position of the input image 2 where one operation was performed, the receiving unit 12 may be configured to receive input of the first class for the existing partial area, calculate a partial area of ​​the object based on the position of the input image where one operation was performed, and receive input of the second class for this calculated partial area.

[0132] The reception unit 12 may be configured to determine the second class based on the type of one operation.

[0133] Furthermore, one operation of the input device 4 may be configured to correspond to one mouse gesture, or one operation of the input device 4 may be configured to correspond to one touch gesture.

[0134] In addition, the receiving unit 12 may be configured to hide the display highlighting the partial region of the detected object when the partial region of the detected object has the same position and the same pre-input class as a partial region in which a class was previously input. [Sixth embodiment] An object detection device 300 according to the sixth embodiment of the present invention will be described with reference to Fig. 27. The object detection device 300 includes a detection unit 301, a reception unit 302, a generation unit 303, and a learning unit 304.

[0135] The detection unit 301 detects an object from an input image using a dictionary.

[0136] The receiving unit 302 displays the input image on the display device with a display that highlights a partial region of the detected object, and receives selection of a partial region and input of a class for the selected partial region through a single operation of the input device.

[0137] The generation unit 303 generates training data from the image of the selected partial region and the input class.

[0138] The learning unit 304 learns the dictionary using the training data.

[0139] According to this embodiment, it is possible to efficiently create high-quality teacher data because the receiving unit 302 displays the input image on the display device with a display that highlights a partial region of a detected object, and receives the selection of a partial region and the input of a class for the selected partial region with a single operation of the input device.

[0140] The present invention has been described above by taking the above-mentioned embodiment as an exemplary example. However, the present invention is not limited to the above-mentioned embodiment. That is, the present invention can be applied in various aspects that can be understood by a person skilled in the art within the scope of the present invention.

[0141] This application claims priority based on Japanese Patent Application No. 2015-055926, filed March 19, 2015, the disclosure of which is incorporated herein in its entirety. [Industrial Applicability]

[0142] INDUSTRIAL APPLICABILITY The present invention can be used in an object detection device that detects objects such as people and vehicles using a dictionary from a road image captured by a surveillance camera. [Explanation of symbols]

[0143] 1. Object detection device 2... Input image 3. Display device 4. Input device 10. Dictionary 11…Detection section 12. Reception 13...Generation section 14…Learning Department 100...Object detection device 101…Detection unit 102...Dictionary 103…Detection result DB 104…Reception 105...Generation section 106…Teacher data memory 107…Learning Department 111...Image 112, 113, 114, 115...Search window 120...Display device 121…Reception screen 122...Graph 123...Slider 123a…Slide bar 124...Image window 125...Rectangular frame 125a...partial area 126, 127...Reception screen 130…Input device 131…Object 132...Circumscribing rectangle

Claims

1. A detection means for detecting an object from an input image; a control means for controlling to highlight a first region corresponding to the object and a second region corresponding to a position selected by a user in the input image; Equipped with the first region is associated with a first class; the second region is associated with a second class different from the first class; Object detection device.

2. The control means controls the highlighting process so that the first region and the second region are highlighted in different ways. The object detection device according to claim 1 .

3. A generating unit for generating teacher data is further provided, the control means accepts an input of the second class for the second area; The generating means generates the teacher data based on at least the second region and the second class. The object detection device according to claim 1 .

4. An object detection device as described in any one of claims 1 to 3, wherein the second class indicates a different type of object from the first class.

5. Detect objects from the input image, Controlling to highlight a first region corresponding to the object and a second region corresponding to a position selected by a user in the input image; the first region is associated with a first class; the second region is associated with a second class different from the first class; Object detection methods.

6. Controlling the first region and the second region so as to be highlighted differently; The object detection method according to claim 5.

7. Accepting an input of the second class for the second area; generating training data based on at least the second region and the second class; The object detection method according to claim 5 or 6.

8. The object detection method of claim 5, wherein the second class indicates a different type of object from the first class.

9. The process of detecting objects from the input image. highlighting a first region corresponding to the object and a second region corresponding to a position selected by a user in the input image; on the computer, the first region is associated with a first class; the second region is associated with a second class different from the first class; program.

10. In the process of highlighting, the first region and the second region are highlighted in different ways. The program according to claim 9.

11. receiving an input of the second class for the second area; A process of generating training data based on at least the second region and the second class; The program according to claim 9 or 10, further comprising:

12. A program described in any one of claims 9 to 11, wherein the second class indicates a different type of object from the first class.

Images