Display support device, display support method, and program

Abstract

This display assistance device (100) comprises a detection result acquisition unit (102) that includes a plurality of detection objects, and acquires detection results of an image in which detection processing of the detection objects has been performed, a display processing unit (104) that displays the detection results of the acquired image, and an instruction acquisition unit (106) that acquires information indicating an instruction for the detection results, wherein: the display processing unit (104) establishes a predetermined number of the detection objects as detection result display objects, and displays, in association with the image, position information indicating positions in the image of the detection objects and a score indicating the certainty of the detection objects; and when the instruction acquisition unit (106) acquires switching information indicating an instruction to switch the detection objects serving as the detection result display objects, the display processing unit (104) switches the detection result display objects to different detection objects in the image, and displays position information and a score relating to the detection objects after the switching.

Description

[Technical Field]

[0001] The present invention relates to a display assistance device, a display assistance method, and a recording medium. [Background technology]

[0002] An example of a system for detecting objects through image analysis using a machine learning model is described in Patent Document 1. The system in Patent Document 1 includes an imaging device that captures an image within a target area, an image processing unit that stores a program that uses deep learning to detect objects within the image captured by the imaging device, and a boundary determination unit that determines the positional relationship between a transparent portion that allows the outside to be seen from inside the target area and surrounding objects, and is configured to perform mask processing on the area occupied by the transparent portion of the image captured by the imaging device, and detect objects based on the image that has undergone mask processing using a program that uses deep learning.

[0003] The results of detection using deep learning are displayed by enclosing the detected object in the target image in a rectangle to indicate the object's location, and are labeled with the detected object's identifier and information (score) indicating the accuracy of the detected object.

[0004] Patent Document 2 describes an example of an image processing device that is designed to facilitate confirmation of verification results and pass / fail judgment results for objects detected from an input image. The image processing device in Patent Document 2 includes an object detection means that detects one or more images of an object from an input image based on a model pattern of the object, and a detection result display means that graphically displays the detection results in a superimposed manner. The detection result display means includes a first frame that displays the entire input image and a second frame that displays a list of partial images that include each of the detected one or more images, with the detection results for all of the detected images superimposed on the input image displayed in the first frame, and the detection results for the images corresponding to each partial image superimposed on the partial images displayed in the second frame.

[0005] In addition, Patent Document 3 describes an example of an image playback device that displays still images in an easy-to-view manner according to the number of human subjects. When this pixel playback device of Patent Document 2 displays a plurality of still images as a slide show, it is determined whether the number of pixels of the still image is greater than a predetermined reference number of pixels. Also, it is determined whether the screen resolution of the display on which the still image is displayed is lower than a predetermined reference resolution. When these conditions are met, the faces of the people in the still image are detected, and it is determined whether the detected number of people is greater than a predetermined reference number of people. When it is determined that the number is greater than the reference number of people, the number of image cutouts is determined according to the detected number of people, and a plurality of images are cut out based on the range in which the number of people less than the reference number of people appears. The plurality of cut-out images are each displayed equivalently to a single still image. As a result, the substantial playback time for displaying one still image becomes longer, and each person is displayed larger.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0007] Since the learning model affects the image analysis result, by having the operator view and confirm the image in which the object detection result is displayed and analyzing the tendency of the learning model, measures can be taken to improve the accuracy of the detection result. The technology described in Patent Document 1 above only relates to object detection using deep learning, and is not intended to evaluate learning models. Furthermore, the technologies described in Patent Documents 2 and 3 describe techniques that make it easy to confirm multiple objects detected from an image, but do not anticipate situations in which the contents of the detection results cannot be confirmed because multiple overlapping detection results are displayed, as in the case of object detection using deep learning.

[0008] In response to this, the inventors have considered improvements to solve the problem that when checking the detection results of object detection using deep learning, the rectangular frames showing the detection results overlap with the identifiers and scores of the detected objects, making the detection results difficult to see, and making it difficult to evaluate the detection results of the learning model.

[0009] One example of the object of the present invention is to provide a display assistance device, a display assistance method, and a recording medium that solve the difficulty of evaluating the detection results of a learning model using deep learning, in view of the above-mentioned problems. [Means for solving the problem]

[0010] According to one aspect of the present invention, a detection result acquisition means for acquiring a detection result of an image that includes a plurality of detection targets and has undergone a detection process for the detection targets; a display processing means for displaying the detection result of the acquired image; an instruction acquisition means for acquiring information indicating an instruction for the detection result, the display processing means displays a predetermined number of the detection targets as detection result display targets, and associates position information indicating positions of the detection targets within the image and scores indicating likelihoods of the detection targets with the image; A display assistance device is provided in which, when the instruction acquisition means acquires switching information indicating an instruction to switch the detection target to be the target for displaying the detection results, the display processing means switches the detection result display target to another detection target within the image and displays the position information and score related to the detection target after the switch.

[0011] According to one aspect of the present invention, One or more computers Acquire a detection result of an image that includes a plurality of detection targets and has undergone a detection process for the detection targets; Displaying the detection result of the acquired image; a predetermined number of the detection targets are displayed as detection result display targets, and position information indicating the positions of the detection targets within the image and a score indicating the likelihood of the detection targets are displayed in association with the image; A display assistance method is provided in which, when switching information indicating an instruction to switch the detection target to be the target for displaying the detection results is acquired, the detection result display target is switched to another detection target within the image, and the position information and score related to the detection target after the switch are displayed.

[0012] According to one aspect of the present invention, On the computer, a step of acquiring a detection result of an image including a plurality of detection targets and subjected to a detection process for the detection targets; a step of displaying the detection result of the acquired image; a step of acquiring information indicating an instruction for the detection result; a step of displaying a predetermined number of the detection targets as detection result display targets, and associating position information indicating the positions of the detection targets within the image and scores indicating the likelihood of the detection targets with the image; When switching information indicating an instruction to switch the detection target to be the display target of the detection result is obtained in the acquisition procedure, the detection result display target is switched to another detection target in the image, and the position information and the score regarding the detection target after the switching are displayed. A computer-readable storage medium storing a program for executing the procedure is provided.

[0013] Note that the present invention may also include a program recorded on a computer-readable recording medium on which a program of one aspect of the present invention is recorded. This recording medium includes a non-temporary tangible medium. This computer program includes computer program code that causes a computer to execute the authentication method on an authentication device when executed by the computer.

[0014] Note that any combination of the above components, and those obtained by converting the expression of the present invention among a method, an apparatus, a system, a recording medium, a computer program, etc., are also effective as aspects of the present invention.

[0015] Also, the various components of the present invention do not necessarily have to exist independently of each other. It is also possible that a plurality of components are formed as one member, one component is formed of a plurality of members, a certain component is a part of another component, a part of a certain component and a part of another component overlap, and so on.

[0016] Also, although a plurality of procedures are described in order in the method and computer program of the present invention, the described order does not limit the order of executing the plurality of procedures. Therefore, when implementing the method and computer program of the present invention, the order of the plurality of procedures can be changed within a range that does not cause any problem in terms of content.

[0017] Furthermore, the multiple procedures of the method and computer program of the present invention are not limited to being executed at different timings individually. Therefore, other procedures may occur during the execution of a certain procedure, or a part or all of the execution timing of a certain procedure and the execution timing of other procedures may overlap, etc.

Advantages of the Invention

[0018] According to one aspect of the present invention, it is possible to solve the difficulty of the evaluation work of the detection results of a learning model using deep learning.

Brief Description of the Drawings

[0019]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Embodiments for Carrying Out the Invention

[0020] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same reference numerals are given to the same components, and the description will be omitted as appropriate. Also, in the following figures, the configuration of parts not related to the essence of the present invention is omitted and not shown.

[0021] In the embodiment, "acquisition" includes at least one of the act of the own device going to obtain data or information stored in another device or storage medium (active acquisition), and the act of the own device inputting data or information output from another device (passive acquisition). Examples of active acquisition include making a request or inquiry to another device and receiving its reply, and accessing another device or storage medium to read it out. Examples of passive acquisition include receiving information distributed (or transmitted, push-notified, etc.). Further, "acquisition" may mean selecting and acquiring from the received data or information, or selecting and receiving the distributed data or information.

[0022] <Minimum Configuration Example> FIG. 1 is a diagram showing an overview of a display support device 100 according to an embodiment. The display support device 100 includes a detection result acquisition unit 102, a display processing unit 104, and an instruction acquisition unit 106. The detection result acquisition unit 102 acquires the detection result of an image in which detection processing of a plurality of detection targets has been performed, including the plurality of detection targets. The display processing unit 104 displays the detection result of the acquired image. The instruction acquisition unit 106 acquires information indicating an instruction for the detection result. The display processing unit 104 uses a predetermined number of detection targets as detection result display targets, associates position information indicating the position within the image of the detection target and a score indicating the likelihood of the detection target with the image, and displays them. When the instruction acquisition unit 106 acquires switching information indicating an instruction to switch the detection target that is the detection result display target, the display processing unit 104 switches the detection result display target to another detection target within the image, and displays the position information and score related to the detection target after the switching.

[0023] The display processing unit 104 causes a display device (not shown) connected to the display support device 100 to display the target image and superimposes and displays the detection result. Since it becomes difficult to view the display when there are multiple detection results, the display processing unit 104 displays a predetermined number of detection targets as detection result display targets. The predetermined number may be, for example, one, but may also be a plurality.

[0024] <Operation Example> FIG. 2 is a flowchart showing an example of the operation of the display support device 100 in FIG. 1. First, in the display support device 100, the detection result acquisition unit 102 acquires the detection result of an image in which a plurality of detection targets are included and the detection process of the detection targets has been performed (step S101). Then, the display processing unit 104 uses a predetermined number of detection targets as detection result display targets, associates position information indicating the position within the image of the detection target and a score indicating the likelihood of the detection target with the image, and displays them (step S103). Then, the instruction acquisition unit 106 acquires information indicating an instruction for the detection result (step S105). When the instruction acquisition unit 106 acquires switching information indicating an instruction to switch the detection target that is the detection result display target (YES in step S107), the display processing unit 104 switches the detection result display target to another detection target within the image (step S109), proceeds to step S103, and displays a predetermined number of the position information and score related to the detection target after the switching.

[0025] As described above, in this display assistance device 100, the detection result acquisition unit 102 acquires the detection results of the image, and the display processing unit 104 displays a predetermined number of detection targets from the acquired detection results as detection result display targets on the display device 110. Then, when the instruction acquisition unit 106 acquires switching information indicating an instruction to switch the detection target as the detection result display target, the display processing unit 104 switches the detection result display target to another detection target in the image, and displays the position information and score related to the detection target after the switch. As a result, the display assistance device 100 can switch between displaying a predetermined number of detection results from among a large number of detection results, making the detection results easier to see and solving the difficulty of evaluating the detection results of learning models using deep learning.

[0026] A detailed example of the display assistance device 100 will be described below.

[0027] (First embodiment) <System Overview> FIG. 3 is a diagram conceptually showing the system configuration of an image analysis system 1 according to an embodiment. The image analysis system 1 includes a display assistance device 100 and an image analysis device 20. The image analysis device 20 performs object detection by analyzing an image through deep learning using a learning model 30, and stores the detection results in a detection result storage unit 40. The display assistance device 100 is connected to a display device 110 and an operation unit 120. The display device 110 is a liquid crystal display, an organic EL (Electro-Luminescence) display, or the like. The operation unit 120 is a keyboard, a mouse, or the like. The display device 110 and the operation unit 120 may be integrated into a touch panel.

[0028] The display assistance device 100 displays the detection results analyzed by the image analysis device 20 on the display device 110. The operator checks the detection results displayed on the display device 110 and analyzes the trends of the learning model 30.

[0029] FIG. 4 is a diagram showing an example of a detection result displayed on the display device 110. In the image 200 to be detected, position information (rectangular frame 210) indicating the position within the image to be detected and a label 220 including a score indicating the likelihood of the detection target are superimposed and displayed.

[0030] In this example, the label 220 includes identification information (for example, "0" for a person) indicating the category of the detected object and a score. The category of the object to be detected is, for example, a person, food, a car, etc.

[0031] The score is a score generated by a deep learning model. The score is indicated by a value from 0 to 1 (a numerical value with three decimal places), and the larger the numerical value, the higher the likelihood of the detection result. In this example, the score is enclosed in [square brackets]. The identification information is shown before the [square brackets]. However, these are just examples, and the display method of the label 220 is not limited to these.

[0032] The score generated by the deep learning model is indicated by a numerical value with three decimal places and is attached with a label 220 for each detection target and displayed. Therefore, the more detection targets there are in the image 200, the more they are displayed overlapping, making it difficult to confirm the score. However, according to the present embodiment, since they can be switched and displayed by a predetermined number, even the score of the deep learning model can be easily confirmed.

[0033] <Hardware Configuration Example> FIG. 5 is a block diagram illustrating the hardware configuration of a computer 1000 that realizes the display support device 100 shown in FIG. 1. The image analysis device 20 of the image analysis system 1 in FIG. 3 is also realized by the computer 1000.

[0034] The computer 1000 includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0035] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0036] The processor 1020 is implemented by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0037] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0038] The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read-only memory (ROM), or the like. The storage device 1040 stores program modules that realize each function of the display assistance device 100 (e.g., the detection result acquisition unit 102, the display processing unit 104, the instruction acquisition unit 106, the storage processing unit 108, etc., of FIG. 1 ). The processor 1020 loads each of these program modules into the memory 1030 and executes them, thereby realizing each function corresponding to the program module. The storage device 1040 also functions as a storage device (not shown) that stores various information used by the display assistance device 100, or as the evaluation result storage unit 130. The storage device 1040 may also function as a storage device that stores various information used by the image analysis device 20 (e.g., the learning model 30, the detection result storage unit 40).

[0039] The program module may be recorded on a recording medium. The recording medium for recording the program module includes a non-transitory tangible medium that can be used by the computer 1000, and program code readable by the computer 1000 (processor 1020) may be embedded in the medium.

[0040] The input / output interface 1050 is an interface for connecting the computer 1000 and various input / output devices.

[0041] The network interface 1060 is an interface for connecting the computer 1000 to a communication network. This communication network is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The method by which the network interface 1060 connects to the communication network may be a wireless connection or a wired connection. However, the network interface 1060 may not be used.

[0042] And the computer 1000 is connected to necessary devices (for example, the display device 110 and the operation unit 120 of the display support device 100) via the input / output interface 1050 or the network interface 1060.

[0043] The display support device 100 and the image analysis device 20 may each be realized by a plurality of computers 1000. Alternatively, the display support device 100 may be incorporated into the image analysis device 20. The computer 1000 that realizes the display support device 100 or the image analysis device 20 may be a personal computer or a server computer. The display support device 100 may be a tablet terminal or a smartphone.

[0044] The image analysis device 20 may be a device incorporated into a device that requires image analysis in various fields.

[0045] Each component of the display support device 100 in each embodiment of FIG. 1 is realized by any combination of the hardware and software of the computer 1000 in FIG. 5. And it is understood by those skilled in the art that there are various modifications to the realization method and device. The functional block diagram showing the display support device 100 in each embodiment shows blocks of logical functional units, not configurations in hardware units.

[0046] <Functional configuration example> Hereinafter, a functional configuration example of the display support device 100 will be described in detail with reference to FIG. 1. The detection result acquisition unit 102 acquires the detection result of the image 200 from the detection result storage unit 40. The detection result includes an identifier indicating the category of the object detected from the image 200, position information indicating the position of the object (for example, the coordinate position information (ymin, xmin, ymax, xmax) of the rectangular frame 210 in the image 200), and a score indicating the certainty of the recognition result.

[0047] The display processing unit 104 displays the detection result of the image 200. FIGS. 6 and 7 are diagrams showing display examples of the detection result of the image 200. FIG. 6 shows an example in which all the detection results of the image 200 are shown. Since many detection results overlap, it can be seen that the scores are also difficult to view.

[0048] Therefore, the display processing unit 104 associates and displays the rectangular frame 210 and the label 220 with the image 200, taking a predetermined number of detection targets as detection result display targets. In the example of FIG. 7, the detection results in the image 200 are displayed one by one.

[0049] The position information is a rectangle surrounding the detection target in the image 200. The display processing unit 104 draws a rectangle surrounding the detected target on the image 200. The display processing unit 104 displays the score outside the rectangle. However, the position information may be in other forms, such as an ellipse surrounding the detection target, or an arrow or balloon indicating the detection target. In the case of an arrow, the score may be displayed at the base of the mark. In the case of a balloon, the score may be displayed inside the balloon.

[0050] Since the position information is displayed as a rectangle surrounding the detection target, the operator can recognize the detection target at a glance.

[0051] The instruction acquisition unit 106 acquires switching information indicating an instruction to switch the detection target to be the detection result display target. The instruction acquisition unit 106 acquires an input from the operator as the switching information of the detection target.

[0052] The switching information may include direction information indicating the direction to switch the detection target to be the detection result display target. The display processing unit 104 displays the position information and the score of the detection target located in the direction indicated by the input direction information from the currently detected detection target to be the next detection result display target.

[0053] Examples of the input from the operator are illustrated below, but are not limited thereto. These may be combined in plural. (1) By pressing the arrow keys on the keyboard, move up and down, or left and right, or up, down, left, and right (2) Scroll up and down by operating the mouse wheel (3) Instructions such as "next", "back", etc. by voice, instructions such as "up", "down", "right", "left", etc.

[0054] When the instruction acquisition unit 106 acquires the switching information, the display processing unit 104 switches the detection result display target to another detection target in the image 200, and displays the position information (rectangular frame 210) and the score (label 220) regarding the detection target after the switching.

[0055] Since the instruction acquisition unit 106 switches the detection target in response to the input of the operator, the detection target can be switched at the timing of the operator to display the detection result, and it becomes easy to confirm each detection result. In addition, since the switching direction of the detection target can be specified using the arrow keys or the like, it is easy to reflect the intention of the operator and the operability is improved.

[0056] Further, the instruction acquisition unit 106 may be the output of a timer indicating the elapse of a certain period of time as switching information for the detection target. In this case, the display processing unit 104 automatically switches the detection target every time the certain period of time elapses and causes it to be displayed.

[0057] <Operation example> Hereinafter, the operation of the display support apparatus 100 according to the embodiment will be described with reference to FIG. 2. First, the detection result acquisition unit 102 acquires the detection result of an image in which detection processing of a plurality of detection targets has been performed from the detection result storage unit 40 (step S101).

[0058] Then, the display processing unit 104 associates and displays, as detection result display targets, a predetermined number (one in the example of FIG. 7) of detection targets, position information (rectangular frame 210) indicating the position within the image 200 of the detection target, and a score (label 220) indicating the likelihood of the detection target with respect to the image 200 (step S103). First, the image 200 in FIG. 7(a) is displayed on the display device 110.

[0059] Then, the instruction acquisition unit 106 acquires information indicating an instruction for the detection result (step S105). Here, it is assumed that the operator presses the up arrow key on the keyboard (operation unit 120). The instruction acquisition unit 106 acquires information indicating that the up arrow key has been pressed.

[0060] When the instruction acquisition unit 106 acquires the switching information (pressing of the up arrow key) (YES in step S107), the display processing unit 104 switches the detection result display target to another detection target within the image 200 (step S109), proceeds to step S103, and displays the position information (rectangular frame 210) and the score (label 220) regarding the detection target after the switching for a predetermined number (one in this example). Here, the image 200 in FIG. 7(b) is displayed on the display device 110.

[0061] In image 200 of FIG. 7(a), the position information (rectangular frame 210) and score (label 220) are superimposed on the person in the foreground as the detection result, but in image 200 of FIG. 7(b), the detection result switches to the person behind him (second person), and the position information (rectangular frame 210) and score (label 220) are superimposed on him.

[0062] Furthermore, suppose that the operator presses the up arrow key on the keyboard (operation unit 120). The instruction acquisition unit 106 acquires information indicating that the up arrow key has been pressed.

[0063] When the instruction acquisition unit 106 acquires switching information (pressing the up arrow key) (YES in step S107), the display processing unit 104 switches the detection result display target to another detection target in the image 200 (step S109), proceeds to step S103, and displays a predetermined number (one in this example) of position information (rectangular frame 210) and scores (labels 220) related to the detection target after the switch. Here, the image 200 in FIG. 7(c) is displayed on the display device 110.

[0064] In image 200 of Figure 7(b), the detection result for the second person in front is superimposed with location information (rectangular frame 210) and score (label 220), but in image 200 of Figure 7(c), the detection result switches to the person further back (third person), and location information (rectangular frame 210) and score (label 220) are superimposed.

[0065] As described above, in this display assistance device 100, the detection result acquisition unit 102 acquires the detection results of the image analyzed by the image analysis device 20 using a deep learning model, and a predetermined number of detection targets from the acquired detection results are displayed on the display device 110 as detection result display targets. Then, when the instruction acquisition unit 106 acquires switching information indicating an instruction to switch the detection target to be the detection result display target, the display processing unit 104 switches the detection result display target to another detection target in the image and displays the position information and score related to the detection target after the switch.

[0066] Accordingly, according to the display support device 100, it is possible to switch and display a predetermined number of detection results from among a large number of detection results, so that the detection results are easier to view, and the difficulty of evaluating the detection results of the learning model using deep learning can be solved.

[0067] (Second Embodiment) This embodiment is the same as the above embodiment, except that it has a configuration in which a partial image is cut out and the detection target result is displayed. Since the display support device 100 of this embodiment has the same configuration as that of the first embodiment, it will be described with reference to FIG. 1. Note that the configuration of this embodiment may be combined within a range that does not conflict with at least any one of the configurations of other embodiments.

[0068] <Functional Configuration Example> The instruction acquisition unit 106 acquires region designation information indicating an instruction to designate a region 240 that is a part of the image 200 and includes a plurality of detection targets. When the instruction acquisition unit 106 acquires the region designation information, the display processing unit 104 cuts out the designated region 240 from the image 200 and displays it, and displays position information (rectangular frame 210) and a score (220) for a predetermined number of detection targets included in the region 240.

[0069] <Operation Example> FIG. 8 is a flowchart showing an operation example of a characteristic part of the display support device 100 according to the embodiment. The flowchart of FIG. 8 is executed between step S101 and step S103 of FIG. 2. Hereinafter, description will be made using the diagrams showing examples of the image 200 in FIGS. 6 and 9 to 11.

[0070] In step S101, after the detection result acquisition unit 102 acquires the detection result of the image 200 from the detection result storage unit 40, the display processing unit 104 displays the detection result of the image 200 acquired in step S101 on the display device 110 (step S121). At this time, the image 200 in FIG. 6 is displayed on the display device 110.

[0071] Assume that the operator uses the operation unit 120 (e.g., a mouse) to specify a region 240 that is a part of the image 200 and includes a plurality of detection targets. In FIG. 9, the region is specified by a rectangle 230.

[0072] When the instruction acquisition unit 106 acquires the region specification information (YES in step S123), the display processing unit 104 cuts out the specified region 240 from the image 200 and displays it (step S125). FIG. 10 is a diagram showing an example of the cut-out region 240. At this time, the region 240 is preferably enlarged. The magnification may be specified by the operator. Alternatively, the operator may be able to enlarge or reduce by pinching out or pinching in the region 240.

[0073] Then, it proceeds to step S103 in FIG. 2. The display processing unit 104 associates and displays, as detection result display targets, a predetermined number (one in the example of FIG. 11) of detection targets, position information (rectangular frame 210) indicating the position of the detection target in the image 200, and a score (label 220) indicating the probability of the detection target, with respect to the image 200 (step S103). First, the image 200 in FIG. 11(a) is displayed on the display device 110.

[0074] Then, the instruction acquisition unit 106 acquires information indicating an instruction for the detection result (step S105). Here, assume that the operator presses the up arrow key on the keyboard (operation unit 120). The instruction acquisition unit 106 acquires information indicating that the up arrow key has been pressed.

[0075] When the instruction acquisition unit 106 acquires the switching information (pressing of the up arrow key) (YES in step S107), the display processing unit 104 switches the detection result display target to another detection target in the image 200 (step S109), proceeds to step S103, and displays the position information (rectangular frame 210) and score (label 220) regarding the detection target after the switching, a predetermined number (one in this example). Here, the image 200 in FIG. 11(b) is displayed on the display device 110.

[0076] In the image 200 of Fig. 11(a), the position information (rectangular frame 210) and the score (label 220) are superimposed and displayed as the detection result on the frontmost person. However, in the image 200 of Fig. 11(b), the detection result switches to the person behind (the second one), and the position information (rectangular frame 210) and the score (label 220) are superimposed and displayed.

[0077] Furthermore, assume that the operator presses the up arrow key on the keyboard (operation unit 120). The instruction acquisition unit 106 acquires information indicating that the up arrow key has been pressed.

[0078] When the instruction acquisition unit 106 acquires the switching information (pressing of the up arrow key) (YES in step S107), the display processing unit 104 switches the detection result display target to another detection target within the image 200 (step S109), proceeds to step S103, and displays the position information (rectangular frame 210) and the score (label 220) regarding the detection target after the switching for a predetermined number (one in this example). Here, the image 200 of Fig. 11(c) is displayed on the display device 110.

[0079] In the image 200 of Fig. 11(b), the position information (rectangular frame 210) and the score (label 220) are superimposed and displayed as the detection result on the second frontmost person. However, in the image 200 of Fig. 11(c), the detection result further switches to the person behind (the third one), and the position information (rectangular frame 210) and the score (label 220) are superimposed and displayed.

[0080] As described above, in this display support device 100, when the instruction acquisition unit 106 acquires the region designating information that cuts out a part of the image 200 and designates the region 240 including a plurality of detection targets, the display processing unit 104 cuts out the designated region 240 from the image 200 and displays it, and displays the position information (rectangular frame 210) and the score (label 220) for a predetermined number of detection targets included in the region 240.

[0081] As a result, the display support device 100 exhibits the same effects as those in the above-described embodiment, and furthermore, since it is possible to cut out the region 240 that is of particular interest or the region 240 in which the detection target is complicated and confirm the detection result, the detection result becomes easier to see.

[0082] (Modification of the Second Embodiment) In the second embodiment, the display processing unit 104 cuts out and displays the region 240 in accordance with the region designation information. As a modification, instead of cutting out the region 240, only the display target of the detection result may be limited to the region 240.

[0083] For example, as shown in FIG. 9, when the operator designates a region of the image 200 with the rectangle 230, the instruction acquisition unit 106 acquires the region designation information. Then, for a plurality of detection targets in the region corresponding to the region 240 designated by the rectangle 230, each time the instruction acquisition unit 106 acquires the switching information, the detection results are sequentially switched and displayed as shown in FIGS. 7(a) to 7(c).

[0084] According to this configuration, since the detection result can be confirmed for the region 240 that is of particular interest, the work efficiency can be improved.

[0085] (Third Embodiment) This embodiment is the same as the first embodiment except that it has a configuration in which the detection results of a plurality of detection targets are displayed in a list, and the search target for displaying the detection result on the image is switched by selecting the list display. Since the display support device 100 of this embodiment has the same configuration as that of the first embodiment, it will be described with reference to FIG. 1. Note that the configuration of this embodiment may be combined within a range that does not conflict with at least any one of the configurations of other embodiments.

[0086] <Functional Configuration Example>

[0087] The display processing unit 104 displays a list of the detection results of a plurality of detection targets. The instruction acquisition unit 106 acquires selection information indicating a detection target selected from the list display. The display processing unit 104 displays the search result of the detection target indicated by the selection information in association with the image 300.

[0088] FIG. 12 is a diagram showing a display example of the detection result of the image 300 of the embodiment. In the image 300 of FIG. 12(a), position information (rectangular frame 310) indicating the position of the detection target is displayed. FIG. 12(b) is a diagram showing an example of the search result list 330. The search result list 330 may be displayed in a window different from the image 300, for example. The two windows may be arranged side by side. One record (row) is assigned to each detected object in the search result list 330.

[0089] Each record 332 has a check box 334 and an identification information display unit 336. Since the search result list 330 includes a plurality of records 332, it may include a scroll bar 338. The check box 334 is a UI (User Interface) that accepts a designation as to whether or not to display the rectangular frame 310 surrounding the object of the detection target corresponding to the record 332 in the image 300. For example, when the check box 334 is checked, the display processing unit 104 displays the corresponding rectangular frame 310 in the image 300, and when the check is removed, the rectangular frame 310 is made invisible in the image 300. The category of the object of the detection target is displayed in the identification information display unit 336.

[0090] The display processing unit 104 collectively displays the detection results according to the attributes of the detection targets. Here, the attribute of the detection target is the category of the object. However, the attribute of the detection target is not limited to this. For example, in the case of a person, the gender may be included in the attribute, and the detection result may also recognize the attributes of the person.

[0091] Also, for example, the plurality of records 332 in the search result list 330 may be sorted and displayed by category of the object. The display processing unit 104 sorts the records 332 by category of the detection result of the detection target of the record 332 and displays the search result list 330. In this example, the categories include cars, bicycles, and people. Also, the display processing unit 104 may display the background colors of the identification information display units 336 in different colors for each category. In other examples, it may be possible to accept a category designation and collectively select or deselect the detection results of the detection targets of the designated category.

[0092] <Operation example> FIG. 13 is a flowchart showing an operation example of the display support device 100 according to the embodiment. FIGS. 14 to 16 are diagrams showing display examples of the detection results of the image 300 according to the embodiment. The flow of FIG. 13 has the same step S101 as FIG. 2, and after step S101, it has steps S131 to S135.

[0093] First, the detection result acquisition unit 102 acquires, from the detection result storage unit 40, the detection results of the images in which the detection processes of a plurality of detection targets have been performed and which include the plurality of detection targets (step S101). Then, the display processing unit 104 causes the display device 110 to display the image 300 and the search result list 330 of FIG. 12 (step S131). Then, when the instruction acquisition unit 106 acquires the selection information of the record 340 as shown in FIG. 14(b) (YES in step S133), the display processing unit 104 causes the score of the detection result of the selected detection target (the label 322 including the identification information and score of the category) to be displayed on the image 300 as shown in FIG. 13(a) (step S135).

[0094] In step S133, although one record 340 is selected, a plurality of records 332 may be selectable.

[0095] The display processing unit 104 may highlight 320 the position information (rectangular frame 310) indicating the position of the detection result of the selected detection target. For example, the rectangular frame 310 may be displayed in a different color, with thicker borders, blinking, or shaded, or a combination of these.

[0096] Furthermore, as shown in Fig. 15(b), when the operator selects the next record 340, the instruction acquisition unit 106 acquires the selection information of the record 340 (YES in step S133). Then, as shown in Fig. 15(a), the display processing unit 104 displays the score of the detection result of the selected detection target (label 322 and highlight 320 including the category identification information and score) on the image 300 (step S135). From Fig. 14(a) to Fig. 15(a), the detection target for which the label 322 and highlight 320 are displayed has moved to the adjacent detection target.

[0097] In the example of Fig. 16, when the operator further selects another record 340, the instruction acquisition unit 106 acquires the selection information of the record 340 (YES in step S133). Then, the display processing unit 104 displays the score of the detection result of the selected detection target (label 322 and highlight 320 including the category identification information and score) on the image 300, as shown in Fig. 16(a) (step S135). From Fig. 15(a) to Fig. 16(a), the detection target for which the label 322 and highlight 320 are displayed has moved to the adjacent detection target.

[0098] As described above, in the display assistance device 100 of this embodiment, the display processing unit 104 displays a list of detection results for a plurality of detection targets, and when the instruction acquisition unit 106 acquires selection information indicating a detection target selected from the list display, the display processing unit 104 displays the search results (label 322, etc.) for the detection target indicated by the selection information in association with the image 300. The display processing unit 104 also displays the detection results grouped by attribute of the detection target.

[0099] As a result, first, a plurality of detection results can be viewed in the search result list 330. Then, the search result list 330 can be confirmed by category of the detection target, enabling systematic confirmation of a large number of detection targets and improving the efficiency of the analysis work.

[0100] (Fourth Embodiment) This embodiment is the same as any of the above-described embodiments except that it has a configuration that can select and store detection results. FIG. 17 is a diagram showing an overview of the display support device 100 of the embodiment.

[0101] <Functional Configuration Example> The display support device 100 of the embodiment further includes a storage processing unit 108 in addition to the configuration of FIG. 1. The instruction acquisition unit 106 acquires selection information indicating an instruction to select a detection target. The storage processing unit 108 stores the detection result of the selected detection target in the evaluation result storage unit 130.

[0102] The operator can select and store the detection results to be confirmed later. For example, it is possible to select and store detection results with a score lower than a predetermined value and then collectively confirm them during later analysis. Also, since the detection results of other images can be stored in the evaluation result storage unit 130 together, the detection results of detection targets included in other images can be analyzed together across images.

[0103] As described above, according to this display support device 100, the storage processing unit 108 stores the detection result of the detection target selected based on the selection information acquired by the instruction acquisition unit 106 in the evaluation result storage unit 130. Therefore, for example, by selecting and storing detection results with a low score, it is possible to collectively confirm the images of detection results with a low score later, thus improving the efficiency of the analysis work.

[0104] Although the embodiments of the present invention have been described above with reference to the drawings, these are examples of the present invention, and various configurations other than the above can also be adopted. For example, in the configuration of any of the above-described embodiments, as switching information, the detection results of the detection target are sequentially switched and displayed by pressing the arrow keys of the operator or the scrolling operation of the mouse. However, by pressing the enter key, escape key, etc., the selection of the detection target may be canceled and the display may return to the display of all detection results.

[0105] Also, in the plurality of flowcharts used in the above description, a plurality of steps (processes) are described in order. However, the execution order of the steps executed in each embodiment is not limited to the order of the description. In each embodiment, the order of the illustrated steps can be changed within a range that does not substantially affect the content. Also, the above-described embodiments can be combined within a range where the contents do not conflict.

[0106] The present invention has been described above with reference to the embodiments, but the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. In the present invention, when information regarding a user (operator) is acquired and used, it is assumed that this is done legally.

[0107] Some or all of the above embodiments can be described as follows in the appended claims, but are not limited thereto. 1. Detection result acquisition means for acquiring detection results of an image in which detection processing of a plurality of detection targets has been performed, the detection targets including a plurality of detection targets; Display processing means for displaying the acquired detection results of the image; Instruction acquisition means for acquiring information indicating an instruction for the detection results, The display processing means displays, as detection result display targets, a predetermined number of the detection targets, associating with the image position information indicating the position of the detection target in the image and a score indicating the likelihood of the detection target. When the instruction acquisition means acquires switching information indicating an instruction to switch the detection target to be the detection result display target, the display processing means switches the detection result display target to another detection target in the image, and displays the position information and the score regarding the detection target after the switching. A display support device. 2. In the display support device according to 1., The position information is a rectangle surrounding the detection target in the image, The display processing means displays the score outside the rectangle. A display support device. 3. In the display support device according to 1. or 2., When the instruction acquisition means acquires area designation information indicating an instruction to designate an area that is a part of the image and includes a plurality of detection targets, the display processing means cuts out and displays the designated area from the image, and for the predetermined number of detection targets included in the area, displays the position information and the score. A display support device. 4. In the display support device according to any one of 1. to 3., The instruction acquisition means acquires an input from an operator as switching information for the detection target. A display support device. 5. In the display support device according to any one of 1. to 4., The switching information includes direction information indicating a direction to switch the detection target to be the detection result display target, The display processing means, From the detection target that is currently the detection result display target, the detection target located in the direction indicated by the input direction information is set as the next detection result display target, and the position information and the score are displayed. A display support device. 6. In the display support device according to any one of 1. to 5., The display processing means, Displays a list of detection results of a plurality of the detection targets, The instruction acquisition means acquires selection information indicating the detection target selected from the list display, The display processing means is a display support device that associates and displays the search results of the detection target indicated by the selection information with the image. 7. In the display support device according to 6., The display processing means is a display support device that collectively displays the detection results by attribute of the detection target. 8. In the display support device according to any one of 1. to 7., The score is a score generated by a learning model, and it is a display support device. 9. In the display support device according to any one of 1. to 8., The instruction acquisition means acquires information indicating an instruction to select the detection target, and further includes a storage processing means for storing the detection results of the selected detection target in a storage means, and it is a display support device.

[0108] 10. One or more computers include a plurality of detection targets, acquire the detection results of an image in which the detection process of the detection target has been performed, display the acquired detection results of the image, regard a predetermined number of the detection targets as detection result display targets, and associate and display position information indicating the position of the detection target in the image and a score indicating the likelihood of the detection target with the image, When switching information indicating an instruction to switch the detection target to be the detection result display target is acquired, the detection result display target is switched to another detection target in the image, and the position information and the score regarding the detection target after the switching are displayed. This is a display support method. 11. In the display support method according to 10., The position information is a rectangle surrounding the detection target in the image, and the one or more computers display the score outside the rectangle. This is a display support method. 12. In the display support method according to 10. or 11., the one or more computers When obtaining region specification information indicating an instruction to specify a region that is part of the image and includes a plurality of detection targets, the specified region is cut out from the image and displayed, and the position information and the score are displayed for the predetermined number of detection targets included in the region. A display support method. 13. In the display support method according to any one of 10. to 12., the one or more computers obtain an input from an operator as switching information for the detection target. A display support method. 14. In the display support method according to any one of 10. to 13., the switching information includes direction information indicating a direction in which the detection target to be the detection result display target is switched, and the one or more computers display the position information and the score with the detection target located in the direction indicated by the input direction information from the detection target that is currently the detection result display target as the next detection result display target. A display support method. 15. In the display support method according to any one of 10. to 14., the one or more computers display a list of detection results of a plurality of the detection targets, obtain selection information indicating the detection target selected from the list display, and display the search result of the detection target indicated by the selection information in association with the image. A display support method. 16. In the display support method according to 15., the one or more computers display the detection results grouped by attributes of the detection target. A display support method. 17. In the display support method according to any one of 10. to 16., the score is a score generated by a learning model. A display support method. 18. In the display support method according to any one of 10. to 17., the one or more computers Obtain information indicating an instruction to select the detection target, A display support method for causing a storage means to store a detection result of the selected detection target.

[0109] 19. On a computer, A procedure for obtaining a detection result of an image in which a detection process of a plurality of detection targets has been performed, A procedure for displaying the obtained detection result of the image, A procedure for obtaining information indicating an instruction for the detection result, A procedure for associating and displaying position information indicating a position of the detection target in the image and a score indicating the likelihood of the detection target as a detection result display target for a predetermined number of the detection targets, When switching information indicating an instruction to switch the detection target to be the detection result display target is obtained in the obtaining procedure, a procedure for switching the detection result display target to another detection target in the image and displaying the position information and the score regarding the detection target after the switching. A program for causing the computer to execute the procedure. 20. In the program according to 19., The position information is a rectangle surrounding the detection target in the image, In the displaying procedure, a program for displaying the score outside the rectangle. 21. In the program according to 19. or 20., When region designation information indicating an instruction to designate a region that is a part of the image and includes a plurality of detection targets is obtained in the procedure for obtaining the information indicating the instruction, In the displaying procedure, Cut out the designated region from the image and display it, A program for displaying the position information and the score for the predetermined number of the detection targets included in the region. 22. In the program according to any one of 19. to 21., In the procedure of acquiring information indicating the instruction, a program that acquires an input from an operator as switching information of the detection target. 23. In the program according to any one of 19. to 22., the switching information includes direction information indicating a direction in which the detection target to be the detection result display target is switched, In the procedure of causing the display, a program that causes the position information and the score to be displayed with the detection target located in the direction indicated by the input direction information as the next detection result display target, starting from the detection target that is currently the detection result display target. 24. In the program according to any one of 19. to 23., in the procedure of causing the display, a list display of detection results of a plurality of the detection targets is performed, in the procedure of acquiring information indicating the instruction, selection information indicating the detection target selected from the list display is acquired, in the procedure of causing the display, a program that causes a search result of the detection target indicated by the selection information to be displayed in association with the image. 25. In the program according to 24., in the procedure of causing the display, a program that causes the detection results to be collectively displayed according to the attributes of the detection targets. 26. In the program according to any one of 19. to 25., a program that the score is a score generated by a learning model. 27. In the program according to any one of 19. to 26., in the procedure of acquiring information indicating the instruction, information indicating an instruction to select the detection target is acquired, Furthermore, a program for causing a computer to execute a procedure of storing the detection result of the selected detection target in a storage means.

[0110] 28. To a computer, A procedure for obtaining a detection result of an image including a plurality of detection targets and having undergone detection processing of the detection targets A procedure for displaying the obtained detection result of the image A procedure for obtaining information indicating an instruction for the detection result A procedure for associating and displaying, for a predetermined number of the detection targets as detection result display targets, position information indicating a position of the detection target in the image and a score indicating the likelihood of the detection target in the image When switching information indicating an instruction to switch the detection target to be the detection result display target in the obtaining procedure is obtained, a computer-readable storage medium storing a program for causing execution of a procedure of switching the detection result display target to another detection target in the image and displaying the position information and the score related to the detection target after the switching 29. In the storage medium according to 28., The position information is a rectangle surrounding the detection target in the image A computer-readable storage medium storing a program for displaying the score outside the rectangle in the displaying procedure 30. In the storage medium according to 28. or 29., When region designation information indicating an instruction to designate a region that is a part of the image and includes a plurality of detection targets is obtained in the procedure for obtaining the information indicating the instruction In the displaying procedure Cut out and display the designated region from the image A computer-readable storage medium storing a program for displaying the position information and the score for the predetermined number of the detection targets included in the region 31. In the storage medium according to any one of 28. to 30., A computer-readable storage medium storing a program for obtaining an input from an operator as switching information for the detection target in the procedure for obtaining the information indicating the instruction 32. In the storage medium according to any one of 28. to 31., the switching information includes direction information indicating a direction for switching the detection target to be the detection result display target, and in the procedure for causing the display, a computer-readable storage medium storing a program that causes the computer to display the position information and the score with the detection target located in the direction indicated by the input direction information from the detection target that is currently the detection result display target as the next detection result display target. 33. In the storage medium according to any one of 28. to 32., in the procedure for causing the display, a list display of detection results of a plurality of the detection targets is performed, in the procedure for acquiring the information indicating the instruction, selection information indicating the detection target selected from the list display is acquired, and in the procedure for causing the display, a computer-readable storage medium storing a program that causes the computer to display the search result of the detection target indicated by the selection information in association with the image. 34. In the storage medium according to 33., in the procedure for causing the display, a computer-readable storage medium storing a program that causes the computer to collectively display the detection results according to the attributes of the detection target. 35. In the storage medium according to any one of 28. to 34., the score is a score generated by a learning model, and a computer-readable storage medium storing a program. 36. In the storage medium according to any one of 28. to 35., in the procedure for acquiring the information indicating the instruction, information indicating an instruction to select the detection target is acquired, and further, a computer-readable storage medium storing a program for causing a computer to execute a procedure of storing the detection result of the selected detection target in a storage unit.

Explanation of Signs

[0111] 1 Image analysis system 3 Communication network 20 Image analysis device 30 Learning model 40 Detection result memory unit 100 Display support device 102 Detection result acquisition unit 104 Display processing unit 106 Instruction acquisition unit 108 Memory processing unit 110 Display device 120 Operation unit 130 Evaluation result memory unit 200 Image 210 Rectangle frame 220 Label 240 Region 300 Image 310 Rectangle frame 320 Highlight 322 Label 330 Search result list 332 Record 334 Check box 336 Identification information display section 338 Scroll bar 340 Record 1000 Computer 1010 Bus 1020 Processor 1030 Memory 1040 Storage device 1050 Input / output interface 1060 Network interface

Claims

1. Detection result acquisition means for including a plurality of detection targets and acquiring a detection result of an image in which detection processing of the detection targets has been performed; Display processing means for displaying the acquired detection result of the image; Instruction acquisition means for acquiring information indicating an instruction for the detection result, comprising: The display processing means displays, as detection result display targets, a predetermined number of the detection targets, associating with the image position information indicating a position of the detection target in the image and a score indicating the likelihood of the detection target. When the instruction acquisition means acquires switching information indicating an instruction to switch the detection target that becomes the detection result display target, the display processing means switches the detection result display target to another detection target in the image and displays the position information and the score regarding the detection target after the switching. A display support device.

2. In the display support device according to Claim 1, The position information is a rectangle surrounding the detection target in the image, The display processing means displays the score outside the rectangle. A display support device.

3. In the display support device according to Claim 1 or 2, When the instruction acquisition means acquires area designation information indicating an instruction to designate an area that is a part of the image and includes a plurality of detection targets, the display processing means cuts out and displays the designated area from the image, and displays the position information and the score for the predetermined number of the detection targets included in the area. A display support device.

4. In the display support device according to any one of Claims 1 to 3, The instruction acquisition means acquires an input from an operator as switching information for the detection target. A display support device.

5. In the display support device according to any one of Claims 1 to 4, The switching information includes direction information indicating a direction to switch the detection target that becomes the detection result display target, The display processing means Displays the position information and the score with the detection target located in the direction indicated by the input direction information from the detection target that is currently the detection result display target as the next detection result display target. A display support device.

6. In the display support device according to any one of Claims 1 to 5, The display processing means Displays a list of detection results of a plurality of the detection targets. The instruction acquisition means acquires selection information indicating the detection target selected from the list display, The display processing means is a display support device that displays the search result of the detection target indicated by the selection information in association with the image.

7. In the display support device according to claim 6, The display processing means, is a display support device that collectively displays the detection results by attribute of the detection target.

8. In the display support device according to any one of claims 1 to 7, The score is a score generated by a learning model, and it is a display support device.

9. One or more computers include a plurality of detection targets, acquire the detection results of an image in which the detection process of the detection target has been performed, display the acquired detection results of the image, uses a predetermined number of the detection targets as detection result display targets, and associates and displays position information indicating the position of the detection target in the image and a score indicating the likelihood of the detection target with the image, When acquisition of switching information indicating an instruction to switch the detection target to be the detection result display target is performed, the detection result display target is switched to another detection target in the image, and the position information and the score regarding the detection target after the switching are displayed. This is a display support method.

10. On a computer, a procedure for acquiring the detection results of an image that includes a plurality of detection targets and in which the detection process of the detection target has been performed, a procedure for displaying the acquired detection results of the image, a procedure for acquiring information indicating an instruction for the detection results, a procedure for using a predetermined number of the detection targets as detection result display targets and associating and displaying position information indicating the position of the detection target in the image and a score indicating the likelihood of the detection target with the image, a program for causing the computer to execute a procedure of switching the detection target to be the detection result display target in the acquisition procedure, switching the detection result display target to another detection target in the image, and displaying the position information and the score regarding the detection target after the switching when acquisition of switching information indicating an instruction to switch the detection target to be the detection result display target is performed.

Images