Object detection device, object detection method, and program

Abstract

The objective of the present invention is to improve the accuracy of object detection for an image in which a large number of objects are captured. An object detection device is provided with: an image division unit configured to divide a target image in which a plurality of objects are captured into a plurality of partial images including boundary regions overlapping adjacent partial images; an object detection unit configured to detect, for each of the partial images, an object included in the partial image; and a detection result determination unit configured to determine, for an object detected within the boundary region, one of detection results relating to the object in the local image as a detection result relating to the object in the target image.

Description

Technical Field

[0001] The present disclosure relates to an object detection device, an object detection method, and a program. Background Art

[0002] Object detection technology is used to detect objects captured in an image. When performing object detection on an image in which a large number of objects are captured, a technique of dividing the image into a plurality of regions and performing object detection on each region is known.

[0003] For example, Non-Patent Document 1 discloses a technique of applying a convolutional neural network to semantic segmentation of a remote sensing image. In the technique disclosed in Non-Patent Document 1, an image having a size that cannot be directly processed by a conventional convolutional neural network is divided into partial images having a smaller size, and label maps of the respective partial images are stitched together to generate a label map of the original image.

[0004] [Cited Document]

[0005] [Non-Patent Document]

[0006] [Non-Patent Document 1] Bohao Huang, Daniel Reichman, Leslie M. Collins, Kyle Bradbury, and Jordan M. Malof, “Tiling and Stitching Segmentation Output for Remote Sensing: Basic Challenges and Recommendations”, arXiv:1805.12219, 2018. Summary of the Invention

[0007] [Technical Problem to be Solved]

[0008] However, the following problem exists in the prior art, that is, the detection accuracy of an object near the boundary of a partial image is low. Non-Patent Document 1 proposes the following solution, that is, in order to avoid dividing an object at the boundary of a partial image, the original image is divided into partial images in such a way that an overlapping portion is provided. In addition, Non-Patent Document 1 also makes the following suggestion, that is, when the detection results of the respective partial images are stitched together to generate the detection result of the original image, the detection results near the boundary are discarded.

[0009] One aspect of the present disclosure aims to improve the accuracy of object detection performed on an image in which a large number of objects are captured.

[0010] [Technical Solution]

[0011] The present disclosure has the following configuration.

[0012] [1] An object detection device, comprising:

[0013] An image segmentation unit configured to segment a target image capturing a plurality of objects into a plurality of partial images, the plurality of partial images including a boundary region overlapping with an adjacent partial image;

[0014] An object detection unit configured to detect an object included in each of the partial images; and

[0015] A detection result determination unit configured to, for an object detected within the boundary region, determine one of the detection results related to the object in the partial image as the detection result related to the object in the target image.

[0016] [2] The object detection device according to [1] above, wherein

[0017] the object detection unit is configured to generate the detection result including the confidence level of each object,

[0018] the detection result determination unit is configured to determine the detection result with the highest confidence level among the detection results related to the object detected within the boundary region as the detection result related to the object in the target image.

[0019] [3] The object detection device according to [2] above, wherein

[0020] the target image is an image obtained by photographing a plurality of particles in a dispersed manner,

[0021] the objects include normal particles photographed without overlapping with other particles and aggregated particles photographed in an overlapping manner with other particles.

[0022] [4] An object detection method, wherein the following steps are performed by a computer:

[0023] A step of segmenting a target image capturing a plurality of objects into a plurality of partial images, the plurality of partial images including a boundary region overlapping with an adjacent partial image;

[0024] A step of detecting an object included in each of the partial images; and

[0025] A step of, for an object detected within the boundary region, determining one of the detection results related to the object in the partial image as the detection result related to the object in the target image.

[0026] [5] A program that causes a computer to execute:

[0027] A step of dividing a target image that has captured a plurality of objects into a plurality of local images, the plurality of local images including a boundary region that overlaps with adjacent local images;

[0028] For each of the local images, a step of detecting an object included in the local image; and

[0029] A step of determining, for an object detected within the boundary region, one of the detection results of the local image related to the object as the detection result of the target image related to the object.

[0030] [Advantageous Effects]

[0031] According to one aspect of the present disclosure, it is possible to improve the accuracy of object detection for an image that has captured a large number of objects. Description of the Drawings

[0032] Figure 1 Figure 1 It is a schematic diagram of an example of a target image.

[0033] Figure 2 Figure 2 It is a schematic diagram of an example of a target image.

[0034] Figure 3 Figure 3 It is a block diagram of an example of the overall configuration of an object detection system.

[0035] Figure 4 Figure 4 It is a block diagram of an example of the hardware configuration of a computer.

[0036] Figure 5 Figure 5 It is a block diagram of an example of the functional configuration of an object detection system.

[0037] Figure 6 Figure 6 It is a flowchart of an example of the processing steps of an object detection method.

[0038] Figure 7 Figure 7 It is a conceptual diagram of an example of image segmentation processing of the prior art.

[0039] Figure 8 Figure 8 It is a conceptual diagram of an example of image segmentation processing of one embodiment.

[0040] Figure 9 ​​​​​​​​​​​​​​​​​​Figure 9 It is a schematic diagram of an example of a partial image.

[0041] Figure 10 Figure 10 It is a conceptual diagram showing an example of the detection result of a partial image.

[0042] Figure 11 Figure 11 It is a conceptual diagram showing an example of the detection result of a target image.

[0043] Figure 12 Figure 12 It is a conceptual diagram showing an example of the detection result of a partial image.

[0044] Figure 13 Figure 13 It is a conceptual diagram showing an example of the detection result of a target image. Detailed implementation manners

[0045] The following describes each implementation manner of the present disclosure with reference to the accompanying drawings. It should be noted that in this specification and the accompanying drawings, for components having substantially the same functional configurations, repeated descriptions are omitted by assigning the same reference numerals.

[0046] [Implementation manner]

[0047] One implementation manner of the present disclosure is an object detection system that detects an object from an image of a large number of objects. Hereinafter, the image to be the object of object detection is also referred to as a "target image (object image)" (note: "" is equivalent to ""). The target image of this implementation manner is an image obtained by photographing a state in which a large number of particles are dispersed. The particles of this implementation manner are inorganic particles containing an inorganic material as a constituent material.

[0048] Figure 1 and Figure 2 It is a schematic diagram of an example of the target image of this implementation manner. Figure 1 It is a schematic diagram of an example of the entire target image. As Figure 1 shown, in the target image 900 of this implementation manner, a large number of particles are photographed in a dispersed state. In the target image 900, for example, thousands of particles are photographed.

[0049] Figure 2 is a Figure 1 schematic diagram of an example of an image obtained by magnifying a part of the target image shown. As Figure 2 shown, the large number of particles photographed in the target image 900 include normal particles photographed in a non-overlapping manner with other particles and aggregated particles photographed in an overlapping manner with other particles. Figure 2 ​​​​​​​​Three aggregated particles 800-1 to 800-3 are shown. Particles other than the aggregated particles 800-1 to 800-3 among the particles captured in the target image 900 are normal particles.

[0050] The object detection system of the present embodiment performs object detection using a trained object detection model. The object detection model of the present embodiment is trained in a manner of detecting normal particles and aggregated particles from a target image. Specifically, the object detection model is trained using training data obtained by adding ground truth labels indicating regions of normal particles and regions of aggregated particles to a plurality of pre-collected images (hereinafter also referred to as "training images") respectively. The object detection model is trained in a manner that optimizes the error between the prediction result based on the training image and the ground truth label added to the training image.

[0051] Regarding a conventional object detection model, if object detection is performed on an image capturing a large number of objects, a huge amount of computing resources are required. For this reason, the target image can be divided into a plurality of local images. By performing object detection on local images with a smaller number of captured objects, the computing resources required for object detection can be reduced.

[0052] On the other hand, if the target image is divided into local images, there is a problem that objects at the boundaries of the local images are also divided. As is well known, since the objects divided at the boundaries are not captured as entire objects in the local images, it may lead to a decrease in detection accuracy or missed detection.

[0053] An object of the present embodiment is to realize an object detection system capable of performing object detection on a target image capturing a large number of objects with high accuracy.

[0054] <Overall Configuration of Object Detection System>

[0055] See Figure 3 The overall configuration of the object detection system of the present embodiment will be described. Figure 3 It is a block diagram showing an example of the overall configuration of the object detection system of the present embodiment.

[0056] As Figure 3 shown, the object detection system 1 of the present embodiment includes an image acquisition device 10, an object detection device 20, and a user terminal 30. The image acquisition device 10, the object detection device 20, and the user terminal 30 are connected via a communication network N1 such as a LAN (Local Area Network) or the Internet so as to enable data communication.

[0057] The image acquisition device 10 is an optical device that acquires a target image which is the object to be detected. The image acquisition device 10 can be a digital camera that captures still images, or a video camera that captures videos. As the image acquisition device 10, an optical microscope, a scanning electron microscope (SEM), a transmission electron microscope (TEM), etc. can be used according to the size (dimension) of the target object. In addition, the image acquisition device 10 can also be an information processing device such as a personal computer connected to various cameras (including video cameras), or an inspection device equipped (installed) with various cameras.

[0058] The user terminal 30 is an information processing terminal such as a personal computer, a tablet terminal, or a smartphone operated by the user. The user terminal 30 acquires the target image from the image acquisition device 10 and sends it to the object detection device 20. In addition, the user terminal 30 also receives the detection result from the object detection device 20 and outputs it to the user.

[0059] The object detection device 20 is an information processing device such as a personal computer, a workstation, or a server that detects an object from the target image obtained by the image acquisition device 10. The object detection device 20 receives the target image from the user terminal 30. In addition, the object detection device 20 also detects an object from the received target image and sends the detection result to the user terminal 30.

[0060] It should be noted that Figure 3 The overall configuration of the object detection system 1 shown is only an example, and various system configuration examples can also be provided according to the use and purpose. For example, the object detection device 20 can be implemented by multiple computers, or can be implemented as a cloud computing service. In addition, for example, the object detection system 1 can also be implemented by an independent (stand-alone) information processing device, which has the functions that the image acquisition device 10, the object detection device 20, and the user terminal 30 should have respectively.

[0061] <Hardware Configuration of the Object Detection System>

[0062] Refer to Figure 4 The hardware configuration of the object detection system 1 of this embodiment will be described.

[0063] "Hardware Configuration of the Computer"

[0064] The image acquisition device 10, the object detection device 20, and the user terminal 30 of this embodiment are implemented by a computer, for example. Figure 4 It is a block diagram showing an example of the hardware configuration of the computer 500 of this embodiment.

[0065] As Figure 4As shown, the computer 500 has a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, an HDD (Hard Disk Drive) 504, an input device 505, a display device 506, a communication I / F (Interface) 507, and an external I / F 508. The CPU 501, the ROM 502, and the RAM 503 form a so-called computer. Each hardware component of the computer 500 is interconnected via a bus 509. It should be noted that the input device 505 and the display device 506 can also be in a form (configuration) that is used by connecting to the external I / F 508.

[0066] The CPU 501 is an arithmetic unit that reads programs and data from storage devices such as the ROM 502 and the HDD 504 onto the RAM 503 and executes processing, thereby achieving the control and functions of the entire computer 500.

[0067] The ROM 502 is an example of a non-volatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off. The ROM 502 functions as a main storage device and stores various programs, data, etc. required for the CPU 501 to execute various programs installed in the HDD 504. Specifically, the ROM 502 stores boot programs such as the BIOS (Basic Input / Output System) and EFI (Extensible Firmware Interface) executed when the computer 500 starts up, data such as OS (Operating System) settings, and network settings.

[0068] The RAM 503 is an example of a volatile semiconductor memory (storage device) in which programs and data are deleted when the power is turned off. The RAM 503 is, for example, DRAM (Dynamic Random Access Memory), SRAM (Static Random Access Memory), etc. The RAM 503 provides a working area where these programs can be expanded when the CPU 501 executes various programs installed in the HDD 504.

[0069] The HDD504 is an example of a non-volatile storage device that stores programs and data. The programs and data stored in the HDD504 include the basic software (i.e., the OS) that controls the entire computer 500, applications that provide various functions on the OS, and the like. It should be noted that the computer 500 can also use a storage device (e.g., SSD: Solid State Drive, etc.) that uses flash memory as the storage medium instead of the HDD504.

[0070] The input device 505 is a touch screen, operation keys, buttons, keyboard, mouse, microphone for inputting voice data for inputting various signals by the user, and the like.

[0071] The display device 506 is composed of a display such as a liquid crystal or an organic EL (Electro-Luminescence) for displaying a screen, a speaker for outputting voice data for outputting voice and the like.

[0072] The communication I / F 507 is an interface for connecting to a communication network so that the computer 500 can be used for data communication.

[0073] The external I / F 508 is an interface with an external device. The external device includes a drive device 510 and the like.

[0074] The drive device 510 is a device for placing the storage medium 511. The storage medium 511 mentioned here includes media that store information optically, electrically, or magnetically, such as CD-ROMs, floppy disks, magneto-optical disks, etc. In addition, the storage medium 511 also includes semiconductor memories that store data electrically, such as ROMs and flash memories. Accordingly, the computer 500 can read from and / or write to the storage medium 511 via the external I / F 508.

[0075] It should be noted that various programs installed in the HDD504 can be installed, for example, in the following manner: the distributed storage medium 511 is placed in the drive device 510 connected to the external I / F 508, and the drive device 510 reads various programs stored in the storage medium 511. Alternatively, various programs installed in the HDD504 can also be installed by downloading them via the communication I / F 507 from a network different from the communication network.

[0076] <Functional Configuration of the Object Detection System>

[0077] See Figure 5 A description will be given of the functional configuration of the object detection system of the present embodiment. Figure 5 is a block diagram showing an example of the functional configuration of the object detection system 1 of the present embodiment.

[0078] 《Functional Configuration of the Image Acquisition Device》

[0079] As Figure 5 shown, the image acquisition device 10 of the present embodiment includes a shooting control unit 101 and an image storage unit 102.

[0080] The shooting control unit 101 is implemented by a camera connected to the Figure 4 shown external I / F 508. The image storage unit 102 is implemented by the Figure 4 shown HDD 504.

[0081] The shooting control unit 101 adjusts the shooting conditions (viewpoint, shooting magnification, etc.) of the camera to be able to shoot multiple objects, and shoots a target image. The shooting control unit 101 can shoot the target image based on the user's operation, or can shoot the target image when a preset condition is satisfied. The preset condition means, for example, when the inspection object is transported to a predetermined position in the inspection device, etc. The shooting control unit 101 can shoot a still image, or can shoot a moving image and extract an image having multiple objects therefrom.

[0082] The target image shot by the shooting control unit 101 is stored in the image storage unit 102. In the image storage unit 102, information related to the target image can also be stored in a manner associated with the target image. Information related to the target image is, for example, the shooting date and time, information indicating the shooting object, shooting conditions, etc.

[0083] "Functional Configuration of Object Detection Device"

[0084] As Figure 5 shown, the object detection device 20 of the present embodiment includes a model storage unit 200, an image input unit 201, an image segmentation unit 202, an object detection unit 203, a same object determination unit 204, a detection result determination unit 205, and a result output unit 206.

[0085] The image input unit 201, the image segmentation unit 202, the object detection unit 203, the same object determination unit 204, the detection result determination unit 205, and the result output unit 206 are implemented by processing in which the CPU 501 executes a program expanded from the Figure 4 shown HDD 504 to the RAM 503. The model storage unit 200 is implemented by the Figure 4 shown HDD 504.

[0086] A trained object detection model is stored in the model storage unit 200. The object detection model is a machine learning model that takes an image as input and outputs a detection result obtained by detecting an object from the image. The detection result includes: region information indicating the range of the region where the detected object is photographed, identification information for identifying the object photographed in each region, and the confidence level of the detection result related to each object.

[0087] The structure of the object detection model is, for example, a convolutional neural network (CNN: Convolutional Neural Network). The object detection model can adopt, for example, Faster R-CNN (Region-Based Convolutional Neural Networks), Mask R-CNN, etc.

[0088] The image input unit 201 receives the input of the target image by receiving the target image from the user terminal 30. The image input unit 201 can also receive the input of the target image by obtaining the target image from the image acquisition device 10 according to a request from the user terminal 30.

[0089] The image segmentation unit 202 divides the target image input to the image input unit 201 into a plurality of partial images. The image segmentation unit 202 performs the segmentation in such a way that the objects at the boundaries of the partial images are segmented and the segmented objects remain intact (i.e., there are no deficiencies). Specifically, the target image is divided into each partial image in such a way that the regions overlap with each other, so that at least one of the adjacent partial images contains the entire object. For this purpose, each partial image segmented by the image segmentation unit 202 may include a region that does not overlap with the adjacent partial image (hereinafter also referred to as the "central region") and a region that overlaps with the adjacent partial image (hereinafter also referred to as the "boundary region").

[0090] For each partial image segmented by the image segmentation unit 202, the object detection unit 203 detects the objects photographed in the partial image and obtains the confidence level of each detected object. The object detection unit 203 calculates the detection results of each partial image by inputting each partial image into the object detection model read from the model storage unit 200.

[0091] The same object determination unit 204 determines the same object among the objects detected in the boundary regions included in a plurality of adjacent partial images. The same object determination unit 204 respectively obtains the regions where objects are detected from the detection results of the plurality of adjacent partial images, and determines the regions where the same object is detected based on the similarity between the regions.

[0092] For the object determined by the same object determination unit 204, the detection result determination unit 205 determines one of the detection results of the partial image related to the object as the detection result of the target image related to the object. The detection result determination unit 205 determines the detection result with the highest confidence level among the detection results of the partial image related to the object as the detection result of the target image related to the object.

[0093] The result output unit 206 generates the detection result of the target image based on the detection results of the partial images. When generating the detection result of the target image, the result output unit 206 uses the detection result determined by the detection result determination unit 205 as the detection result related to the object detected within the boundary region. The result output unit 206 sends the generated detection result of the target image to the user terminal 30.

[0094] 《Functional Configuration of User Terminal 30》

[0095] As Figure 5 shown, the user terminal 30 of the present embodiment includes an image acquisition unit 301 and a result display unit 302.

[0096] The image acquisition unit 301 and the result display unit 302 are implemented by the CPU 501 executing processes based on a program expanded from the Figure 4 shown HDD 504 to the RAM 503.

[0097] The image acquisition unit 301 acquires the target image from the image acquisition device 10 according to the user's operation. The image acquisition unit 301 sends the acquired target image to the object detection device 20.

[0098] The result display unit 302 receives the detection result of the target image from the object detection device 20. The result display unit 302 displays the received detection result of the target image on the display device 506.

[0099] <Processing Steps of Object Detection System>

[0100] Refer to Figure 6 to describe the processing steps of the object detection method executed by the object detection system 1 of the present embodiment. Figure 6 is a flowchart showing an example of the processing steps of the object detection method of the present embodiment.

[0101] In step S1, the shooting control unit 101 of the image acquisition device 10 adjusts the viewing angle of the camera to be able to shoot a plurality of objects and shoots the target image. Then, the shooting control unit 101 saves the shot target image in the image storage unit 102.

[0102] In step S2, the image acquisition unit 301 of the user terminal 30 sends a request for acquiring the target image to the image acquisition device 10 according to the user's operation. The image acquisition device 10 reads the target image stored in the image storage unit 102 according to the acquisition request received from the user terminal 30 and sends it to the user terminal 30. The image acquisition unit 301 sends the target image received from the image acquisition device 10 to the object detection device 20.

[0103] In step S3, the image input unit 201 included in the object detection device 20 receives a target image from the user terminal 30. Then, the image input unit 201 sends the received target image to the image segmentation unit 202.

[0104] In step S4, the image segmentation unit 202 included in the object detection device 20 receives the target image from the image input unit 201. Then, the image segmentation unit 202 divides the received target image into a plurality of partial images. Then, the image segmentation unit 202 sends the plurality of partial images to the object detection unit 203.

[0105] See Figure 7 and Figure 8 A description will be given of the image segmentation process of the present embodiment.

[0106] Figure 7 is a schematic diagram of an example of an image segmentation process of the prior art. As Figure 7 shown, in the image segmentation process of the prior art, the target image 900 is divided into partial images 910-1 to 910-20 in such a way that they are equally spaced in the horizontal and vertical directions and do not overlap with each other. Figure 7 In the example of, the target image 900 is divided in such a way that the interval in the horizontal direction is W1 and the interval in the vertical direction is H1. For this reason, each of the partial images 910-1 to 910-20 is a rectangle having a width W1 and a height H1. It should be noted that the intervals W1 and H1 used to divide the target image 900 can be arbitrarily determined, and the number of partial images 910 can also be arbitrarily determined.

[0107] Figure 8 is a schematic diagram of an example of the image segmentation process of the present embodiment. As Figure 8 shown, in the image segmentation process of the present embodiment, the division is performed in the following manner, that is, the objects at the boundaries of the partial images are divided and the divided objects are kept complete (that is, there is no deficiency). Specifically, the target image is divided into partial images in such a way that they include overlapping regions, so that at least one of the adjacent partial images includes the entire object.

[0108] Figure 8 In the example of, a boundary region having a width of M1 is provided between the horizontally adjacent partial images, and a boundary region having a width of M2 is provided between the vertically adjacent partial images, whereby the target image 900 is divided into partial images 920-1 to 920-20. It should be noted that the widths M1 and M2 of the overlapping regions can be arbitrarily determined according to the sizes of the objects in the target image 900 and the intervals between the objects.

[0109] Figure 9 is a schematic diagram of an example of a partial image of the present embodiment. AsFigure 9 As shown, the partial image 920 of the present embodiment includes a central region 921 that does not overlap with adjacent partial images and a boundary region 922 that overlaps with adjacent partial images. The central region 921 is a rectangle with a width of (W1 - M1) and a height of (H1 - M2). The boundary region 922 is the region of the partial image 920 other than the central region 921.

[0110] Return Figure 6 A description will be given. In step S5, the object detection unit 203 of the object detection device 20 receives a plurality of partial images from the image segmentation unit 202. Then, the object detection unit 203 reads the trained object detection model from the model storage unit 200.

[0111] Next, the object detection unit 203 inputs the received plurality of partial images into the trained object detection model respectively, thereby detecting the objects captured in each partial image and obtaining the confidence of each detected object. For example, by using the Softmax function in the output layer of the object detection model, the confidence expressed as a percentage can be obtained. Here, for an object that is segmented at the edge of the partial image and causes a lack of the whole, it is not detected. The object detection unit 203 sends the detection results of each partial image output from the object detection model to the same object determination unit 204 and the result output unit 206.

[0112] In step S6, the same object determination unit 204 of the object detection device 20 receives the detection results of each partial image from the object detection unit 203. Then, the same object determination unit 204 determines the same object among the objects detected in the boundary regions included in adjacent partial images. After that, the same object determination unit 204 sends the information indicating the same object to the detection result determination unit 205.

[0113] Specifically, first, the same object determination unit 204 respectively obtains the regions where objects are detected from the detection results of adjacent partial images. Then, the same object determination unit 204 calculates the similarity between regions for all combinations of the region obtained from a certain partial image and the region obtained from an adjacent partial image.

[0114] The similarity of the present embodiment can use, for example, the IoU (Intersection over Union) evaluation index. The IoU evaluation index is an index indicating the overlapping degree of two regions. The IoU evaluation index is the value obtained by dividing the intersection of two regions by the union of these two regions. The value of the IoU evaluation index is 0 or more and 1 or less, and the closer the value is to 1, the more similar the two regions are. However, the similarity between regions is not limited to the IoU evaluation index, and any index capable of calculating the similarity of images can be used.

[0115] Next, the same object determination unit 204 determines combinations of regions whose calculated similarity is equal to or greater than a preset threshold. After that, the same object determination unit 204 outputs the determined combinations of regions as information indicating the same object.

[0116] In step S7, the detection result determination unit 205 included in the object detection device 20 acquires information indicating the same object from the same object determination unit 204. Next, the detection result determination unit 205 acquires the detection result of the partial image in which the object is photographed based on the information indicating the same object. Next, the detection result determination unit 205 determines the detection result with the highest confidence among the detection results related to the object obtained from the acquired partial image. After that, the detection result determination unit 205 sends the determined detection result to the result output unit 206 as the detection result related to the object detected in the boundary region.

[0117] See Figures 10 to 13 A description will be given of the detection result determination process of the present embodiment.

[0118] Figure 10 It is a conceptual diagram showing an example of the detection result of a partial image. Figure 10 It is an example of the detection result of an object detection model that identifies the region where the object is photographed in units of pixels, and shows partial images of two points adjacent in the vertical direction in the figure.

[0119] Figure 10 In, the black circle and the white circle indicate particles of different types detected. Here, if Figure 2 the normal particles and aggregated particles exemplified in are used for explanation, the black circle indicates normal particles and the white circle indicates aggregated particles. The percentage inside the circle indicates the confidence of the detection result represented by the circle. It should be noted that Figure 10 in the example of, although only the confidence of the object detected in the boundary region is shown, in the actual detection result, the confidence of all objects is calculated.

[0120] As Figure 10 shown, the object 811-n detected from the partial image 920-n and the object 811-m detected from the partial image 920-m are detection results indicating particles of different types. Similarly, the object 812-n detected from the partial image 920-n and the object 812-m detected from the partial image 920-m are also detection results indicating particles of different types.

[0121] Figure 11 It is a conceptual diagram showing an example of the detection result of a target image. Figure 11 It is byFigure 10 An example of a conceptual target image generated by superimposing the same object determined by the same object determination unit 204 on a partial image of two points shown, and its detection result.

[0122] By Figure 10 and Figure 11 Comparing, it can be seen that the object 811-n detected from the partial image 920-n and the object 811-m detected from the partial image 920-m are the same object 811, and the object 812-n detected from the partial image 920-n and the object 812-m detected from the partial image 920-m are the same object 812. Here, Figure 10 Different classification results for the same object are shown in the normal particles of the object 811-n and the aggregated particles of the object 811-m shown. Similarly, Figure 10 Different classification results for the same object are also shown in the aggregated particles of the object 812-n and the normal particles of the object 812-m shown. That is, different classification results are obtained for the objects detected within the boundary region.

[0123] In the present embodiment, the detection result determination unit 205 determines the detection result with the highest confidence in the detection results related to the objects detected within the boundary region as the detection result related to the object. Specifically, as Figure 11 shown, the object 811 is determined as normal particles, and the object 812 is also determined as normal particles.

[0124] Figure 12 is a conceptual diagram showing an example of the detection result of a partial image. Figure 12 is an example of the detection result of an object detection model that identifies the position where an object is photographed within a rectangular region, showing partial images of two points adjacent in the up-down direction in the figure.

[0125] Figure 12 In, the solid-line rectangle and the dashed-line rectangle indicate that particles of different types are detected. Here, if the normal particles and aggregated particles exemplified in Figure 2 are used for explanation, the solid-line rectangle indicates normal particles, and the dashed-line rectangle indicates aggregated particles. The percentage within the rectangle indicates the confidence of the detection result represented by the rectangle. It should be noted that, Figure 12 in the example of

[0126] As Figure 12As shown, the object 811-n detected from the partial image 920-n and the object 811-m detected from the partial image 920-m are detection results representing different types of particles. Similarly, the object 812-n detected from the partial image 920-n and the object 812-m detected from the partial image 920-m are also detection results representing different types of particles.

[0127] Figure 13 It is a conceptual diagram showing an example of the detection result of the target image. Figure 13 It is Figure 12 An example of a conceptual target image generated by superimposing the same object determined by the same object determination unit 204 on the partial images at the two points shown and its detection result.

[0128] By comparing Figure 12 and Figure 13 it can be seen that the object 811-n detected from the partial image 920-n and the object 811-m detected from the partial image 920-m are the same object 811, and the object 812-n detected from the partial image 920-n and the object 812-m detected from the partial image 920-m are the same object 812. Here, Figure 12 In the normal particles of the object 811-n and the aggregated particles of the object 811-m shown, different classification results for the same object are shown. Similarly, Figure 12 In the aggregated particles of the object 812-n and the normal particles of the object 812-m shown, different classification results for the same object are also shown. That is, different classification results are obtained for the objects detected within the boundary region.

[0129] In the present embodiment, the detection result determination unit 205 determines the detection result with the highest confidence in the detection results related to the objects detected within the boundary region as the detection result related to the object. Specifically, as Figure 13 shown, the object 811 is determined to be normal particles, and the object 812 is also determined to be normal particles.

[0130] Return Figure 6 A description will be given. In step S8, the result output unit 206 provided in the object detection device 20 receives the detection results of the respective partial images from the object detection unit 203. Then, the result output unit 206 receives the detection results related to the objects detected within the boundary region from the detection result determination unit 205.

[0131] Next, the result output unit 206 generates a detection result of the target image based on the detection results of the local images. At this time, the result output unit 206 uses the detection result determined by the detection result determination unit 205 as the detection result related to the object detected within the boundary region. The result output unit 206 sends the generated detection result of the target image to the user terminal 30.

[0132] In step S9, the result display unit 302 included in the user terminal 30 receives the detection result of the target image from the object detection device 20. The result display unit 302 displays the received detection result of the target image on the display device 506.

[0133] <Effect of the Embodiment>

[0134] The object detection device 20 of the present embodiment divides a target image capturing a plurality of objects into a plurality of local images, and performs object detection for each local image. After that, for an object detected within the boundary region, one of the detection results of the local image related to the object is determined as the detection result of the target image related to the object. Since the local image of the present embodiment includes a boundary region that overlaps with an adjacent local image, an object at the boundary of the local image will not be divided. In addition, the detection result of the present embodiment includes the detection result related to the object detected within the boundary region. Therefore, according to the object detection device 20 of the present embodiment, the accuracy of object detection for an image capturing a large number of objects can be improved.

[0135] The object detection device 20 of the present embodiment generates a detection result including the confidence level of each object, and uses the detection result with the highest confidence level among the detection results related to the objects detected within the boundary region as the detection result related to the object. Therefore, according to the object detection device 20 of the present embodiment, the accuracy of object detection for an image capturing a large number of objects can be further improved.

[0136] The object detection device 20 of the present embodiment detects normal particles captured in a non-overlapping manner with other particles and aggregated particles captured in an overlapping manner with other particles from a target image in which a plurality of particles are captured in a dispersed manner. Therefore, according to the object detection device 20 of the present embodiment, normal particles and aggregated particles can be detected with high accuracy from an image in which a large number of particles are captured in a dispersed manner.

[0137] [Supplementary]

[0138] Each function of the above-described embodiments can be implemented by one or more processing circuits. Here, the "processing circuit" in this specification includes a processor programmed by software to execute each function, such as a processor implemented by an electronic circuit, and devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and a conventional circuit module designed to execute the above-described respective functions.

[0139] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to these embodiments, and various modifications or changes can be made within the scope of the gist of the present invention described in the claims.

[0140] This application claims the priority of Japanese Patent Application No. 2022-191312 filed with the Japan Patent Office on November 30, 2022, and incorporates its content in its entirety herein.

[0141] [Description of Reference Numerals]

[0142] 1 Object detection system

[0143] 10 Image acquisition device

[0144] 101 Shooting control unit

[0145] 102 Image storage unit

[0146] 20 Object detection device

[0147] 200 Model storage unit

[0148] 201 Image input unit

[0149] 202 Image segmentation unit

[0150] 203 Object detection unit

[0151] 204 Identical object determination unit

[0152] 205 Detection result determination unit

[0153] 206 Result output unit

[0154] 30 User terminal

[0155] 301 Image acquisition unit

[0156] 302 Result display unit.

Claims

1. An object detection device, comprising: an image segmentation unit configured to segment a target image capturing a plurality of objects into a plurality of partial images, the plurality of partial images including a boundary region overlapping with an adjacent partial image; an object detection unit configured to detect an object included in each of the partial images; and a detection result determination unit configured to determine, for an object detected within the boundary region, one of the detection results related to the object in the partial image as the detection result related to the object in the target image.

2. The object detection device according to claim 1, wherein the object detection unit is configured to generate the detection result including a confidence level for each of the objects, the detection result determination unit is configured to determine, as the detection result related to the object in the target image, the detection result with the highest confidence level among the detection results related to the object detected within the boundary region.

3. The object detection device according to claim 2, wherein the target image is an image obtained by photographing a plurality of particles in a dispersed manner, the objects include normal particles photographed without overlapping with other particles and aggregated particles photographed with overlapping other particles.

4. An object detection method, wherein, Performed by a computer: the step of segmenting a target image capturing a plurality of objects into a plurality of partial images, the plurality of partial images including a boundary region overlapping with an adjacent partial image; the step of detecting an object included in each of the partial images; and the step of determining, for an object detected within the boundary region, one of the detection results related to the object in the partial image as the detection result related to the object in the target image.

5. A program that causes a computer to perform: the step of segmenting a target image capturing a plurality of objects into a plurality of partial images, the plurality of partial images including a boundary region overlapping with an adjacent partial image; the step of detecting an object included in each of the partial images; and the step of determining, for an object detected within the boundary region, one of the detection results related to the object in the partial image as the detection result related to the object in the target image.

Images