INFERENCE DEVICE, OBJECT COUNTING METHOD, AND PROGRAM

JP7593616B2Active Publication Date: 2024-12-03NTT DOCOMO BUSINESS INC
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2020192531
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2020-11-19
Publication Date
2024-12-03
Estimated Expiration
2040-11-19

AI Technical Summary

Benefits of technology

【0008】 開示の技術によれば、低コストで高精度に物体カウントを行うことを可能とする技術が提供される。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007593616000001
    Figure 0007593616000001
  • Figure 0007593616000002
    Figure 0007593616000002
  • Figure 0007593616000003
    Figure 0007593616000003
Patent Text Reader

Abstract

To provide a technique that enables high-precision object counting at low cost.SOLUTION: An inference device that counts objects passing through a predetermined line from a video capturing the objects, includes a detection unit configured to detect, from the video, a target object to be counted when it passes through the line, and a count unit configured to increase the count value when it is detected that the target object has passed through the line and moved from a first area to a second area. A line neighborhood area is set in the vicinity of the line on the video, and the line neighborhood area does not belong to each of the first area and the second area.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a technique for counting objects that pass through a predetermined line among objects captured in video data. [Background technology]

[0002] There is object counting technology that detects objects (such as humans) from video data and counts the number of objects passing through a line drawn on the boundary between the inside and outside of a facility.

[0003] Counting objects that pass a predetermined line among those captured in video data allows us to grasp the number of objects present in a given space, which can be applied to judging the degree of congestion in a facility and for marketing purposes. Considering the recent COVID-19 pandemic and the need to shift to new normal work styles and lifestyles, the industrial applicability of object counting technology is extremely high. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2012-242947 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, when counting objects using conventional techniques such as optical flow calculations or general-purpose object detectors, there are issues such as high training costs and low counting accuracy.

[0006] The present invention has been made in consideration of the above-mentioned points, and has an object to provide a technique that enables object counting to be performed with high accuracy at low cost. [Means for solving the problem]

[0007] According to the disclosed technology, there is provided an inference device that counts objects passing through a predetermined line from an image of the objects, the inference device comprising: a detection unit that detects, from the video, a target object that is to be counted for passing through the line; a counting unit that increases a count value when it is detected that the target object has passed through the line and moved from the first area to the second area, A line vicinity region is set in the vicinity of the line on the image, and the line vicinity region does not belong to either the first region or the second region. figure, The counting unit is storing the first area or the second area in a storage unit as an area in which the target object is located, and when the area in which the target object is located is the line vicinity area, not updating the area stored in the storage unit; When the area stored in the storage unit is the first area and the area in which the target object is located becomes the second area, the count value is increased. A reasoning apparatus is provided. Effect of the Invention

[0008] The disclosed technology provides a technology that enables object counting to be performed with high accuracy at low cost. [Brief description of the drawings]

[0009] [Figure 1] FIG. 13 is a diagram illustrating an example of object counting. [Diagram 2] FIG. 13 illustrates a case where counting fails. [Diagram 3] FIG. 1 is a configuration diagram of an inference device according to a first embodiment. [Figure 4] FIG. 2 is a configuration diagram of a learning device according to the first embodiment. [Diagram 5] FIG. 2 is a diagram for explaining the operation of the inference device of the first embodiment. [Figure 6] FIG. 2 is a diagram for explaining each region. [Figure 7] 11A and 11B are diagrams for explaining an example of a relationship between a line vicinity region and a line. [Figure 8] FIG. 13 is a diagram for explaining a counting method. [Figure 9] FIG. 2 is a diagram for explaining the operation of the learning device of the first embodiment. [Figure 10] FIG. 11 is a configuration diagram of an inference device according to a second embodiment. [Figure 11] FIG. 11 is a diagram for explaining the operation of the inference device of the second embodiment. [Figure 12] FIG. 2 illustrates an example of a hardware configuration of the apparatus. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention (the present embodiment) will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment.

[0011] (Outline of this embodiment) In this embodiment, an inference device 100 (and an inference device 300) described below counts objects that pass through a predetermined line among one or more objects captured in video data. Fig. 1 shows an example of object counting.

[0012] In the example in Figure 1, the objects to be counted are people, and a line is drawn on the boundary between the inside and outside of the facility. On the left side of Figure 1, an object passes through the line from the outside of the facility to the inside, so the number of objects entering the facility is incremented by 1 (represented as "In+1"). On the right side of Figure 1, an object passes through the line from the inside of the facility to the outside, so the number of objects leaving the facility is incremented by 1 (represented as "Out+1").

[0013] (About the assignment) To perform object counting, for example, a method (referred to as Method 1) can be used in which moving object regions are extracted from an input image based on a velocity field on a line obtained by optical flow calculation, which is a conventional technology, and the number of target objects contained in each of the obtained regions is counted.

[0014] In this method 1, when counting the number of target objects from a moving object region, training data with ground truth data is used to learn a regression model that inputs appearance feature regions related to colors, edges, etc. extracted from the moving object region and outputs the number of objects. However, the appearance of a moving object region changes significantly depending on the tilt angle at which the imaging device captures the target region or objects. Therefore, in this method 1, it is necessary to prepare and acquire training data for each tilt angle of the camera, which increases the cost.

[0015] Another possible method for counting objects is to detect the target object from each image frame, extract the trajectory of each object by matching the detection results between image frames, and count by determining the position of each object at each time (method 2).In method 2, it is assumed that the passing of the line is determined by simply comparing the position of the object with the position of the line.

[0016] In this method 2, it is possible to use a general-purpose object detector, such as open source or commercial software. Method 2 solves the problem of the high cost of acquiring and preparing training data, which is present in method 1. However, method 2, which simply compares the object position with the line position, has a problem in that the counting results are easily affected by deterioration in the position detection accuracy of the object detection process.

[0017] An example is shown in Figure 2. In this example, three image frames are shown: left, center, and right. Also shown are the In and Out correct answers and the predictions of Method 2 for each of the three image frames: left, center, and right.

[0018] As shown in Figure 2, the object indicated by A goes outside the line in the center image frame (Out:0 → Out:1) and then continues to move outside the line. However, the object detection result in the right image frame is inaccurate, and the estimated object position of A differs from the actual one, resulting in a miscount of it entering the line (In:0 → In:1).

[0019] Reasons for the decrease in accuracy of object position detection include low image quality or resolution, and occlusion caused by multiple objects being crowded together, which are very common. Therefore, Method 2 has the problem of low counting accuracy in many situations.

[0020] That is, in the methods 1 and 2, there are problems in that the cost of preparing and acquiring training data is high, or the counting accuracy is low. Below, a technique according to the present embodiment, which has low cost and high counting accuracy, will be described. Below, as the technique according to the present embodiment, examples 1 and 2 will be described.

[0021] Example 1 <Configuration of inference device 100> Fig. 3 shows an example of the configuration of the inference device 100 in the first embodiment. As shown in Fig. 3, the inference device 100 has a detection unit 110, a tracking unit 120, a counting unit 130, a video data storage unit 140, a line information storage unit 150, and a line neighborhood buffer size storage unit 160. Note that the functional unit that combines the detection unit 110 and the tracking unit 120 may be referred to as a "detection unit." The functions of each unit are outlined below.

[0022] Time-series image frames are stored as video data in the video data storage unit 140. Real-time video data may be sequentially accumulated in the video data storage unit 140, and object counting may be performed in real time by processing the video data, or past video data may be accumulated in the video data storage unit 140, and object counting may be performed at a past point in time by processing the video data.

[0023] Information on lines that are the reference for object counting is stored in the line information storage unit 150. A line neighborhood buffer size obtained by the learning device 200, which will be described later, is stored in the line neighborhood buffer size storage unit 160. As the line neighborhood buffer size to be stored in the line neighborhood buffer size storage unit 160, a line neighborhood buffer size determined by a system administrator or the like may be used without using the learning device 200.

[0024] 3 shows an example in which each storage unit is provided within inference device 100, but this is just one example. Any one or more or all of video data storage unit 140, line information storage unit 150, and line neighborhood buffer size storage unit 160 may be provided outside inference device 100. When a storage unit is provided outside, for example, inference device 100 accesses the storage unit via a network to obtain information.

[0025] The detection unit 110 detects a target object from each image frame constituting the video data read from the video data storage unit 140. The "target object" is an object that is the subject of line passage determination.

[0026] The tracking unit 120 receives the image frame at the current time, the detection result at the current time, and the past tracking results, and outputs the tracking result of the object at the current time. Note that the "current time" is the time of the latest image frame that is the target of the detection and tracking process.

[0027] The counting unit 130 receives the tracking results at each time, determines whether each tracking result is inside or outside the line, and performs object counting. More specifically, the counting unit 130 classifies the area in which the object is located according to each tracking result at each time into either an area divided by a line or unknown, based on the object's position, line information, and line vicinity buffer size, and counts the object when the area in which the object is located changes from one area to another except for unknown. In other words, when the object is located in the area near the line, the area in which the object is located is not explicitly specified. This makes it possible to perform robust counting even if the position of the object detected near the line is inaccurate.

[0028] This configuration of the inference device 100 makes it possible to count objects at low cost without the need to prepare and acquire training data for each tilt angle of the camera. In addition, the use of a line neighborhood buffer allows for highly accurate counting.

[0029] <Configuration of learning device 200> Fig. 4 shows an example of the configuration of the learning device 200. As shown in Fig. 4, the learning device 200 has a detection unit 210, a buffer size calculation unit 220, a video data storage unit 230, and a line information storage unit 240. The functions of each unit are outlined below.

[0030] Video data storage unit 230 stores a time series of image frames as video data. Line information storage unit 240 stores information on lines that serve as a reference for object counting. The images used by learning device 200 and inference device 100 may be the same or different. However, it is desirable that the locations and arrangements of cameras that capture images be consistent between learning device 200 and inference device 100.

[0031] 4 shows an example in which each storage unit is provided inside learning device 200, but this is just one example. Any one or more of video data storage unit 230 and line information storage unit 240 may be provided outside learning device 200. When a storage unit is provided outside, for example, learning device 200 accesses the storage unit via a network to acquire information.

[0032] The detection unit 210 detects an object from each of a predetermined number of image frames. The buffer size calculation unit 220 calculates and outputs a line neighborhood buffer size based on the object detection result by the detection unit 210.

[0033] Here, the line neighborhood buffer size is a parameter that determines the size of the line neighborhood area, and the larger the area, the more difficult it is to classify the area in which the object is located near the line. The buffer size calculation unit 220 adaptively determines the line neighborhood buffer size according to the position and size of the object detected by the detection unit 210, making it possible to automatically determine the line neighborhood buffer size suitable for the size of the object reflected in the video, reducing the cost of tuning and enabling accurate counting.

[0034] <Details of the operation of the inference device 100> As described above, image frames in a time series are stored in the video data storage unit 140 of the inference device 100, and the detection unit 110 (as well as the tracking unit 120 and counting unit 130) processes each image frame read out from the video data storage unit 140. An image of processing the image frames at each time is shown in FIG. 5. As shown in FIG. 5, processing is performed on the image frames at each time, starting from the image frame at time t=0. The operation of the inference device 100 will be described in detail below with reference to FIGS. 5 to 8.

[0035] <About line information, line neighborhood buffer size, and area> First, the line information, line neighborhood buffer size, and area in this embodiment will be described.

[0036] In this embodiment, one piece of line information is defined by, for example, the (x, y) coordinates of both ends of the line in the image. In this embodiment, only one line is defined in the image, but the technology according to the present invention can be similarly applied to the case where there are multiple lines.

[0037] In this embodiment, based on the line information and the line vicinity buffer size, as shown in FIG. 6, the area in the image is divided into an In area, an Out area, a line vicinity area, and a non-target area.

[0038] In the example of Fig. 6, the In area is the lower area of ​​the area defined by the normal line of the line passing through both end points of the line. "Area defined by the normal line of the line" means that an area exists in a direction perpendicular to the line. However, this is just one example of the definition of an area, and is not limited to this.

[0039] 6, the Out area is the upper area of ​​the area defined by the normal line of the line passing through both end points of the line. Which side of the line is to be the In area / Out area is determined in advance.

[0040] The line neighborhood region is a region near the line that is defined by the normal of the line passing through the line end point, and the size of the region is defined by the line neighborhood buffer size d. In the example of Figure 6, the region with a width of d in the direction perpendicular to the line and a length (length in the direction parallel to the line) of the line is defined as the line neighborhood region.

[0041] The non-target region is a region that is not an In region, an Out region, or a region near a line. The In region and the Out region may be defined in other ways. For example, the region other than the region near the line in the lower part of the region defined by the line information may be defined as the In region, and the region other than the region near the line in the upper part of the region defined by the line information may be defined as the Out region.

[0042] As is clear from the above description of the line neighborhood buffer size, the line neighborhood buffer size is defined as one-dimensional data, and is used in combination with line information to define the line neighborhood area. In this embodiment, as shown in Fig. 6, the line neighborhood area is defined as a rectangle having the line as its center line and defined by the line width and the line neighborhood buffer size, but the method of defining the line neighborhood area is not limited to this.

[0043] In the example of Fig. 6, the length from the line to the Out area side is 0.5d, and the length from the line to the In area side is 0.5d in the line vicinity area, and the line vicinity area is symmetrical with respect to the line. However, this is only an example, and the line vicinity area may be asymmetrical with respect to the line, for example, as shown in cases A and B in Fig. 7.

[0044] 5, inference device 100 outputs object count results for each time in the video based on the video data, line information, and line neighborhood buffer size. Here, the count results for each time in the video consist of the number of target objects that have moved from the In area to the Out area and the number of target objects that have moved from the Out area to the In area up to each time, counting from the start of the video.

[0045] <Detection unit 110> Next, the detection unit 110 will be described. The detection unit 110 receives image frames at each time that constitute a video image, and outputs the position of a target object that appears in the image. In this embodiment, the position of the target object is output as position information of a rectangle that exactly surrounds the target object. One example of a method for expressing a target object is to represent the target object as a rectangle that completely surrounds the object. In addition to a rectangle, the target object and its position may be represented by mask information that is composed of a set of pixel positions that capture the target object.

[0046] The method of detecting a target object from a video is itself a conventional technique, and can be, for example, the techniques disclosed in References 1 and 2. The names of the references are listed at the end of the specification.

[0047] <Tracking Unit 120> Next, a description will be given of the tracking unit 120. The tracking unit 120 receives an image frame at the current time, a target object detection result at the current time output from the detection unit 120, and a past tracking result (specifically, a tracking result at the immediately previous time), and outputs a tracking result at the current time.

[0048] Here, the tracking result is composed of an ID assigned to each target object captured in the video and the position of the target object in the current image frame. The tracking process itself performed by the detection unit 120 is a conventional technique, and the techniques disclosed in, for example, Reference 3 and Reference 4 can be used.

[0049] When using the technology disclosed in Reference 3, the tracking unit 120 matches objects between image frames based only on the positional proximity of the detection results output from the detection unit 110, whereas when using the technology disclosed in Reference 4, the tracking unit 120 also takes into account appearance information extracted from the target object region in the image frame, and matches past tracking results with objects in the current image frame. The technology in Reference 4 that takes appearance information into account generally has higher tracking performance. On the other hand, in terms of processing speed, the technology in Reference 3 that tracks based only on position is faster, and there is a trade-off between the two.

[0050] <Counting section 130> Next, the counting unit 130 will be described. The counting unit 130 receives the tracking results output by the tracking unit 120 as input, and outputs the counting results at each time in the video. The counting procedure executed by the counting unit 130 is as follows. The counting unit 130 includes a position storage unit that stores, for each target object, the area in which it is located (In area or Out area).

[0051] Based on the position of each tracked object at the current time, the counting unit 130 classifies the area in which each object is located into one of the In area, Out area, area near the line, and non-target area. Any method of classification may be used, and for example, it may be determined in which of the above areas the representative point of a rectangle corresponding to the object (e.g., the x, y coordinates of the center of the bottom side of the rectangle) is included.

[0052] Next, when it is determined that the area stored in the position storage unit up to the present time is the In area and the position at the current time is the Out area, the count unit 130 increments the count value indicating that the object has moved from the In area to the Out area by 1 (adds 1 to the count value). Similarly, when it is determined that the area stored in the position storage unit up to the present time is the Out area and the position at the current time is the In area, the count unit 130 increments the count value indicating that the object has moved from the Out area to the In area by 1.

[0053] Finally, when the current position of the target object is in the In area or the Out area, counting unit 130 updates the area in which the target object is located that is stored in the position storage unit to the area in which the current target object is located.

[0054] More specifically, if for a certain target object, the In area is stored in the position memory unit at time t=k, then if the area determination result at time t=k+1 is either the In area, the area near the line, or a non-target area, the stored information in the position memory unit remains the In area, and if the area determination result at time t=k+1 is the Out area, the stored information in the position memory unit is updated to the Out area.

[0055] Similarly, for a certain target object, if the Out area is stored in the position memory unit at time t=k, if the area determination result at time t=k+1 is either the Out area, the area near the line, or the non-target area, the stored information in the position memory unit will remain the Out area, and if the area determination result at time t=k+1 is the In area, the stored information in the position memory unit will be updated to the In area.

[0056] An example is shown in Figure 8. As shown by the trajectory of the target object (points connected by arrows), the target object is located in the In region at times t = 1 and t = 2, and at these times, the In region is stored in the position memory unit as the region in which the target object exists. At times t = 3 and t = 4, the target object is located in the region near the line, so the region stored in the position memory unit is not updated and remains the In region.

[0057] At time t=5, the target object reaches the Out area. At this time, the area in which the target object exists changes from the In area to the Out area, so counting unit 130 increments the count value of the movement of the target object from the In area to the Out area by 1, and updates the area stored in the position storage unit from the In area to the Out area.

[0058] The counting unit 130 performs the above processing on the tracking results of all target objects, and outputs a count value indicating the total number of movements from the In area to the Out area and a count value indicating the total number of movements from the Out area to the In area as the counting results at the current time.

[0059] <Operation details of learning device 200> Next, a detailed description will be given of the learning device 200. As described above, the video data storage unit 230 of the learning device 200 stores image frames in a time series, and the detection unit 210 performs processing for each image frame read out from the video data storage unit 230. An image of processing the image frames at each time is shown in Fig. 9.

[0060] The operation of detection unit 210 is the same as the operation of detection unit 110 in inference device 100. Below, the operation of buffer size calculation unit 220 in learning device 200 will be described in detail.

[0061] <Buffer Size Calculation Unit 220> The buffer size calculation unit 220 calculates the line neighborhood buffer size using the object detection results obtained by the detection unit 210 for a predetermined number of image frames as input. The buffer size calculation unit 220 adaptively determines the line neighborhood buffer size according to the size and position of the object detected by the detection unit 210.

[0062] The method of calculating the line neighborhood buffer size based on the object detection result is not limited to a specific method, but for example, the buffer size calculation unit 220 may calculate the line neighborhood buffer size so as to be proportional to the average size of the target objects detected by the detection unit 210. The size of the target object here can be defined by, for example, the height of a rectangle obtained as the detection result, or the length of the diagonal of the rectangle.

[0063] For example, if the target objects passing through the line are mainly humans, the average size (height) of the target objects can be set to, for example, 165 cm, and the buffer size calculation unit 220 can set the height on the image of a 165 cm human being appearing in the image at the line position x a constant as the line vicinity buffer size.

[0064] Alternatively, the buffer size calculation unit 220 may indirectly calculate the depth of the line representative point from the camera based on the object detection result, and determine the line neighborhood buffer size to be proportional to the calculated value. An example of a specific method of determination is as follows.

[0065] The set of detected target objects detected by the detection unit 210 and input to the buffer size calculation unit 220 is defined as {(x i ,y i ,w i ,h i )} N i=1 Here, (x i ,y i) is a representative point in a rectangle that represents a target object on an image. This point may be, for example, the center of gravity of the rectangle, a vertex of the rectangle, a point on a side of the rectangle (for example, the center of a side), or any other point. i is the width of the rectangle, and h i is the height of the rectangle. N is the number of objects. w i and i are examples of the size of the target object.

[0066] Here, it is assumed that the line and the detected target objects are on the same plane, and the heights of the detected multiple target objects are approximately constant. For example, assume that a line is drawn on the floor (or ground) at the boundary of the entrance and exit of a facility, and an object (here, a human being) walks on the floor (or ground) to enter and exit the facility.

[0067] In this case, a, b, and c are each one-dimensional parameters, and for all object detection results, h i =ax i +by i +c (formula 1) The buffer size calculation unit 220 uses the object detection set to find the parameters a, b, and c in the formula 1 by using an optimization method such as the least squares method.

[0068] Here, the representative x and y coordinates of the line are (x l ,y l ), where d0 is a constant, the buffer size calculation unit 220 calculates the line neighborhood buffer size d as follows: d=d0(ax l +by l +c) (Formula 2) It can be calculated as:

[0069] In the above example, the heights of the detected multiple target objects are approximately the same. If there are multiple types of target objects, d may be calculated for each type.

[0070] For example, assume that there is a mixture of children and adults. The buffer size calculation unit 220 calculates the line neighborhood buffer size d1 from a group of adults as target objects by the above-mentioned method, calculates the line neighborhood buffer size d2 from a group of children as target objects by the above-mentioned method, and outputs d1 and d2.

[0071] In this case, detector 110 of inference apparatus 100 determines whether a detected object is an adult or a child. Counter 130 uses d1 as the line neighborhood buffer size for adult objects and d2 as the line neighborhood buffer size for child objects.

[0072] Example 2 Next, a description will be given of Example 2. In Example 1, the calculation of the line neighborhood buffer size by the learning device 200 and the counting process by the inference device 100 are independent. On the other hand, in Example 2, while the counting process is performed on the input video data, the line neighborhood buffer size is successively updated based on the detection result of the target object, and the counting process is performed using the result.

[0073] Fig. 10 shows an example of the configuration of an inference device 300 in Example 2. As shown in Fig. 10, the inference device 300 has a detection unit 110, a tracking unit 120, a counting unit 130, a video data storage unit 140, a line information storage unit 150, and a buffer size update unit 310. The buffer size update unit 310 may be referred to as a buffer size calculation unit.

[0074] 10 shows an example in which each storage unit is provided inside inference device 300, but this is just one example. Any one or more of video data storage unit 140 and line information storage unit 150 may be provided outside inference device 300. When a storage unit is provided outside, for example, inference device 300 accesses the storage unit via a network to obtain information.

[0075] The detection unit 110, the tracking unit 120, and the counting unit 130 in the second embodiment perform the same processes as the detection unit 110, the tracking unit 120, and the counting unit 130 in the first embodiment, respectively.

[0076] The video data storage unit 140 stores image frames in a time series, and the detection unit 110 (as well as the tracking unit 120, counting unit 130, and buffer size update unit 310) processes each image frame read out from the video data storage unit 140. An image of processing the image frames at each time is shown in FIG.

[0077] In the second embodiment, the buffer size update unit 310 receives the object detection results from a certain past time to the current time, and performs the same processing as the buffer size calculation unit 220 described in the first embodiment to calculate the line vicinity buffer size at the current time, and inputs the line vicinity buffer size to the counting unit 130.

[0078] Here, the object detection results used to update the line vicinity buffer size from a certain time in the past to the current time may be all currently and past detection results stored, or detection results obtained from up to a predetermined number of past image frames may be used.

[0079] (Hardware configuration example) Inference device 100, learning device 200, and inference device 300 can all be realized, for example, by having a computer execute a program. This computer may be a physical computer or a virtual machine on the cloud. Note that, hereinafter, inference device 100, learning device 200, and inference device 300 are collectively referred to as "devices."

[0080] That is, the device can be realized by executing a program corresponding to the processing performed by the device, using hardware resources such as a CPU and memory built into a computer. The program can be recorded on a computer-readable recording medium (such as a portable memory) and stored or distributed. The program can also be provided via a network such as the Internet or email.

[0081] Fig. 12 is a diagram showing an example of the hardware configuration of the computer. The computer in Fig. 12 has a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, and the like, all of which are connected to one another via a bus BS. Note that some of these devices may not be included. For example, if no display is performed, the display device 1006 may not be included.

[0082] The program for realizing the processing on the computer is provided by a recording medium 1001 such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.

[0083] The memory device 1003 reads out and stores the program from the auxiliary storage device 1002 when an instruction to start the program is received. The CPU 1004 realizes the functions related to the device in accordance with the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network, and functions as a transmitting unit and a receiving unit. The display device 1006 displays a GUI (Graphical User Interface) or the like according to a program. The input device 1007 is composed of a keyboard and mouse, buttons, a touch panel, or the like, and is used to input various operation instructions. The output device 1008 outputs the results of calculations.

[0084] (Effects of the embodiment) As described above, according to the present embodiment, a technique is provided that enables object counting to be performed with high accuracy at low cost.

[0085] (Summary of the embodiment) This specification discloses at least the following inference device, learning device, object counting method, and program. (Section 1) An inference device that counts objects passing through a predetermined line from a captured image of the objects, a detection unit that detects, from the video, a target object that is to be counted for passing through the line; a counting unit that increases a count value when it is detected that the target object has passed through the line and moved from the first area to the second area, A line vicinity region is set in the vicinity of the line on the image, and the line vicinity region does not belong to either the first region or the second region. Reasoning device. (Section 2) The counting unit is storing the first area or the second area in a storage unit as an area in which the target object is located, and when the area in which the target object is located is the line vicinity area, not updating the area stored in the storage unit; When the area stored in the storage unit is the first area and the area in which the target object is located becomes the second area, the count value is increased. 2. The inference apparatus of claim 1. (Section 3) When detecting that the target object has passed through the line and moved from the first area to the second area, the count unit increases a count value corresponding to the movement from the first area to the second area, and when detecting that another target object has passed through the line and moved from the second area to the first area, the count unit increases a count value corresponding to the movement from the second area to the first area. 3. The inference device according to claim 1 or 2. (Section 4) The image processing device further includes a buffer size update unit that calculates a line neighborhood size, which is a size of the line neighborhood area, based on sizes of the plurality of target objects detected by the detection unit, and inputs the line neighborhood size to the count unit. 1. An inference device according to any one of claims 1 to 3. (Section 5) A learning device that calculates a line neighborhood size, which is a size of the line neighborhood area used in the inference device according to any one of claims 1 to 3, comprising: A buffer size calculation unit is provided to calculate the line neighborhood size based on the sizes of a plurality of target objects detected from a video. Learning device. (Section 6) The buffer size calculation unit generates a relational expression indicating a relationship between the positions and sizes of the target objects in the image based on the positions and sizes of the multiple target objects in the image, and calculates the line neighborhood size by applying the position of the line to the relational expression. 6. A learning device as described in claim 5. (Section 7) 1. An object counting method executed by an inference device for counting objects passing through a predetermined line in a video of an object, comprising: a detection step of detecting a target object from the image, the target object being counted for line crossing; a counting step of increasing a count value when it is detected that the target object has passed through the line and moved from the first area to the second area, A line vicinity region is set in the vicinity of the line on the image, and the line vicinity region does not belong to either the first region or the second region. Object counting method. (Section 8) A program for causing a computer to function as each part of an inference device described in any one of claims 1 to 4. (Section 9) A program for causing a computer to function as a buffer size calculation unit in the learning device described in claim 5 or 6.

[0086] Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and variations are possible within the scope of the gist of the present invention described in the claims.

[0087] [References] Reference 1: X. Zhou, D. Wang, and P. Krahenbuhl. Objects as points. In arXiv preprint arXiv:1904.07850, 2019. Reference 2: E. Xie, P. Sun, X. Song, W. Wang, D. Liang, C. Shen, and P. Luo. Polarmask: Single shot instance segmentation with polar representation. In CVPR, 2020. Reference 3: A. Bewley, Z. Ge, L. Ott, F. Ramos, and B. Upcroft. 7Simple online and realtime tracking. In ICIP, 2016. Reference 4: Y. Zhang, C. Wang, X. Wang, W. Zeng, and W. Liu. Fairmot: On the fairness of detection and re-identification in multiple object tracking. In arXiv preprint arXiv:, 2020. [Explanation of symbols]

[0088] 100 Reasoning device 110 Detection unit 120 Tracking Department 130 Counting section 140 Video data storage unit 150 Line information storage section 160 Line neighborhood buffer size storage section 200 Learning Device 210 Detection unit 220 Buffer size calculation unit 230 Video data storage unit 240 Line information storage section 300 Reasoning device 310 Buffer size update section 1000 Drive Device 1001 Recording media 1002 Auxiliary storage 1003 Memory device 1004 CPU 1005 Interface device 1006 Display device 1007 Input Device 1008 Output device

Claims

1. An inference device that counts objects passing through a predetermined line from a captured image of the objects, a detection unit that detects, from the video, a target object that is to be counted for passing through the line; a counting unit that increases a count value when it is detected that the target object has passed through the line and moved from the first area to the second area, a line vicinity region is set in the vicinity of the line on the image, the line vicinity region not belonging to either the first region or the second region; The counting unit is storing the first area or the second area in a storage unit as an area in which the target object is located, and when the area in which the target object is located is the line vicinity area, not updating the area stored in the storage unit; When the area stored in the storage unit is the first area and the area in which the target object is located becomes the second area, the count value is increased. Reasoning device.

2. When detecting that the target object has passed through the line and moved from the first area to the second area, the count unit increases a count value corresponding to the movement from the first area to the second area, and when detecting that another target object has passed through the line and moved from the second area to the first area, the count unit increases a count value corresponding to the movement from the second area to the first area. The inference device of claim 1 .

3. The image processing device further includes a buffer size update unit that calculates a line neighborhood size, which is a size of the line neighborhood area, based on sizes of the plurality of target objects detected by the detection unit, and inputs the line neighborhood size to the count unit.

3. An inference device according to claim 1 or 2.

4. 1. An object counting method executed by an inference device for counting objects passing through a predetermined line in a video of an object, comprising: a detection step of detecting a target object from the image, the target object being counted for line crossing; a counting step of increasing a count value when it is detected that the target object has passed through the line and moved from the first area to the second area, a line vicinity region is set in the vicinity of the line on the image, the line vicinity region not belonging to either the first region or the second region; In the counting step, storing the first area or the second area in a storage unit as an area in which the target object is located, and when the area in which the target object is located is the line vicinity area, not updating the area stored in the storage unit; When the area stored in the storage unit is the first area and the area in which the target object is located becomes the second area, the count value is increased. Object counting method.

5. A program for causing a computer to function as each unit in the inference device according to any one of claims 1 to 3.

Citation Information

Patent Citations

  • Moving-body detector

    JP1996136219A

  • Method for measuring number of passing objects, number-of-passing object measuring device, and program

    JP2012242947A

  • System for counting moving objects by direction

    JP2015018330A

  • Image processing apparatus, image processing method, and program

    JP2021022315A

  • Person movement analysis device, person movement analysis system, and person movement analysis method

    WO2015075870A1