Program, device and method

Abstract

To provide a method capable of identifying an object, an apparatus and a non-transitory computer readable medium.SOLUTION: A method executed by a computer system comprises the steps of: receiving an image of a target subject; determining a direction in response to the receipt of the image, the direction being one in which the target subject was likely to move during a time period in the past or is likely to move during a time period in the future; determining a target area within which another image of the target subject can be expected to appear based on the determined direction; and determining if a portion of a subsequent image is outside or inside the determined target area to identify if the subsequent image is one relating to the target subject, wherein the subsequent image is one taken during the time period in the past or during the time period in the future.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to methods, apparatus and non-transitory computer readable media. [Background technology]

[0002] Video analytics has become increasingly popular in recent years due to the rapid advancements in deep learning algorithms and the availability of affordable and more powerful hardware.

[0003] In particular, facial recognition is being adopted in the following areas; public safety solutions to assist with law enforcement investigations, authentication in online transaction payment solutions, and contactless ID recognition in physical access authentication.

[0004] More recently, facial recognition solutions have separated face detection and person recognition functions into different software services, allowing the creation of one unified system through distributed processing driven by different machines, with a centralized list of registered people that is easier to maintain, compared to traditional siloed solutions that are less integrated and more expensive to scale up when more cameras are introduced. Summary of the Invention [Problem to be solved by the invention]

[0005] In the distributed processing architecture, a detection system is used to interpret images from a video streaming input and generate detection data events, such as faces or people. These detection data events are then transmitted to other services for further processing. During transmission of data events from one system to another within the distributed processing architecture, the individual data events are typically not transmitted in chronological order due to the nature of multi-threaded processing.

[0006] Most of the existing detection de-duplication algorithms rely on sequential or unique identifier numbers (persons) of data to reduce detection duplication, and therefore cannot identify the target in the above scenario.

[0007] It is an object of the present disclosure to provide a method, an apparatus and a non-transitory computer readable medium capable of identifying an object. [Means for solving the problem]

[0008] In a first exemplary aspect, a method comprises receiving an image of an object; determining a direction in response to receiving the image, the direction being a direction along which the object is likely to move during a past time period or a future time period; determining a region of interest in which another image of the object is expected to appear based on the determined direction; and identifying whether the subsequent image is related to the object by determining whether a portion of the subsequent image is within the determined region of interest, the subsequent image being taken during the past time period or the future time period.

[0009] In a second exemplary aspect, an apparatus includes at least one processor and at least one memory including computer program code configured to, in cooperation with the at least one processor, cause the apparatus to at least perform the following: receive an image of an object; determine a direction in response to receiving the image, the direction being a direction along which the object is likely to move during a past period or a future period; determine a region of interest in which another image of the object is expected to appear based on the determined direction; and identify whether the subsequent image is related to the object by determining whether a portion of the subsequent image is within the determined region of interest, the subsequent image being taken during the past period or the future period.

[0010] In a third exemplary aspect, a non-transitory computer-readable medium stores a program that causes a computer to receive an image of an object; determine a direction in response to receiving the image, the direction being a direction along which the object is likely to move during a past period of time or a future period of time; determine a region of interest within which another image of the object is expected to appear based on the determined direction; and identify whether the subsequent image is related to the object by determining whether a portion of the subsequent image is within the determined region of interest, the subsequent image being taken during the past period of time or the future period of time. Effect of the Invention

[0011] According to the present disclosure, a method, an apparatus, and a non-transitory computer-readable medium can be provided that can identify an object. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 shows a block diagram of an apparatus according to one embodiment. [Diagram 2] 1 shows a block diagram of a system in which objects can be adaptively identified according to one embodiment. [Diagram 3] 1 shows a flowchart illustrating a method for adaptively identifying objects according to an embodiment of the present disclosure. [Figure 4A] 3 shows a flow chart 300 illustrating how objects can be adaptively identified. [Figure 4B] 3 shows a flow chart 300 illustrating how objects can be adaptively identified. [Figure 5A] 3 shows a flowchart 330 illustrating how objects may be adaptively identified, according to one embodiment of the present disclosure. [Figure 5B] 3 shows a flowchart 330 illustrating how objects may be adaptively identified, according to one embodiment of the present disclosure. [Figure 5C]3 shows a flowchart 330 illustrating how objects may be adaptively identified, according to one embodiment of the present disclosure. [Figure 6A] 1 depicts a schematic diagram of how the orientation of an object can be determined according to one embodiment; [Figure 6B] 1 shows example coordinates for determining a direction and a face. [Figure 6C] 1 shows a schematic diagram of how the orientation of an object in a future time period can be determined. [Figure 6D] 1 shows a schematic diagram of how the orientation of an object during a past period of time can be determined. [Figure 7] FIG. 2 depicts a schematic diagram of how a region of interest can be determined. [Figure 8] 3-5C depict an exemplary computing device that may be used to perform the methods of FIGS. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Before describing the embodiments of the present disclosure, a few notes should be made.

[0014] In the field of video analytics, most of the existing solutions focus on detecting and identifying registered persons, which may be persons assigned a unique identifier for data processing.

[0015] Conventionally, facial recognition solutions combine face detection and person recognition functions, and generate warnings in application software by focusing on matching registered users. This is cost-inefficient and difficult to maintain when the number of cameras increases. Therefore, a facial recognition solution has been developed that splits face detection and person recognition functions into different software services. However, almost all existing detection and de-duplication algorithms cannot identify objects if individual data events are not sent in chronological order. The present disclosure can solve this problem.

[0016] Furthermore, other desirable features and characteristics will become apparent from the subsequent detailed description and the appended claims, taken in conjunction with the accompanying drawings and this background of the disclosure.

[0017] Embodiments of the present disclosure will be better understood and readily apparent to those skilled in the art from the following description, by way of example only, and in conjunction with the drawings in which:

[0018] Embodiments of the present disclosure will now be described, by way of example only, with reference to the drawings in which like reference numerals and letters refer to like elements or equivalents.

[0019] Some portions of the description which follow are presented either explicitly or implicitly in terms of algorithms and functional or symbolic representations of operations on data within a computer memory. These algorithmic descriptions and functional or symbolic representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art, and an algorithm is here, generally, conceived to be a self-consistent sequence of steps leading to a desired result. The steps are those requiring physical manipulations of physical quantities, such as electrical, magnetic or optical signals, which can be stored, transferred, combined, compared, and otherwise manipulated.

[0020] Unless otherwise indicated, and as will become apparent hereinafter, discussions throughout this specification using terms such as "receive," "calculate," "determine," "update," "generate," "initialize," "output," "receive," "search," "identify," "distribute," "authenticate," and the like will be understood to refer to operations and processes of a computer system or similar electronic device that manipulates and transforms data represented as physical quantities within a computer system into other data similarly represented as physical quantities within a computer system, or other information storage, transmission, or display.

[0021] This specification also discloses apparatus for performing the operations of these methods. Such apparatus may be specially constructed for the required purposes, or may include a computer or other apparatus selectively activated or reconfigured by a computer program stored in the computer. The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various machines may be used with programs in accordance with the teachings of this specification. Alternatively, the construction of more specialized apparatus to perform the required method steps may be appropriate. The structure of the computer will be apparent from the following description.

[0022] Moreover, this specification also implicitly discloses a computer program in that it will be apparent to one skilled in the art that the individual steps of the methods described herein may be implemented by computer code. The computer program is not intended to be limited to a particular programming language and its implementation. It will be understood that a variety of programming languages ​​and their coding can be used to implement the teachings of the disclosure contained herein. Furthermore, the computer program is not intended to be limited to a particular control flow. There are many other variations of the computer program, which may use different control flows without departing from the spirit or scope of the present disclosure.

[0023] Furthermore, one or more steps of a computer program may be executed in parallel rather than sequentially. Such a computer program may be stored on any computer readable medium. The computer readable medium may include storage devices such as magnetic or optical disks, memory chips, or other storage devices suitable for interfacing with a computer. The computer readable medium may also include hardwired media (e.g., electrical wires or optical fibers), such as typified by the Internet system, or wireless media, such as typified by the Global System for Mobile Communications (GSM) mobile phone system. Furthermore, the computer readable medium may be any type of non-transitory computer readable medium. The non-transitory computer readable medium may include any type of tangible storage medium. Examples of non-transitory computer-readable media include magnetic storage media (such as floppy disks, magnetic tapes, hard disk drives, etc.), magneto-optical storage media (e.g. magneto-optical disks), compact disc read only memory (CD-ROM), compact disc recordable (CD-R), compact disc rewritable (CD-R / W), and semiconductor memory (e.g. mask ROM, programmable ROM (PROM), erasable PROM (EPROM), flash ROM, random access memory (RAM), etc.). Also, the program may be provided to the computer using any type of transitory computer-readable medium. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. A computer program loaded and executed on such a computer effectively results in an apparatus for performing the steps of the preferred method.

[0024] Various embodiments of the present disclosure relate to a method and apparatus for adaptively identifying an object. In one embodiment, the method and apparatus identify whether a subsequent image is of an object by determining whether a portion of the subsequent image is within a determined region of interest.

[0025] (First embodiment) A first embodiment of the present disclosure will now be described with reference to FIG.

[0026] 1 shows a block diagram of a device 10. The device 10 includes at least one processor 11 and at least one memory 12. In this embodiment, the device 10 includes one processor 11 and one memory 12, but the number of these components is not limited to this example. The memory 12 includes computer program code.

[0027] The memory 12 and computer program code are configured to cause the processor 11 to cause the device 10 to at least: receiving an image of the object; determining a direction in response to receiving the image, the direction being a direction in which the object is likely to move during the past period or the future period; determining an area of ​​interest in which another image of the object is expected to appear based on the determined orientation; and Identifying whether a subsequent image is related to the object by determining whether a portion of the subsequent image is within the determined region of interest, the subsequent image being taken during a past time period or a future time period.

[0028] The method performed by the device 10 allows the device 10 to identify the object because it is capable of detecting subsequent images associated with the object, regardless of their temporal order.

[0029] Second embodiment A second embodiment of the present disclosure will now be described with reference to the accompanying drawings, in which: As shown below, the method and apparatus disclosed herein can identify objects, particularly for event throttling.

[0030] FIG. 2 illustrates a block diagram of a system 100 capable of adaptively identifying an object, according to one embodiment. Referring to FIG. 2, the object identification process includes a device 102 operatively coupled to at least one imaging device 110. In FIG. 2, there are N imaging devices 110_1 through 110_N. Each imaging device 110 is configured to capture and transmit at least one image of an object at a location. The imaging devices 110 may include, among others, a face detection system, a video capture device, a camera, and a motion sensor. The device 102 is configured to receive an image of the object.

[0031] The imaging device 110 is capable of wireless communication with the device 102 using an appropriate protocol. For example, embodiments can be implemented using an imaging device 110 capable of communicating with a WiFi® / Bluetooth® enabled device 102. Depending on the wireless communication protocol used, those skilled in the art will appreciate that appropriate handshaking procedures may need to be performed to establish communication between the imaging device 110 and the device 102. For example, in the case of Bluetooth® communication, the imaging device 110 and the device 102 may be discovered and paired to establish communication.

[0032] In one example, images of an object are captured (or detected) by imaging device 110 at a predetermined frames per second (or fps). Images of an object may be captured as the object moves through an area, and multiple images are captured and an fps for imaging device 110 is determined. If imaging device 110 has an fps of 5, then 5 frames of the object are captured per second and transmitted to device 102. For various reasons, the captured images may not be transmitted to device 102 in a chronological manner, where consecutive images captured over a period of time are transmitted in a sequential manner.

[0033] The device 102 may include a processor 104 and a memory 106. In an embodiment of the present disclosure, the memory 106 and computer program code are configured to, in cooperation with the processor 104, cause the device 102 to receive an image of an object, and in response to receiving the image, determine a direction, the direction being a direction in which the object is likely to move during a past time period or a future time period, and based on the determined direction, determine a region of interest in which another image of the object is expected to appear, and identify whether the subsequent image is related to the object by determining whether a portion of the subsequent image is outside the determined region of interest, the subsequent image being taken during a past time period or a future time period.

[0034] Device 102 may be a server. In embodiments of the present disclosure, use of the term "server" may refer to a single computing device, or at least a computer network of interconnected computing devices that work together to perform a particular function. In other words, a server may be housed within a single hardware unit or may be distributed among multiple or many different hardware units.

[0035] Such a server may be used to implement the method 200 shown in Figure 3. Figure 3 shows a flow chart illustrating the method 200 for adaptively identifying objects according to an embodiment of the present disclosure.

[0036] Previously, images not related to the object could still be considered for identifying the object, which reduced the effectiveness and accuracy of the technique.

[0037] Thus, embodiments of the present disclosure can adaptively identify an object based on any image associated with the object. The image does not have to be the first image in a series of previously received images or an image in the series. That is, the images sent to device 102 do not have to be in chronological order. This is possible because various embodiments determine the direction in which an object is likely to move during a past or future time period. In contrast, prior art techniques can only process images received in chronological order.

[0038] The method 200 broadly includes the following. Step 202: receiving, by a processor, an image of an object; Step 204: determining, by the processor, a direction in response to receiving the image, the direction being a direction in which the object is likely to move during a past period or a future period; Step 206: determining, by the processor, a region of interest in which another image of the object is expected to appear based on the determined orientation; Step 208: The processor identifies whether the subsequent image is related to the object by determining whether a portion of the subsequent image is outside or inside the determined region of interest, the subsequent image being taken during a past time period or a future time period.

[0039] In existing face recognition solutions, face detection and person recognition are often used separately as two functions, making it difficult to build an integrated system with distributed processing. This is especially true for an integrated system through distributed processing driven by different machines, with the list of registered people centralized for easy maintenance. This is in contrast to traditional silo solutions that are not integrated and are costly to scale when more cameras are introduced.

[0040] 4A and 4B show a flow chart 300 illustrating how objects may be adaptively identified. The system may include an identification server operatively coupled to the imaging device and a transmitter for transmitting other data related to the vehicle.

[0041] An identification server is typically associated with a surveillance provider or party providing surveillance. A surveillance provider may be an entity (e.g., a business or organization) that manages (e.g., manages) the images (e.g., buses). As mentioned above, a server may include one or more computers used to establish communication with other devices (e.g., sensors or servers) by exchanging messages with them and / or passing information to them.

[0042] In step 302, an image of an object (detection target) may be received. This image may be any frame of a series of images taken during a period of time, for example the first, second or third frame of a series of images taken at 5 fps.

[0043] In step 304, the method includes comparing the image information received in step 302 to an exclusion list that includes previously identified objects.

[0044] In step 305, the method includes comparing information of the image received in step 302 to determine whether the information of the image matches information of any of the images in the exclusion list.

[0045] If it is determined that the information of the image matches the information of any of the images in the exclusion list (Yes at step 305), the method ends. It should be understood that steps 302, 304 and 305 may not be required in some embodiments.

[0046] If it is determined that the image does not match any image in the exclusion list (No in step 305), the method proceeds to step 306. In step 306, a list of images of the object captured by the same image capture device is retrieved, i.e., a cached list of detected objects of the same image source can be obtained.

[0047] At step 308, there is a process that determines whether the target list can be retrieved, i.e., whether a cached list is found.

[0048] If, in step 310, it is determined that the list of objects is searchable (Yes in step 308), a process is performed to determine whether the object is a duplicate, i.e., whether an image of the object has been previously received.

[0049] In step 312, a process is performed to determine whether the target images overlap. For example, the images overlap if they are entirely within the target region, i.e., no portion of the image touches the exclusion region. If it is determined that the target images overlap (Yes in step 312), the method ends. If it is determined that the target images do not overlap (No in step 312), step 314 is performed.

[0050] If step 308 is determined to be negative (i.e., the list of objects cannot be retrieved) in step 314, the method includes storing information of the image, including images captured by the same image capture device, past and future throttle zones, and past and future exclusion regions in the cache of the same image source. At this step, the object region is analyzed (end).

[0051] As described above, if in step 312 it is determined that the target images are not overlapping, step 314 is executed.

[0052] An alert signal is generated and transmitted in step 316. This step may be performed by the processor or a separate alert generating device operatively coupled to the processor.

[0053] 5A-5C show a flowchart 330 illustrating a method that may adaptively identify objects according to an embodiment of the present disclosure.

[0054] Step 352 is similar to step 306 described in Figure 4A. In this step, the first cached object is obtained.

[0055] In step 354, the method includes determining whether an image of the object detected in step 352 was captured in the past. In other words, the method includes determining whether the detected object occurred in the past before the cached object.

[0056] Responsive to step 354, a determination is made in step 356 to discover whether the received image occurred in the past.

[0057] In response to step 356, if it is determined that an image of the object detected in step 352 was previously captured (Yes in step 356), then step 358 is performed.

[0058] In step 358, a historical region of interest is retrieved based on the direction of the object, which is the direction the object is likely to move in during the past period prior to the image received in step 302. The direction may be determined based on at least two points on the body (e.g., a shoulder point and a face point). In one example, this step is performed based on a timestamp of the image to determine when the image was received. The region of interest may also be referred to as a throttle zone, and the area outside the determined region of interest may be referred to as an exclusion zone. In step 358, the cached historical throttle zone and exclusion zone of the object are obtained.

[0059] If, in response to step 356, it is determined that the received image was not captured prior to the object for which the image was received (No at step 356), then step 360 is performed. At step 360, a future region of interest is searched for based on the direction of the object, which is the direction the object is likely to move in during a future time period prior to the image received at step 302. In one example, this step is performed based on a timestamp of the image to determine when the image was received. The future region of interest may also be referred to as a throttle zone in a future time zone, and the region outside the determined region of interest may also be referred to as an exclusion region. At step 360, the future throttle zone and exclusion region of the cached object are obtained.

[0060] Based on the object area determined in step 358 or step 360, step 362 is executed. In step 362, it is determined whether a part of the image is within the object area. In other words, it is determined whether the detected object is within the throttle zone. That is, this step determines whether the object is moving or has moved in the direction determined in step 358 or step 360. Details are shown in the following figures.

[0061] In response to step 362, step 364 is performed. In step 364, the method includes determining whether a portion of the image is within the region of interest determined in either step 358 or step 360. If the portion of the image is determined to be within the region of interest (Yes at step 364), step 366 is performed. If the portion of the image is determined to not be within the region of interest (No at step 364), step 370 is performed.

[0062] If, responsive to step 364, it is determined that a portion of the subsequent image is outside the determined region of interest, and to identify whether the subsequent image is related to the object, step 366 is performed. In step 366, it is determined whether a portion of the image crosses any of the exclusion regions.

[0063] In one example, the method is performed to determine if a portion of the image is outside of the region of interest (or throttle zone) determined in step 358 or step 360. That is, this step is performed to discover if the object is moving outside of the direction expected for that object within the time period.

[0064] In response to step 366, determining whether any part of the image of the object is outside the region of interest is performed in step 368.

[0065] If, in response to step 368, it is determined that a portion of the image of the target is within the target region (No at step 368), step 374 is executed and the image is marked as being a duplicate. Steps 362 through 364 can be considered a method of comparing person detection.

[0066] If, in response to step 368, it is determined that no part of the image of the object is outside the region of interest (Yes at step 368), then step 370 is performed to determine whether there are other images of the same object.

[0067] If, in response to step 370, it is determined that there is information for other images of the same object (Yes at step 370), the method continues to obtain information for other images of the object at step 372. The steps from step 354 are then repeated.

[0068] If, in response to step 370, it is determined that there is no other image information of the same subject (No at step 370), the method proceeds to step 376 where the image is recorded as a unique image.

[0069] FIG. 6A shows a schematic diagram of how the orientation of an object can be determined according to one embodiment.

[0070] 6A shows the current detection of an object. This can be determined based on an image of the object. Those skilled in the art will understand that there are various ways to determine the moving direction of an object. For example, the direction can be determined based on two body points (such as two pointing points) or two face points.

[0071] In Figure 6A the object is represented by 402. In order to determine the direction an object is likely to move at some time in the past, or likely to move at some time in the future, two points must be considered for distance calculation: the center of the head and the center of the eye positions.

[0072] As shown in FIG. 6B, the coordinates of the head center and eye center are used to determine the direction of the object's movement. Objects 402_1 to 402_3 are examples of faces, each with a different facial orientation. The coordinates of these points are obtained and used to calculate the H CX represents the "x" at the center of the head, and H CY represents the head-centered "y", and H X represents the "x" in the upper left corner of the head, and H W represents the width of the head. E CX represents the center of both eyes, "x", and E CY represents the center of both eyes, "y", and E LX represents the left eye point "x", and E RX represents the right eye point "x", and FH H represents the height of the forehead. The direction can be determined as follows: H CX = H X + (H W / 2), E CX = E LX + ((E RX -E LX ) / 2), and FH H = E LY -H CY .

[0073] As shown in FIG. 6A, the current location of the object is indicated by reference numeral 402. Based on the orientation of the object's head, the object indicated by 402d is likely to indicate the orientation of the object at some time in the past (indicated as "estimated past" in FIG. 6A). A region of interest is also determined based on 402d and indicated by 430a. Based on the orientation of the object's head, the object indicated by 402b is likely to indicate the orientation of the object at some time in the future (indicated as "estimated future" in FIG. 6A). A region of interest is also determined based on 402b and indicated by 410a.

[0074] 6C shows a schematic diagram of how the orientation of an object at some time in the future can be determined. As mentioned above, the orientation can be determined as follows. H CX = H X + (H W / 2), E CX = E LX + ((E RX -E LX ) / 2), and FH H = E LY -H CY

[0075] In FIG. 6C, H C1 (H C1X ,H C1Y ) represents the position coordinate of the head center in image I1, and H C1X represents the "x" at the center of the head, and H C1Y represents the head-centered "y". E C1 (E C1X ,E C1Y ) represents the position coordinates of the centers of the eyes in image I1, and E C1X represents the center of both eyes, "x", and E C1Y represents the center of both eyes, "y". E C2 (E C2X ,E C2Y ) represents the position coordinates of the centers of the eyes in image I2, and E C2X represents the center of both eyes, "x", and H C2Yrepresents the center of both eyes, y. Image I2 is the next estimated image of image I1. E C2X and E C2y can be determined as follows: E C2X = H C1X - (H h H C1X -H h E C1X ) / FH H E C2Y = H C1Y + H h

[0076] FIG. 6D shows a schematic diagram of how the orientation of an object at some time in the past can be determined. C1 (H C1X ,H C1Y ) represents the position coordinate of the head center in image I3, and H C1X represents the "x" at the center of the head, and H C1Y represents the head-centered "y". C2 (H C2X ,H C2Y ) represents the position coordinate of the head center in image I4, and H C2X represents the "x" at the center of the head, and H C2Y represents the head-centered "y". E C2 (E C2X ,E C2Y ) represents the position coordinates of the centers of the eyes in image I4, and E C2X represents the center of both eyes, "x", and E C2Y represents the center of both eyes, y. Image I3 is the estimated previous image of image I4. H C1X and H CIY can be determined as follows: H C1X = (E C2X (H C1Y -H C2Y ) - H C2X (H C1Y -E C2Y )) / (E C2Y -H C2Y ) H C1Y = EC2Y -H h

[0077] 6C and 6D show how the orientation of an object can be determined taking into account the height of the object's forehead, since this parameter affects how much the object moves. Later, this affects the size of the object region. Further details on how the object region is determined are provided in FIG.

[0078] FIG. 7 shows a schematic diagram of how the interest region can be determined. Based on the current position of the object 402, and in the next 2 seconds (indicated as "+1 sec" and "+2 sec"), the direction in which the object is likely to move for some time in the future is indicated as 402a and 402b. Then, based on the determined direction, the interest region 410a is determined, which is the region in which other images of the object are expected to appear. As an example, the interest region 410a, 410b may be the outline of the image of the object that is expected to appear for some time in the future. To adaptively identify the object, images that are within the object and have parts of them touching outside the interest region 420a, 420b are ignored. The regions 420a, 420b are also called exclusion regions. These are shown as outlier images indicated as 450 in FIG. 7.

[0079] The direction in which the object is likely to move in a past period is shown in 402c and 402d for the previous 3 seconds (denoted as "-1 sec", "-3 sec"). Then, based on the determined direction, a region of interest 430a is determined, which is a region in which other images of the object are expected to appear. As an example, the region of interest 430a, 430b may be the outline of an image of the object expected to appear in a past period. To adaptively identify the object, images that are within the object and have parts of them touching outside the region of interest 440a, 440b are ignored. The regions 440a, 440b are also called rejection regions. These are shown as outlier images, denoted as 450 in FIG. 7.

[0080] Figure 8 illustrates an exemplary computer device 600, hereinafter interchangeably referred to as computer system 600, where such computer device 600 may be used to perform the methods of Figures 4A to 5C. The exemplary computer device 600 may be used to implement the system 100 shown in Figure 2. The following description of computer device 600 is provided by way of example only and is not intended to be limiting.

[0081] As shown in Figure 8, the exemplary computing device 600 includes a processor 607 for executing software routines. Although a single processor is shown for clarity, the computing device 600 may include a multi-processor system. The processor 607 is connected to a communications infrastructure 606 for communicating with other components of the computing device 600. The communications infrastructure 606 may include, for example, a communications path, a crossbar, or a network.

[0082] The computing device 600 further includes a main memory 608, such as a random access memory (RAM), and a secondary memory 610. The secondary memory 610 may include, for example, a hard disk drive 612, which may be a solid state drive or hybrid drive, and a removable storage drive 617, which may include a magnetic tape drive, an optical disk drive, a solid state storage drive (Universal Serial Bus (USB) flash drive, a flash memory device, a solid state drive, a memory card, etc.). The removable storage drive 617 reads from and / or writes to the removable storage medium 677 in a well-known manner. The removable storage medium 677 may include a magnetic tape, an optical disk, a non-volatile memory storage medium, etc., that is read from and written to by the removable storage drive 617. As will be appreciated by those skilled in the art, the removable storage medium 677 includes a computer readable storage medium having computer executable program code instructions and / or data stored thereon.

[0083] In alternative implementations, the secondary memory 610 may additionally or alternatively include other similar means for allowing computer programs or other instructions to be loaded into the computer system 600. Such means may include, for example, a removable storage unit 622 and interface 650. Examples of removable storage units 622 and interfaces 650 include program cartridges and cartridge interfaces (such as those found in video game console devices), removable memory chips (such as EPROMs (Erasable Programmable Read Only Memory) or PROMs (Programmable Read Only Memory)) and associated sockets, removable solid-state storage drives (such as USB flash drives, flash memory devices, solid-state drives, memory cards, etc.), and other removable storage units 622 and interfaces 650 that allow software and data to be transferred from the removable storage unit 622 to the computer system 600.

[0084] The computing device 600 also includes at least one communication interface 627. The communication interface 627 allows software and data to be transferred between the computing device 600 and external devices via the communication path 626. In various disclosed embodiments, the communication interface 627 allows data to be transferred between the computing device 600 and a data communication network, such as a public or private data communication network. The communication interface 627 may be used to exchange data between different computing devices 600, such computing devices 600 forming part of an interconnected computer network. Examples of the communication interface 627 may include a modem, a network interface (such as an Ethernet card), a communication port (such as serial, parallel, printer, General Purpose Interface Bus (GPIB), IEEE 1394, RJ45, USB, etc.), an antenna with associated circuitry, etc. The communication interface 627 may be wired or wireless. The software and data transferred via the communication interface 627 are in the form of signals, which may be electronic, electromagnetic, optical or other signals receivable by the communication interface 627. These signals are provided to the communications interface via communications path 626.

[0085] As shown in FIG. 8, computing device 600 further includes a display interface 602 that performs operations for rendering images on an associated display 655, and an audio interface 652 that performs operations for playing audio content via associated speaker(s) 657.

[0086] The term "computer program product" as used herein may refer in part to removable storage medium 677, removable storage unit 622, hard disk installed in storage drive 612, or carrier that carries software to communication interface 627 via communication path 626 (wireless link or cable). Computer readable storage medium refers to any non-transitory, non-volatile, tangible storage medium that provides recorded instructions and / or data to computer device 600 for execution and / or processing. Examples of such storage media include magnetic tape, CD-ROM (Compact Disc Read Only Memory), DVD (Digital Versatile Disc), Blu-ray Disc, hard disk drive, ROM (Read Only Memory) or integrated circuit, solid state storage drive (USB flash drive, flash memory device, solid state drive, memory card, etc.), hybrid drive, magneto-optical disk, or computer readable card such as PCMCIA (Personal Computer Memory Card International Association) card, whether these devices are internal or external to computer device 600. Examples of transitory or non-tangible computer-readable transmission media that may be involved in providing software, application programs, instructions and / or data to computer device 600 include wireless or infrared transmission channels, as well as network connections to other computers or networked devices, and the Internet or intranets, including email transmissions and information stored on websites and the like.

[0087] Computer programs (also referred to as computer program code) are stored in the main memory 608 and / or the secondary memory 610. Computer programs may also be received via the communications interface 627. Such computer programs, when executed, enable the computing device 600 to perform one or more features of the embodiments described herein. In various embodiments, the computer programs, when executed, enable the processor 607 to perform the features of the above-mentioned embodiments. Such computer programs thus represent controllers of the computing system 600.

[0088] The software may be stored in a computer program product and loaded into the computing device 600 using the removable storage drive 617, the storage drive 612, or the interface 650. The computer program product may be a non-transitory computer readable medium. Alternatively, the computer program product may be downloaded to the computing system 600 via the communications path 627. The software, when executed by the processor 607, causes the computing device 600 to perform the necessary operations to execute the method 200 as shown in FIG.

[0089] It should be understood that the embodiment of FIG. 8 is presented merely as an example to illustrate the operation and structure of system 100. Thus, in some embodiments, one or more features of computer device 600 may be omitted. Also, in some embodiments, one or more features of computer device 600 may be combined. Furthermore, in some embodiments, one or more features of computer device 600 may be split into one or more components.

[0090] It will be appreciated that the elements illustrated in FIG. 8 function to provide means for performing the various functions and operations of the server as described in the above embodiments.

[0091] When the computing device 600 is configured to perform the above functions, the computing system 600 has a non-transitory computer readable medium having stored thereon an application that, when executed, causes the computing system 600 to perform steps including: receiving an image of an object; determining a direction in response to receiving the image, the direction being a direction along which the object is likely to move during the past time period or during a future time period; determining a region of interest within which another image of the object is expected to appear based on the determined direction; identifying whether a subsequent image is related to the object by determining whether any portion of the subsequent image is outside of the determined region of interest, the subsequent image being taken during the past time period or during a future time period.

[0092] The processor illustrated in the embodiments may be, for example, a microprocessor, a Micro Processing Unit (MPU), a Central Processing Unit (CPU), etc. The processor may include multiple processors.

[0093] The memory shown in the embodiment is a combination of volatile and non-volatile memory. The memory may include a storage device located remotely from the processor. In this case, the processor may access the memory via an I / O interface.

[0094] The memory shown in the embodiment is used to store a group of software modules. The processor can read the group of software modules from the memory and execute the read software modules to perform the processing performed by the information processing device described in the above embodiment.

[0095] Each processor included in the apparatus of the above-described embodiment executes one or more programs including a group of instructions, causing the computer to execute the algorithm described above with reference to the drawings.

[0096] The device may further include a network interface for communicating with other network node devices constituting the communication system, and may include, for example, a network interface card (NIC) conforming to the IEEE 802.3 series.

[0097] A part or all of the above-described embodiments can be described as follows, but the present invention is not limited thereto. (Appendix 1) Receive an image of the object; determining a direction in response to receiving the images, the direction being a direction in which the object is likely to move during a past period of time or a future period of time; determining an area of ​​interest in which another image of the object is expected to appear based on the determined orientation; identifying whether a subsequent image is related to the object by determining whether a portion of the subsequent image is within the determined region of interest, the subsequent image being taken during the time period in the past or the time period in the future; How to prepare for this. (Appendix 2) comparing information of the received image with information of one or more images in a list, each of the one or more images relating to an object in respect of which a detection has been made; determining whether there is a listing associated with the object when information of the received image corresponds to information of one or more images in the list. The method described in Appendix 1. (Appendix 3) When it is determined that there is no listing associated with the object, the step of determining an analysis of the region of interest is performed. The method described in Appendix 2. (Appendix 4) If it is determined that there is a listing associated with the object, determining whether the received image was captured before the one or more objects on the list were captured; and when it is determined that the received image was taken before the one or more objects on the list were photographed, the determined direction is a direction in which the object is likely to move during the past period of time, and the object region is one in which another image of the object is expected to appear during the past period of time. 4. The method according to claim 2 or 3. (Appendix 5) If it is determined that there is a listing associated with the object, determining whether the received image was captured before the one or more objects on the list were captured; when it is determined that the received image was not captured before the one or more objects on the list were captured, the determined direction is a direction in which the object is likely to move during the future period, and the object region is one in which another image of the object is expected to appear during the future period; 5. The method according to any one of claims 2 to 4. (Appendix 6) if it is determined that a portion of the subsequent image is within the determined region of interest; further comprising identifying objects in the subsequent images as overlapping. 6. The method according to any one of claims 1 to 5. (Appendix 7) generating a warning signal. 7. The method according to any one of claims 1 to 6. (Appendix 8) the determining the orientation is based on at least two body parts of the object; 8. The method according to any one of claims 1 to 7. (Appendix 9) At least one processor; at least one memory containing computer program code; The at least one memory and the computer program code, in cooperation with at least one processor, cause an apparatus to: Receive an image of the object; determining a direction in response to receiving the images, the direction being a direction in which the object is likely to move during a past period of time or a future period of time; determining an area of ​​interest in which another image of the object is expected to appear based on the determined orientation; identifying whether a subsequent image is related to the object by determining whether a portion of the subsequent image is within the determined region of interest, the subsequent image being taken during the time period in the past or the time period in the future; An apparatus configured to at least perform the following: (Appendix 10) The at least one memory and the computer program code, in cooperation with at least one processor, cause the apparatus to: comparing information of the received image with information of one or more images in a list, each of the one or more images relating to an object in respect of which a detection has been made; determining whether there is a listing associated with the object when information of the received image corresponds to information of one or more images in the listing; 10. The apparatus of claim 9. (Appendix 11) The at least one memory and the computer program code, in cooperation with at least one processor, cause the apparatus to: determining to perform an analysis of the region of interest when it is determined that there is no listing associated with the object. 11. The apparatus of claim 10. (Appendix 12) The at least one memory and the computer program code, in cooperation with at least one processor, cause the apparatus to: If it is determined that there is a listing associated with the object, determining whether the received image was captured before the one or more objects on the list were captured; performing at least when it is determined that the received image was taken before one or more of the objects on the list were photographed, the determined direction is a direction in which the object is likely to move during the past period of time, and the object region is one in which another image of the object is expected to appear during the past period of time; 12. The apparatus of claim 10 or 11. (Appendix 13) The at least one memory and the computer program code, in cooperation with at least one processor, cause the apparatus to: If it is determined that there is a listing associated with the object, determining whether the received image was captured before the one or more objects on the list were captured; performing at least when it is determined that the received image was not captured before the one or more objects on the list were captured, the determined direction is a direction in which the object is likely to move during the future period, and the object region is one in which another image of the object is expected to appear during the future period; 13. The apparatus of any one of claims 10 to 12. (Appendix 14) the at least one memory and the computer program code in cooperation with the at least one processor to cause the device to at least identify objects in the subsequent image as overlapping if a portion of the subsequent image is determined to be within the determined region of interest. 14. The apparatus of any one of claims 9 to 13. (Appendix 15) the at least one memory and the computer program code in cooperation with at least one processor cause the device to at least generate a warning signal; 15. The apparatus of any one of claims 9 to 14. (Appendix 16) the at least one memory and the computer program code, in cooperation with at least one processor, cause the device to determine the orientation based on at least two body parts of the object. 16. The apparatus of any one of claims 9 to 15. (Appendix 17) Receive an image of the object; determining a direction in response to receiving the images, the direction being a direction in which the object is likely to move during a past period of time or a future period of time; determining an area of ​​interest in which another image of the object is expected to appear based on the determined orientation; identifying whether a subsequent image is related to the object by determining whether a portion of the subsequent image is within the determined region of interest, the subsequent image being taken during the time period in the past or the time period in the future; A non-transitory computer-readable medium that stores a program that causes a computer to execute a program.

[0098] As will be appreciated by those skilled in the art, many variations and / or modifications may be made to the present disclosure as illustrated in the specific embodiments without departing from the spirit or scope of the disclosure as broadly described, and the present embodiments are therefore to be considered in all respects as illustrative and not restrictive.

[0099] This application claims priority to Singapore Patent Application No. 10201906302X, filed on July 5, 2019, the disclosure of which is incorporated herein in its entirety. [Explanation of symbols]

[0100] 10 equipment 11 Processors 12. Memory 102 Equipment 104 processors 106 Memory 110 Imaging device 600 Computer Systems 602 Display Interface 606 Communications Infrastructure 607 Processor 608 Main Memory 610 Secondary Memory 612 Hard Disk 617 Removable Storage Drive 622 Removable Storage Unit 626 Communication Path 627 Communication Interface 650 Interface 652 Audio Interface 655 Display 657 (multiple) speakers 677 Removable Storage Media

Claims

1. One or more computers, means for acquiring a first image of an object; means for acquiring a second image after acquiring the first image; a means for determining whether a part of the second image is captured in a first direction of the first image when the second image is captured chronologically before the first image; a means for determining whether a part of the second image is captured in a second direction of the first image when the second image is captured chronologically after the first image; and an image processing program that causes the image processing program to operate as a means for determining the relevance between the object and the second image depending on the result of the judgment.

2. and further operating as a means for determining the first direction or the second direction based on two points on the object.

2. A program for processing an image according to claim 1.

3. and determining that the second image is related to the object when a part of the second image is captured in a first region in the first direction or a second region in the second direction.

3. An image processing program according to claim 1 or 2.

4. A means for acquiring a first image showing an object; means for acquiring a second image after acquiring the first image; a means for determining whether a part of the second image is captured in a first direction of the first image when the second image is captured chronologically before the first image; a means for determining whether a part of the second image is captured in a second direction of the first image when the second image is captured chronologically after the first image; means for determining a relevance between the object and the second image in response to the result of the determination; An apparatus comprising:

5. Obtaining a first image showing an object; acquiring a second image after acquiring the first image; When the second image is an image captured chronologically before the first image, it is determined whether a part of the second image is captured in a first direction of the first image; When the second image is an image captured chronologically after the first image, it is determined whether a part of the second image is captured in a second direction of the first image; determining a relevance between the object and the second image depending on a result of the determination; The way computers run.

Images