Image processing apparatus and method for searching of object thereof

Abstract

An embodiment of the present invention discloses an image processing device comprising: an analysis unit that acquires position data of an object from a plurality of images; a sampling unit that generates quantized first vectors representing changes in the position of the object based on the position data of the object; a comparison unit that generates quantized second vectors representing changes in the position of a user trajectory input as a search condition and compares the first vector and the second vector at the same position in a reference image; and a search unit that detects an object moving along a trajectory matching the user trajectory according to the comparison result.

Description

Technology Field

[0001] An embodiment of the present invention relates to an image processing device having an object search function. Background Technology

[0002] Systems such as CCTV operate 24 hours a day, 365 days a year. In the case of playback rather than real-time monitoring, significant effort is required to find desired video content if a specific time cannot be determined. Recently, technology for searching for objects using intelligent video analysis has been introduced. Prior art literature

[65535] Published Patent Application No. 10-2016-0104054 (September 2, 2016) The problem to be solved

[0003] Embodiments of the present invention aim to provide a method for rapidly searching for objects satisfying search conditions while minimizing computational data, and an image processing device for performing the same. means of solving the problem

[0004] An image processing device according to one embodiment of the present invention comprises: an analysis unit that acquires position data of an object from a plurality of images; a sampling unit that generates quantized first vectors representing changes in the position of the object based on the position data of the object; a comparison unit that generates quantized second vectors representing changes in the position of a user trajectory input as a search condition and compares the first vector and the second vector at the same position in a reference image; and a search unit that detects an object moving along a trajectory matching the user trajectory according to the comparison result.

[0005] In one embodiment, the first vectors are located in sub-regions corresponding to the position data of the object among a plurality of sub-regions constituting the reference image, and the second vectors are located in sub-regions corresponding to the user trajectory in the reference image, and each of the first vectors and the second vectors may have the size of the sub-region and one of the quantized directions.

[0006] In one embodiment, the sampling unit may select sub-regions corresponding to the position data of the object in the reference image as first occupied regions, generate direction vectors representing the position change of the position data of the object at a certain time interval, and generate first vectors from the direction vectors, each of which is located in one of the first occupied regions.

[0007] In one embodiment, the sampling unit can propagate the first vectors to first occupied regions among the first occupied regions where the first vectors are not located.

[0008] In one embodiment, the comparison unit may select sub-regions corresponding to the user trajectory in the reference image as second occupied regions, and generate second vectors from the user trajectory, each of which is located in one of the second occupied regions.

[0009] In one embodiment, the comparison unit can expand each of the first vectors into a set of vectors that are different in direction from the first vector and the first vector.

[0010] In one embodiment, the direction different from the direction of the first vector may be one of the quantized directions.

[0011] In one embodiment, the comparison unit performs an AND operation on the first bit value of the expanded first vector and the second bit value of the second vector at the same location, and if the bit operation value is not 0, determines that the expanded first vector includes the second vector, and determines that an object in which the ratio of the first vectors including the second vectors is greater than or equal to the user requirement ratio is an object that matches the search condition.

[0012] An object search method of an image processing device according to an embodiment of the present invention comprises: a step of obtaining position data of an object from a plurality of images; a step of generating quantized first vectors representing a change in the position of the object based on the position data of the object; a step of generating quantized second vectors representing a change in the position of a user trajectory input as a search condition and comparing the first vector and the second vector at the same position in a reference image; and a step of detecting an object moving along a trajectory that matches the user trajectory according to the comparison result.

[0013] In one embodiment, the first vectors are located in sub-regions corresponding to the position data of the object among a plurality of sub-regions constituting the reference image, and the second vectors are located in sub-regions corresponding to the user trajectory in the reference image, and each of the first vectors and the second vectors may have the size of the sub-region and one of the quantized directions.

[0014] In one embodiment, the step of generating the first vectors may include: selecting sub-regions corresponding to the position data of the object in the reference image as first occupied regions; generating direction vectors representing the position change of the position data of the object at a predetermined time interval; and generating the first vectors from the direction vectors, each of which is located in one of the first occupied regions.

[0015] In one embodiment, the step of generating the first vectors may further include the step of propagating the first vectors to the first occupied regions among the first occupied regions where the first vectors are not located.

[0016] In one embodiment, the step of comparing the first vector and the second vector may include: selecting sub-regions corresponding to the user trajectory in the reference image as second occupied regions; and generating the second vectors from the user trajectory, each of which is located in one of the second occupied regions.

[0017] In one embodiment, the step of comparing the first vector and the second vector may further include the step of expanding each of the first vectors into a set of vectors that are different in direction from the first vector and the first vector.

[0018] In one embodiment, the direction different from the direction of the first vector may be one of the quantized directions.

[0019] In one embodiment, the step of comparing the first vector and the second vector may include: a step of ANDing the first bit value of the expanded first vector and the second bit value of the second vector at the same location, and determining that the expanded first vector includes the second vector if the bit operation value is not 0; and a step of determining that an object in which the ratio of the first vectors including the second vectors is greater than or equal to the user requirement ratio is an object that matches the search condition. Effects of the invention

[0020] An image processing device according to an embodiment of the present invention can quickly search for objects satisfying search conditions while minimizing computational data. Brief explanation of the drawing

[0021] FIG. 1 is a schematic diagram showing a security system according to one embodiment. FIG. 2 is a schematic diagram showing the configuration of an image processing device according to one embodiment. FIG. 3 is a diagram showing a plurality of sub-regions constituting a reference image with a resolution of 32×16 according to one embodiment. FIG. 4 is a diagram showing bounding boxes (BBs) arranged as position data of an object in a reference image according to one embodiment. FIG. 5 is a diagram showing sub-regions where bounding boxes are located in a reference image according to one embodiment. FIG. 6 is a drawing in which sub-regions where bounding boxes are located in a reference image according to one embodiment are selected as first occupied regions (OA1). FIG. 7 is a diagram showing direction vectors generated by time sampling the bounding boxes shown in FIG. 4 according to one embodiment, together with the first occupied regions of FIG. 6. Figure 8 is an example diagram explaining the triangle method. FIG. 9 is a diagram illustrating vector direction quantization according to one embodiment. FIGS. 10a and FIGS. 10b are drawings illustrating the sampling of a direction vector according to one embodiment. FIG. 11 is a diagram showing the result of converting the direction vectors shown in FIG. 7 into first vectors according to one embodiment. FIG. 12 illustrates the result of propagating the first vectors shown in FIG. 11 to adjacent first occupied regions where the first vectors are not located, according to one embodiment. FIG. 13 is a diagram showing a user trajectory in a reference image according to one embodiment. FIG. 14 is a drawing in which sub-regions where the user trajectory is located in a reference image according to one embodiment are selected as second occupied regions. FIG. 15 is a diagram illustrating the process of selecting second occupied regions according to one embodiment. FIG. 16 is a diagram showing the result of converting the user trajectory illustrated in FIG. 13 into second vectors according to one embodiment. FIG. 17 is a diagram of the result of expanding each of the eight directional vectors according to one embodiment. FIG. 18 is a diagram showing the result of expanding the first vectors shown in FIG. 12 according to one embodiment. Figure 19a shows the bit value of each of the eight directional vectors, and Figure 19b shows the bit value of each of the dilated vectors. FIG. 20 is a diagram showing the bit values ​​of second vectors corresponding to the user trajectory of the corresponding sub-region in a reference image according to one embodiment and the first vectors that have been dilated and processed to correspond to the object trajectory. FIG. 21 is a diagram showing an example in which the second vectors of the user trajectory are included in the first vectors of the object trajectory as a result of comparison by a comparison unit according to one embodiment. FIGS. 22 to 24 are schematic diagrams illustrating a method for searching for a moving object that meets a user's search conditions in an image processing device according to one embodiment. Specific details for implementing the invention

[0022] The following description merely illustrates the principles of the present invention. Therefore, those skilled in the art may invent various devices that embody the principles of the present invention and are included within the concept and scope of the present invention, even though they are not explicitly described or illustrated in this specification. Furthermore, all conditional terms and embodiments listed in this specification should be understood as being explicitly intended, in principle, only for the purpose of understanding the concept of the present invention and not as being limited to the embodiments and conditions specifically listed as such. Additionally, all detailed descriptions enumerating specific embodiments, as well as the principles, aspects, and embodiments of the present invention, should be understood as being intended to include structural and functional equivalents thereof. Furthermore, such equivalents should be understood to include not only currently known equivalents but also equivalents to be developed in the future, that is, all elements invented to perform the same function regardless of structure.

[0023] Accordingly, the functions of the various elements depicted in the drawings, including functional blocks indicated as processors or similar concepts, may be provided by dedicated hardware as well as by the use of hardware capable of executing software in conjunction with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, a single shared processor, or multiple individual processors, some of which may be shared. Furthermore, the use of terms presented as processors, controls, or similar concepts should not be interpreted as exclusively referring to hardware capable of executing software, but should be understood to implicitly include, without limitation, digital signal processor (DSP) hardware, ROM, RAM, and non-volatile memory for storing software. Other hardware of common knowledge may also be included.

[0024] The aforementioned objectives, features, and advantages will become more apparent through the following detailed description in conjunction with the attached drawings. In describing the present invention, if it is determined that a detailed description of related prior art may unnecessarily obscure the essence of the invention, such detailed description shall be omitted or briefly explained.

[0025] Meanwhile, when it is said that a part "includes" a certain component, unless specifically stated otherwise, this means that it does not exclude other components but may include additional components.

[0026] Hereinafter, the present invention according to a preferred embodiment will be described in detail with reference to the attached drawings.

[0027] FIG. 1 is a schematic diagram showing a security system according to one embodiment.

[0028] Referring to FIG. 1, a security system (1) according to one embodiment may include a camera (100), an image processing device (200), and a user terminal (300). The camera (100), the image processing device (200), and the user terminal (300) may be connected via a wired and / or wireless network (20).

[0029] The network (20) may include wired networks such as LANs (Local Area Networks), WANs (Wide Area Networks), MANs (Metropolitan Area Networks), and ISDNs (Integrated Service Digital Networks), wireless networks such as wireless internet such as 3G, 4G (LTE), 5G, WiFi, Wibro, and WiMAX, and short-range communication such as Bluetooth, RFID (Radio Frequency Identification), infrared communication (IrDA, infrared Data Association), UWB (Ultra Wideband), ZigBee, and NFC (Near Field Communication). In the case of wireless mobile communication, the communication network may additionally include components such as a base station (BTS), a mobile switching center (MSC), a home location register (HLR), an access gateway that enables the transmission and reception of wireless packet data, and a PDSN (Packet Data Serving Node). The scope of the present invention is not limited thereto.

[0030] The camera (100) may be a single fixed camera with a fixed shooting range that is placed at a fixed position in a specific location, or a PTZ camera with pan / tilt / zoom functions. The camera (100) may be a surveillance camera including a visual camera, a thermal camera, a special purpose camera, etc. The camera (100) may include an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). The camera (100) is installed inside or outside of offices, homes, hospitals, as well as banks or public buildings requiring security, or outdoors such as parks, and may have various shapes such as a straight shape or a dome shape depending on the installation location and purpose of use.

[0031] The camera (100) can acquire an image by capturing a scene in the direction of observation. The camera (100) can capture an assigned surveillance area and transmit the captured image to an image processing device (200).

[0032] The image processing device (200) can analyze the received image, such as background detection, foreground and object detection, object counting, face detection, vehicle detection, and license plate recognition. The image processing device (200) can store the analysis results of the image as text-based metadata or image-based metadata. The image processing device (200) can structure and organize the metadata to facilitate searching and store it in a storage device.

[0033] The image processing device (200) receives search conditions from a user terminal (300), searches for metadata and images corresponding to the search conditions from a storage device, and transmits the search results to the user terminal (300). The image processing device (200) may include a server or other types of computing devices.

[0034] In the aforementioned embodiment, the image processing device (200) analyzes the image received from the camera (100) to generate metadata, but the embodiment of the present invention is not limited thereto. In another embodiment, the camera (100) analyzes the image to generate metadata and can transmit the metadata to the image processing device (200) together with the image or separately from the image. The image processing device (200) can receive the image and / or metadata from the camera (100) and store it in a storage device. The image processing device (200) can structure and organize the metadata to facilitate searching and store it in the storage device.

[0035] The user terminal (300) may refer to various devices capable of transmitting and receiving data with the image processing device (200). The user terminal (300) may be a mobile device such as a smartphone (201), PDA (202), smartphone, tablet, or handheld device, or a computer. The user terminal (300) can transmit search conditions entered by the user to the image processing device (200) and receive search results that match the search conditions from the image processing device (200).

[0036] FIG. 2 is a schematic diagram showing the configuration of an image processing device according to one embodiment.

[0037] Referring to FIG. 2, an image processing device (200) according to one embodiment may include an analysis unit (201), a sampling unit (203), a comparison unit (205), a search unit (207), and a storage unit (209).

[0038] The analysis unit (201) can analyze an image input from the camera (100), detect at least one object from the image, and generate metadata representing the attributes of each detected object. The metadata may include information such as the type, color, and location of the object, such as a person, face, or car.

[0039] A series of object location data extracted from sequentially input images can represent the object's trajectory (object trajectory). The object's location data can be acquired at fixed time intervals regardless of the direction and speed of movement from the time of the object's appearance to the time of its end. For example, the analysis unit (201) can extract the object's location data 10 times per second (10 Hz). The object's location data can be stored in the storage unit (209) in chronological order. The object's location data can be expressed as the coordinate values ​​(pixel positions) of a bounding box containing the object within the image. For example, the object's location data can be expressed as the top, bottom, left, and right coordinate values ​​of the bounding box, such as left="15.0", top="141.0", right="51.0", and bottom="291.0".

[0040] The sampling unit (203) can spatially and temporally sample the position data of an object. The sampling unit (203) can generate quantized first vectors representing changes in the position of an object based on the position data of the object. The first vectors may be located in sub-regions (hereinafter referred to as 'first occupied areas') that correspond to the position data of the object among a plurality of sub-regions constituting a reference image.

[0041] The sampling unit (203) can spatially sample the position data of an object. The sampling unit (203) can select sub-regions corresponding to the position data of an object in a reference image. The reference image may be a background image of a surveillance area. The reference image may include NxM sub-regions, and each sub-region may be an area containing KxL pixels. That is, the reference image may be understood to have an NxM sub-region resolution. The sampling unit (203) can select sub-regions (hereinafter referred to as 'first occupied areas') corresponding to the coordinate values ​​of the bounding boxes of an object in the reference image. An embodiment of the present invention converts a plurality of time-series coordinate values ​​into area data in image space, and this is referred to in this specification as spatial sampling of position data.

[0042] FIG. 3 is a diagram showing a plurality of sub-regions constituting a reference image with a resolution of 32×16 according to one embodiment. FIG. 4 is a diagram showing bounding boxes (BB) arranged as position data of an object in a reference image according to one embodiment. The arrangement order of the bounding boxes may represent an object trajectory. FIG. 5 is a diagram showing sub-regions where the bounding boxes are located in a reference image according to one embodiment. FIG. 6 is a diagram showing that the sub-regions where the bounding boxes are located in a reference image according to one embodiment are selected as first occupied regions (OA1).

[0043] The sampling unit (203) can time sample the position data of an object. The sampling unit (203) can select the position data of a second time interval from the position data of an object acquired at a first time interval. In this specification, this is referred to as time sampling of position data. The second time interval may be larger than the first time interval. For example, the sampling unit (203) can select the position data of an object at 1 Hz from the position data of an object acquired at 10 Hz. The amount of data can be reduced by time sampling.

[0044] The sampling unit (203) can generate direction vectors representing the amount of position change between selected bounding boxes among the bounding boxes of an object, which are position data selected by time sampling. The direction vector may be a vector having a direction and magnitude connecting the centers of two adjacent bounding boxes. If the object's movement speed is fast, the magnitude of the direction vector is large, and if the object's movement speed is slow, the magnitude of the direction vector is small. That is, the magnitude (length) of the direction vector may correspond to the value in which the object's time information is converted into speed. By generating a direction vector containing time information from the coordinate values ​​of bounding boxes listed in chronological order, i.e., position data, and placing it in image space, the time series data characteristics can be removed.

[0045] FIG. 7 is a diagram showing direction vectors generated by time sampling the bounding boxes shown in FIG. 4 according to one embodiment, together with the first occupied regions of FIG. 6.

[0046] The sampling unit (203) can sample direction vectors with the same resolution as spatial sampling. The sampling unit (203) can decompose or combine direction vectors to generate first vectors that are arranged continuously at sub-region intervals. The first vectors may correspond to object trajectories. By decomposing or combining the direction vectors, the number of first vectors may be greater or less than the number of direction vectors. Alternatively, the number of first vectors may be equal to the number of direction vectors.

[0047] FIG. 8 is an example diagram illustrating the triangle method. FIG. 9 is a diagram illustrating the quantization of vector directions according to one embodiment. FIG. 10a and FIG. 10b are diagrams illustrating the sampling of direction vectors according to one embodiment.

[0048] As illustrated in FIG. 8, a single vector can be expressed as the sum of multiple vectors or the difference of multiple vectors by the triangulation method. The sampling unit (203) can generate first vectors by using the triangulation method to decompose a single direction vector into two or more vectors along the direction of movement of an object, or by combining two or more direction vectors into a single vector. The sampling unit (203) can normalize the magnitude (length) of the first vector to the sub-region size and quantize the direction of the first vector into eight directions. Accordingly, each of the first vectors is located corresponding to one of the first occupied regions and may have one of the eight directions illustrated in FIG. 9. The sub-region size may be a size corresponding to the horizontal length, vertical length, or diagonal length of the sub-region depending on the direction.

[0049] The sampling unit (203) can select a first occupied area where the first vector is located and the direction of the first vector so that the total length of the first vectors is the shortest length. For example, as shown in FIG. 10a, one direction vector (A) can be converted into six first vectors (B), or as shown in FIG. 10b, eight direction vectors (A) can be converted into six first vectors (B). As shown in FIG. 10a and FIG. 10b, each of the first vectors (B) is located in one of the first occupied areas and can have one of eight directions. The first vectors (B) can be arranged adjacently (continuously) along the direction of the direction vector.

[0050] FIG. 11 is a diagram showing the result of converting the direction vectors illustrated in FIG. 7 into first vectors (VT1) according to one embodiment. The first vectors are not located in some of the first occupied regions.

[0051] The sampling unit (203) can propagate first vectors located in a first occupied area to adjacent first occupied areas where the first vectors are not located. Accordingly, data loss of the vector collection can be prevented. FIG. 12 illustrates the result of propagating the first vectors shown in FIG. 11 to adjacent first occupied areas where the first vectors are not located. The sampling unit (203) can store information about the first occupied areas and first vectors for each object in the storage unit (209).

[0052] The search unit (207) receives search conditions from the user terminal (300), detects objects that match the search conditions, and can transmit the object detection results to the user terminal (300).

[0053] The user can input search conditions by using an input device to sketch a trajectory using a line (straight line, curve, etc.) or a shape, etc., on an image provided to the user terminal (300). The image provided to the user terminal (300) may be a live image or a reference image.

[0054] The search unit (207) can transmit condition data corresponding to the search condition to the comparison unit (205). The condition data may be location data (hereinafter referred to as "user trajectory data") of a trajectory designated by the user (hereinafter referred to as "user trajectory"). The user trajectory data may represent a change in spatial location between the starting point and the ending point of the user trajectory.

[0055] The comparison unit (205) receives user trajectory data and can compare the user trajectory data with the object's location data. The comparison unit (205) can generate quantized second vectors representing the location change of the user trajectory input as a search condition. The comparison unit (205) can generate second vectors corresponding to the user trajectory by spatially sampling the user trajectory data. The comparison unit (205) can compare the second vector and the first vector at the same location in the reference image.

[0056] The comparison unit (205) can spatially sample user trajectory data by selecting sub-regions where the user trajectory is located in the reference image. The comparison unit (205) can select sub-regions corresponding to the user trajectory (hereinafter referred to as 'second occupied areas') in the reference image illustrated in FIG. 3.

[0057] FIG. 13 is a diagram showing a user trajectory in a reference image according to one embodiment. FIG. 14 is a diagram showing sub-regions where the user trajectory is located in a reference image according to one embodiment being selected as second occupied regions (OA2). FIG. 15 is a diagram explaining the process of selecting the second occupied regions (OA2) according to one embodiment.

[0058] When selecting a second occupied area, the comparison unit (205) may select one sub-area along the direction of the user trajectory from among the adjacent sub-areas centered on each of the second occupied areas as the second occupied area. For example, as shown in FIG. 15(a), among the eight surrounding sub-areas centered on the second occupied area (OA21) at the location where the user trajectory starts, the sub-area closest to the location and direction of the user trajectory may be selected as the next second occupied area (OA22). Subsequently, as shown in FIG. 15(b), among the eight surrounding sub-areas centered on the second occupied area (OA22), the sub-area closest to the location and direction of the user trajectory may be selected as the next second occupied area (OA23). Next, as illustrated in FIG. 15(c), among the eight surrounding sub-regions centered on the second occupied area (OA23), the sub-region closest to the position and direction of the user trajectory can be selected as the next second occupied area (OA24). The comparison unit (205) can select the second occupied area in a similar manner up to the end position of the user trajectory.

[0059] The comparison unit (205) can generate second vectors corresponding to the user trajectory from the user trajectory data. The comparison unit (205) can normalize the magnitude (length) of the second vector to the sub-region size and quantize the direction of the second vector into eight directions. Accordingly, each of the second vectors is located corresponding to one of the second occupied regions and can have one of the eight directions shown in FIG. 9. FIG. 16 is a diagram showing the result of converting the user trajectory shown in FIG. 13 into second vectors (VT2) according to one embodiment.

[0060] The comparison unit (205) can compare the user trajectory with the object trajectory. The comparison unit (205) can compare the second vectors of the user trajectory with the first vectors of the object trajectory. The comparison unit (205) can expand the direction of each of the first vectors of the object trajectory for comparison. The comparison unit (205) can expand each of the first vectors into a set of vectors that are different from the first vector and the direction of the first vector. By expanding the first vector, the comparison set to be compared with the user trajectory can be expanded.

[0061] FIG. 17 is a diagram showing the result of expanding each of the eight directional vectors according to one embodiment. As shown in FIG. 17, each of the eight directional vectors can be expanded into a set of vectors in the main direction and vectors in adjacent directions 45 degrees to the left and right of the main direction. FIG. 18 is a diagram showing the result of expanding the first vectors shown in FIG. 12 according to one embodiment.

[0062] The comparison unit (205) can perform an AND bitwise operation on the second bit values ​​of the second vectors of the user trajectory at the same location in the reference image and the first bit values ​​of the first vectors of the dilated object trajectory.

[0063] FIG. 19a shows the bit values ​​of each of the eight directional vectors, and FIG. 19b shows the bit values ​​of each of the dilated vectors. The bit values ​​can be determined according to the direction of the vector. The number of bits is not specifically limited, and FIG. 19a and FIG. 19b are examples in which the bit values ​​of the dilated first vector and second vector are represented as 8 bits.

[0064] FIG. 20 is a diagram showing the bit values ​​of second vectors corresponding to the user trajectory of a corresponding sub-region in a reference image according to one embodiment and the dilated first vectors corresponding to the object trajectory. Sub-regions where the first vector and the second vector are not located have a bit value of 0.

[0065] The comparison unit (205) can determine that the first vector that has been expanded includes the second vector if the bit operation result is not 0. The comparison unit (205) can determine that the object is an object that matches the search condition if the ratio of the second vector that has been expanded to the first vector that has been expanded is greater than or equal to the ratio requested by the user (e.g., 80%, 90%, 100%, etc.).

[0066] FIG. 21 shows an example in which the second vectors of the user trajectory are included in the first vectors of the object trajectory as a result of comparison by a comparison unit (205) according to one embodiment.

[0067] The search unit (207) can detect at least one object satisfying the search conditions. The search unit (207) can transmit the object detection result to the user terminal (300).

[0068] FIGS. 22 to 24 are schematic diagrams illustrating a method for searching for a moving object that meets a user's search conditions in an image processing device according to one embodiment. The method for searching for a moving object according to one embodiment can be performed by the monitoring device (200) shown in FIG. 1. Hereinafter, detailed descriptions of contents that overlap with those described with reference to FIGS. 1 to 21 are omitted.

[0069] Referring to FIG. 22, the image processing device (200) receives an image from the camera (100) (S21) and can store the received image.

[0070] The image processing device (200) can analyze an image to detect an object and generate object location sampling information by sampling the object's location data (S23). A method for generating object location sampling information will be explained below with reference to FIG. 23.

[0071] The image processing device (200) can extract location data of an object detected from an image (S231). The image processing device (200) can extract location data of an object from each of the images input in chronological order. The input images may be images of the same surveillance area.

[0072] The image processing device (200) can spatially sample the position data of an object. The image processing device (200) can select sub-regions corresponding to the position data of an object in a reference image of NxM sub-region resolution composed of sub-regions of a predetermined size as first occupied regions (S233).

[0073] The image processing device (200) can time-sample the position data of an object and generate direction vectors from the time-sampled position data (S235). Each direction vector may have a magnitude and direction that includes velocity information. Accordingly, the time information of the position data can be represented as a vector in image space.

[0074] The image processing device (200) can generate first vectors corresponding to object trajectories by sampling direction vectors at a resolution equal to the NxM sub-region resolution (S237). The image processing device (200) can generate first vectors having one of eight quantized directions and the size of a sub-region by decomposing or synthesizing the direction vectors. The image processing device (200) can generate first vectors corresponding to all first occupied regions by propagating the first vectors to adjacent first occupied regions where the first vectors are not placed.

[0075] The image processing device (200) can receive search conditions including user trajectories from the user terminal (300) (S25).

[0076] The image processing device (200) can generate user trajectory sampling information by sampling user trajectory data and compare it with object position sampling information (S27). A method for generating and comparing user trajectory sampling information will be explained below with reference to FIG. 24.

[0077] The image processing device (200) can spatially sample user trajectory data. The image processing device (200) can select sub-regions corresponding to the user trajectory in a reference image of NxM sub-region resolution composed of sub-regions of a predetermined size as second occupied regions (S271).

[0078] The image processing device (200) can decompose the user trajectory into second vectors having one of eight quantized directions and the size of a sub-region (S273). Each of the second vectors can be placed in second occupied regions.

[0079] The image processing device (200) can expand the direction of the first vectors by expanding the first vectors of the object trajectory (S275). Each of the expanded first vectors can be represented as a set of a main direction vector and adjacent direction vectors as shown in FIG. 17.

[0080] The image processing device (200) can compare the second bit value of the second vector and the first bit value of the expanded first vector in corresponding sub-regions by performing an AND bitwise operation (S277). If the result of the bitwise operation is not 0, the image processing device (200) can determine that the expanded first vector contains the second vector.

[0081] The image processing device (200) can determine that an object is an object that matches the search condition if the ratio of the second vector, which is the result of bit operation, to the first vector that has been expanded is greater than or equal to the ratio required by the user (S279).

[0082] The image processing device (200) can search for at least one object satisfying the search conditions and transmit the object search result to the user terminal (200) (S29).

[0083] Object search can be performed by conducting intelligent video analysis in advance to store the characteristics and locations of each object, and by calculating and recording in advance whether they satisfy predefined rules (line crossing, area entry). Another method of object search involves the user arbitrarily generating trajectory rules (right turn, U-turn) afterward and searching for objects that satisfy those rules. Such object search methods face a computational overload problem because they require reading a large amount of moving object location data from a storage module and filtering the massive amount of data through calculations.

[0084] An embodiment of the present invention can reduce the effort required by a user to find desired video content by using a trajectory as a search condition, and by sampling position change information of a moving object to generate and store summary information, and by providing fast search using the summary information, the required data storage capacity and the data computation requirement can be minimized, thereby increasing the utility of trajectory search.

[0085] Embodiments of the present invention can reorganize each data into a data model in the form of an individually independent set of vectors by eliminating the time constraints, sequential constraints, directional constraints, velocity (vector length) constraints, and spatial positional proximity calculation constraints inherent in conventional trajectory data comparison calculations. Along with the reorganization of the data model, user-requested trajectories are minimized and object trajectories are maximized, thereby ensuring that if two actual trajectories are similar, the user-requested trajectory is included in the object trajectory. Through this, storage capacity and computational load can be improved by radically changing the calculation method to a computer-specific bitwise logic approach rather than mathematical optimization such as existing computational or numerical analysis methods.

[0086] The object search method according to the present invention can be implemented as computer-readable code on a computer-readable recording medium. A computer-readable recording medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices. Furthermore, the computer-readable recording medium may be distributed across networked computer systems, allowing computer-readable code to be stored and executed in a distributed manner. Additionally, functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers skilled in the art to which the present invention belongs.

[0087] Although the present invention has been described with reference to an embodiment illustrated in the accompanying drawings, this is merely illustrative, and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true scope of protection of the present invention should be determined only by the appended claims.

Claims

Claim 1 An image processing device comprising: an analysis unit for acquiring position data of an object from a plurality of images; a sampling unit for generating quantized first vectors representing changes in the position of the object based on the position data of the object; a comparison unit for generating quantized second vectors representing changes in the position of a user trajectory input as a search condition and comparing the first vector and the second vector at the same position in a reference image; and a search unit for detecting an object moving along a trajectory that matches the user trajectory according to the comparison result of the first vector and the second vector. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 delete Claim 7 delete Claim 8 delete Claim 9 A method for searching for an object in an image processing device, comprising: a step of acquiring position data of an object from a plurality of images; a step of generating quantized first vectors representing changes in the position of the object based on the position data of the object; a step of generating quantized second vectors representing changes in the position of a user trajectory input as a search condition and comparing the first vector and the second vector at the same position in a reference image; and a step of detecting an object moving along a trajectory that matches the user trajectory according to the result of comparing the first vector and the second vector. Claim 10 delete Claim 11 delete Claim 12 delete Claim 13 delete Claim 14 delete Claim 15 delete Claim 16 delete

Images