IMAGE RETRIEVING DEVICE

Abstract

The image retrieval device according to the disclosed technology is an image retrieval device comprising a feature reduction processing circuit (100) and a retrieval processing unit (200). A matrix calculator (120) included in the feature reduction processing circuit (100) calculates a matrix (C d ) based on stored data (X d-1 ). A projection transformation section (130) included in the feature reduction processing circuit (100) multiplies the matrix (C d ) with a sequentially acquired image feature set {f i}. A second projection transformation section (210) included in the retrieval processing unit (200) calculates a low-dimensional image feature (g target_x ) from the input image feature (f target ) of the search target with the matrix (C d). A search execution section (220) included in the retrieval processing unit (200) calculates distances between the vectors calculated by the projection transformation section (130) and the low-dimensional image feature (g target_x ) in the feature space and performs a search for plausible image features using the distances. The vectors calculated by the projection transformation section (130) belong to a time-window-divided database.

Description

[TECHNICAL FIELD]The present disclosure relates to an image pickup apparatus.[BACKGROUND OF THE PRIOR ART]In the field of image search, it is known to reduce dimensionality of the image feature amount using a matrix obtained from statistical analysis to enable high-speed search. The statistical analysis here refers to, for example, the principal component analysis and the discriminant analysis. Here, the image feature amount is also a concept used in the technical field of machine learning, and is a vector in the feature space generated from the image. An image feature, as the name suggests, represents the characteristics of an image.For example, Patent Document 1 discloses a technique for facilitating classification problems by converting the image features by the first linear conversion parameter, wherein the first linear conversion parameter is a matrix obtained from the result of statistical analysis.[REFERENCE LIST][PATENT LITERATURE][PTL 1] WO 2014 / 167880 A1[SUMMARY OF THE INVENTION][TECHNICAL PROBLEM]The image pickup apparatus shown in Patent Document 1 can be used as a system for finding lost children or suspect persons from video images of security cameras in facilities such as airports. In this application, a system for finding lost or suspect persons from video images captured by security cameras in facilities such as airports is referred to as a "retrieval system.".The inventor of the disclosed technology found that when the image retrieval apparatus shown in Patent Document 1 is applied to the retrieval system, the matrices suitable for dimensionality reduction gradually vary when viewed in a time window in units of day, even if it is an image derived from the same surveillance camera.An object of the disclosed technology is to provide an image fetch apparatus that solves the problem that matrices suitable for use in dimensionality reduction gradually change.[SOLUTION TO PROBLEM]The image fetch apparatus according to the disclosed technology is an image fetch apparatus including a feature reduction processing circuit and a fetch processing unit.A matrix calculator included in the feature reduction processing circuit calculates a matrix (C d) based on stored data (X d-1).A projection transformation section included in the feature reduction processing circuit multiplies the matrix (C d) by a sequentially acquired image feature amount {f i}.A second projection transformation section included in the retrieval processing unit calculates a low-dimensionallyized image feature (g target_x) from the input image feature (f target) of the search target with the matrix (C d).A search execution section included in the retrieval processing unit calculates distances between the vectors calculated by the projection transformation section and the low-dimensionallyized image feature (g target_x) in the feature space, and performs a search for plausible image features using the distances. The vectors calculated by the projection transformation section belong to a time-window-divided database.ADVANTAGEOUS EFFECTS OF THE INVENTION]Since the image fetch apparatus according to the disclosed technology has the above configuration, the problem of the matrices suitable for use in dimensionality reduction and gradually changing can be solved.[BRIEF DESCRIPTION OF THE DRAWINGS]FIG. 1 is a block diagram showing the functional configuration of the feature reduction processing circuit 100 according to the disclosed technology. FIG. 2 is a timing diagram showing the process of the feature reduction processing circuit 100 according to the disclosed technology in the form of a time series. FIG. 3 is a block diagram showing the functional configuration of the retrieval processing unit 200.[DESCRIPTION OF EMBODIMENTS]As described above, the image feature amount that is a premise of the disclosed technology is a concept used in the technical field of machine learning, and is a vector amount in the feature space generated for the image. The disclosed technology is not limited to what type of learning model the image feature amount is generated from, or even how the image feature amount is generated. However, the image feature is easily understood when it is regarded as an intermediate of a learning model such as CNN (Convolutional Neural Network). CNN is a type of artificial neural network that is recognized as an effective means, particularly in the field of image analysis technology.Embodiment 1.The image retrieval apparatus according to the disclosed technology includes a feature reduction processing circuit 100 and a retrieval processing unit 200.The feature reduction processing circuit 100 is a processing circuit that acquires the image feature amount and performs a feature reduction process such as dimension reduction.The retrieval processing unit 200 is a processing circuit that performs a search for plausible image features from a database divided for each period of time (hereinafter, referred to as "time window") when the image feature to be searched is input.FIG. 1 is a block diagram showing the functional configuration of the feature reduction processing circuit 100 according to the disclosed technology. As illustrated in FIG. 1, the feature reduction processing circuit 100 includes a data storage unit 110, a matrix calculator 120, and a projection transformation section 130.FIG. 2 is a timing diagram showing the process of the feature reduction processing circuit 100 according to the disclosed technology in the form of a time series. In FIG. 2, the part indicated as ST 110 represents the time at which the processing by the data storage unit 110 is performed. Similarly, the part indicated as ST120 indicates the time at which the processing is performed by the matrix calculator 120. Similarly, the part indicated as ST 130 also indicates the time at which the processing by the projection transformation section 130 is performed.(Data Storage Unit 110 Included in Feature Reduction Processing Circuit 100)The data storage unit 110 included in the feature reduction processing circuit 100 is a component that stores sequentially acquired image features as data. It is assumed that the image feature amount to be acquired sequentially is expressed as {f i} i=1, 2,..., m. The data (X) to be stored is expressed by the following equation.As shown in Equation 1, the sequentially acquired image feature amount {f i} is a real vector having a size of 1 xn. The image feature set {f i} is defined herein as a line vector. As shown in Equation 1, the image feature is generally a real vector with real numbers as components, but the definition can also be extended to complex vectors with complex numbers as components.When the total number of the image feature amount {f i}, which is sequentially acquired in a certain time window, e.g., d-1st time window, is m, the data (X) stored in the data storage unit 110 is a matrix having a size of m×n. As illustrated in Equation 1, the data (X) is generally a real matrix. However, when the definition of the image feature amount {f i} is expanded to a complex vector, the data (X) becomes a complex matrix.To emphasize that the data (X) is the data of the (d-1) -ten time window, "d-1" is added in the form of a lower right index and is denoted by "X d-1". In the following, the database composed of the data of the d-th time window is also referred to as a "d-th database".(Matrix Calculator 120 Included in Feature Reduction Processing Circuit 100)The matrix calculator 120 included in the feature reduction processing circuit 100 is a component that calculates a matrix (C d) for reducing the feature amounts based on the data (X d-1) of the (d-1) -ten time window. As shown in FIG. 2, the matrix (C d) calculated by the matrix calculator 120 is used for the arithmetic processing to sequentially reduce the feature amount of the image feature amount {f i}, which has been acquired at a time point belonging to the d-th time window. An important technical feature of the image fetch apparatus according to the disclosed technology is that C d, the matrix used for the arithmetic processing with respect to the d-th time window, is calculated based on the data (X d-1) with respect to the d-1st time window. Note that the letter "C" used in "C d" expressing the feature amount reduction matrix originates from the acronym for "Compression".The matrix (C d) calculated by the matrix calculator 120 may be, for example, a matrix obtained from the results of statistical analysis such as the principal component analysis and the discriminant analysis as shown in Patent Document 1. The matrix (C d) calculated by the matrix computer 120 may be, in particular, a matrix obtained on the basis of singular value decomposition (in the following, singular value decomposition is referred to as "SVC" for singular value decomposition). It is assumed that the SVC of the data (X d-1), belonging to the (d-1) -ten time window, is as shown in the following equation."U" in Equation 2 is a matrix consisting of the left singular vectors {u i}, i=1, 2,..., r. Also, "V" in Equation 2 is a matrix consisting of the right singular vectors {v i}, i=1, 2,..., r. "T" in the upper right index means the transposition operation. "S" in Equation 2 is a diagonal matrix in which the singular values are arranged in the order of their size. "m" is the total number of data lines (X d-1), belonging to the (d-1) -ten time window, and the total number of acquired image feature amounts {f i}. " n" is the total number of data columns (X d-1), belonging to the (d-1) -ten time window, and the number of elements in each image feature amount is {f i}. It is assumed that "m" is larger than "n". The "r" in the lower right index in Equation 2 is an acronym for "rank", representing the rank of X d-1. If X d-1 has a full column rank, then "r" is equal to "n". The fact that the rank of X d-1 is "r" is equivalent to the fact that the dimension of the space spanned by the vectors {f i}, i=1, 2,..., m is "r".The matrices U and V composed of singular vectors have the following properties.I(r) on the right side of Equation 3 is an identity matrix with a magnitude of rxr.When X d-1 is not a zero matrix, the general inverse matrix of X d-1 may be defined using the result of singular value decomposition (hereinafter simply referred to as "SVD") according to Equation 2 and the characteristics indicated in Equation 3. According to the definition of the general inverse of the Moore-Penrose type, the general inverse matrix of X d-1 is given by the following equation.If X d-1 is a regular matrix, then the general inverse matrix given by equation 4 is equivalent to the inverse matrix of X d-1.The subspace of R n, spanned by the row vectors {f i}, i=1,2,...,m, is referred to as the row domain of X d-1 "row domain" is sometimes also referred to as "row space", but in this document the name "row domain" is used. The right singular vectors {v i}, i=1,2,..., r transposed are the normal orthogonal base of the row domain of X d-1.One aspect of the image retrieval apparatus according to the disclosed technology employs a projection matrix that projects vectors onto the row domain of X d-1 onto the interrogation system. The projection matrix (P v) is given by the following equation. P v in Equation 5 is the projection matrix having a size of n×n and a rank r. Note that VV T in Equation 5 is different from V T V in Equation 3.Hereinafter, the multiplication from the right is referred to as "right multiplication". Assume that there is an image feature amount of the image to be interrogated, namely "f target ". By right multiplication of the projection matrix (P v) f target can be projected onto the row domain of X d-1. In the retrieval system, the ability to restrict the search range to the row domain of X d-1 may contribute to shortening the processing time. It should be noted that the letter "P" representing the projection matrix originates from the acronym for "Projection".By right multiplying V on each side of the equation and applying the properties shown in equation 3, equation 2 can be transformed into the following equation.Here, {g i}, i=1, 2,..., m is an image feature amount whose dimension has been reduced (hereinafter referred to as "low-dimensioned image feature").From equation 6, the following facts can be derived. If a particular row vector (f x) belongs to the row domain of X d-1 i.e. f x can be expressed by a linear combination of {f i}, i=1, 2,..., m, then the resulting vector obtained by right multiplying "V" by the row vector (f x) can be expressed by a linear combination of {g i}, i=1, 2,..., m.Here, each {g i}, i=1, 2,..., m is a row vector having a size of 1×r. Since f x V is the linear combination of {g i} f x V is a row vector having a size of 1xr. {a i}, i=1, 2,..., m is each a coefficient.It is assumed that there is another row vector f y. It is assumed that f y does not belong to the row domain of X d-1 unlike f x. For f y, which does not belong to the row domain, a projection onto the row domain could be considered by right multiplying the projection matrix given in equation 5 (P v). Since the projected vector (f y P v) belongs to the line domain, f y P v can be represented as f x. The above-described process can be expressed as follows.Finally, the resulting vector f y V is also expressable for f y, which does not belong to the row domain, by right multiplication only of "V" always by the linear combination of {g i}, i=1,2,..., m (see equation 8).As described above, the matrix (C d) may be "V" calculated by the matrix calculator 120, which is the matrix composed of singular vectors. If X d-1 has a full column rank and "r" is "n", the multiplication of "V" does not help to change the size of the vector. In such a case, a method of dimensionality reduction by an approximation in which a singular value close to 0 is set to 0 is considered. Details of this method are shown in Embodiment 2.(Projection transformation section 130 included in feature reduction processing circuit 100)A projection transformation section 130 included in the feature reduction processing circuit 100 is a component that multiplies the matrix (C d) by a sequentially acquired image feature amount {f i}. C d is the matrix calculated by the matrix calculator 120. The image feature set {f i} is defined here as a line vector, respectively. Therefore, the multiplication of the matrix (C d) is a right multiplication by the image feature amount {f i}, which gives f i C d. When C d is considered to be "V", the size of C d is n×r(r<n). That is, the projection transformation section 130 is a component that converts the image feature amount {f i} into a low-dimensionallyized image feature {g i}.Although the low-dimensionallyized image feature {g i} is calculated as "V" in Equation 6, the disclosed technology is not limited thereto. Hereinafter, each low-dimensionallyized image feature calculated by the projection transformation section 130, which may be the result of multiplication by a matrix (C d) other than "V", is referred to as {g i}.As shown in FIG. 1, the low-dimensionallyized image feature {g i} generated by the projection transformation section 130 is stored in a storage device. The image fetch apparatus is accessible to the storage means. The low-dimensionallyized image feature {g i} is stored in a database for each time window. The matrix (C d) used for multiplication by the projection transformation section 130 is also used in the retrieval processing unit 200 of the image retrieval apparatus. Therefore, the matrix (C d) is shared between the feature reduction processing circuit 100 and the retrieval processing unit 200.FIG. 3 is a block diagram showing the functional configuration of the retrieval processing unit 200. The feature reduction processing circuit 100 of the image fetch apparatus is a component used in the phase of collecting image data. In contrast, the retrieval processing unit 200 of the image retrieval apparatus is a component used in the phase of image search. In explaining the processing contents of the image retrieval apparatus according to the disclosed technology, the process is separately defined as "image data collection phase" and "image searching phase". However, in situations where an image fetch apparatus is put into practice, both phases may run simultaneously. That is, the image may be acquired from the acquisition processing unit 200 in parallel with the collection of image data by the feature reduction processing circuit 100.The "f target" shown in FIG. 3 represents the image feature amount of the image to be searched. If the image retrieval device is a retrieval system, "f target" is the image feature for the retrieved image.As shown in FIG. 3, the retrieval processing unit 200 includes a second projection transformation section 210 ( 210- 0, 210- 1,..., 210- x,...) and a search execution section 220 ( 220- 0, 220- 1,..., 220- x,...). As shown in FIG. 3, the function block having the reference of "-0" appended at the end is a function block that manages a database concerning the d-th time window. Similarly, the function block having the reference "-1" appended at the end is a function block that manages a database concerning the (d-1) -te time slot. In addition, the function block having the reference with "-x" appended at the end is, as a generalized expression, a function block that manages a database concerning the (d-x) -te time window. How many databases search the retrieval processing unit 200 is a question of the design of the image retrieval device. This question can be determined based on the purpose of use and the specifications of the image fetch apparatus.(Second Projection Transformation Section 210 Included in Retrieval Processing Unit 200)The second projection transformation section 210 included in the retrieval processing unit 200 is a component that calculates the low-dimensionallyized image feature for the input f target. It can be determined that the second projection transformation section 210 corresponds to the projection transformation section 130 of the feature reduction processing circuit 100. As mentioned above, the matrices (C d, C d-1, C d-2,..., C d-x,...) used for multiplication by the projection transformation section 130 are used together with the second projection transformation section 210 of the retrieval processing unit 200.The low-dimensionallyized image feature (g target_x), which is calculated in the second projection transformation section 210, is given by the following equation.When the matrix (C d-x) used for multiplication by the projection transformation section 130 is "V", the matrix (C d-x) used for multiplication by the second projection transformation section 210 is also "V".(Search Execution Section 220 Included in Retrieval Processing Unit 200)The search execution section 220 included in the retrieval processing unit 200 is a component that searches for plausible image features from the databases of different time slots based on the low-dimensionallyized image feature (g target_x), calculated in the second projection transformation section 210. As used herein, the term "search" is the same as used in computer technology and refers to a process for retrieving desired data or determining the storage location.Specifically, the search execution section 220 calculates a distance in the feature space and extracts a plausible image feature on the basis of the size of the calculated distance. The distance that the search execution section 220 calculates is a distance used as an index for the evaluation of similarity, such as the Euclidean distance, the Mahalanobis distance, the Bhattacharyya distance, and the like.As shown in Equations 2 and 6, when using SVC in the order of increasing singularity values, the low-dimensionallyized image feature (g target_x) is generated such that the most important element is the first component and the next more important element is the second component (the same applies hereinafter). Thus, the search execution section 220 can calculate distances and extract candidates for plausible image features by using only the first component or using only the first k components, for example (k is a natural number greater than 1 and less than r). Alternatively, for example, the search execution section 220 may calculate the distance using only the first component or only the first k components, and perform filtering that separates those to be considered for the search from those to be not considered for the search.The idea of calculating the distance using only the first k components is common to the method of low-dimensionallyized representation, which represents an approximation in SVD form by replacing the singular value close to 0 with 0. As described above, in Embodiment 2, a method of dimensionality reduction by an approximation in which a singular value close to 0 is set to 0 is shown.The search performed by the search executing section 220 is achieved by calculating the distance between the low-dimensionallyized image feature (g target_x) of the search target and the low-dimensionallyized image feature {g i}, which is stored in the database. Note that it is not necessary to know the distance for all of the low-dimensionallyized image features {g i}.The value of each component of the low-dimensionallyized image feature to be searched (g target_x) and the value of each component of the low-dimensionallyized image feature {g i} of the database can be expressed in binary numbers. Binary notation helps to confine the candidates for the search (the confinement of the candidates is referred to as "filtering"). Specifically, the search performed by the search executing section 220 checks whether or not the two data represented by binary numbers coincide, starting with the higher-order bits. Comparing the first component of the low-dimensionallyized image feature to be searched (g tar-get_x) and the first component of a particular low-dimensionallyized image feature {g i}, focused on if the top bits do not match each other, that pair is unlikely to have the shortest distance. In this way, data in which the top bits do not match each other can be excluded from the search candidates. After filtering by the top bit, similar filtering by the second top bit is performed. This search method is conceptually similar to the search method called dichotomy.Note that the above-mentioned method is not limited to binary numbers. A method of checking whether or not two data expressed in the general N-position notation coincide in the order of the upper digits may also be regarded as filtering in the search.As described above, the process of retrieval by the retrieval processing unit 200 may be executed in parallel with the collection of the image data by the feature reduction processing circuit 100. For example, when the current time belongs to the d-th time window, the search execution section 220 may perform a search in the d-th database in parallel with the image data collected by the feature reduction processing circuit 100 in the d-th database. As illustrated in FIG. 2, the process ST 120 in the d-th time window may be executed only when all the processes ST 110 in the d-th time window are completed. Here, ST 110 is the process of the data storage unit 110, and ST 120 is the process of the matrix calculator 120. Therefore, the image fetch apparatus according to the conventional concept can perform the projection transformation processing (ST 130) by the projection transformation section 130 only when the data storage unit 110 has completed the entire processing (ST 110) of acquiring the image feature. That is, according to the conventional way of thinking, it is not possible to perform a search in the database of the d-th time window in parallel with the collection of the image data in the d-th time window.The use of a fixed matrix (C) to generate a low-dimensionallyized image feature is conceivable. As already mentioned, however, the matrices suitable for dimensionality reduction gradually change when viewed in a time window of units of a day, even if it is an image originating from the same surveillance camera. That is, over time, low-dimensionality through a fixed matrix (C) is becoming more and more unsuitable for interrogation systems.One of the excellent effects of the image fetch apparatus according to Embodiment 1 is that the projection transformation section 130 can perform projection transformation sequential processing (ST 130) even before the data storage unit 110 has completed the entire processing (ST 110) of acquiring image features.Another excellent effect of the image retrieval apparatus according to Embodiment 1 is that the search execution section 220 can perform a search in the d-th database in parallel with the image data collected in the d-th database by the feature reduction processing circuit 100.In summary, the excellent effect of the image retrieval apparatus according to Embodiment 1 is that it solves the problem of "variation of matrices suitable for dimensionality reduction" and can perform high-speed searches that target multiple databases relating to different time slots.These effects are obtained by actively utilizing the property that, for example, when a time window is set by one day unit, the matrix suitable for dimensionality reduction is changed stepwise.Focusing on the characteristic that the change is stepwise, the image fetch apparatus according to Embodiment 1 uses the matrix C d for the projection on the d-th time window, where C d is the calculated result based on the data (X d_1) with respect to the (d-1)-th time window. Due to this technical feature, the image pickup apparatus according to Embodiment 1 has the excellent effects described above.Embodiment 2.An image fetch apparatus according to Embodiment 2 is a modification of the image fetch apparatus according to the disclosed technology. The image fetch apparatus according to Embodiment 2 corresponds in functional configuration to the image fetch apparatus according to Embodiment 1. Therefore, the same reference numerals as in Embodiment 1 are used in Embodiment 2, unless otherwise specified.It can be noted that the image fetch apparatus according to Embodiment 2 is an embodiment that embodies a matrix calculated by the matrix calculator 120. The matrix calculated by the matrix calculator 120 according to Embodiment 2 is based on an approximation in SVD form. The approximation in SVD form is an approximation that replaces the singular value near 0 with 0.The SVD represented in Equation 2 can be approximated by the following equation.The right side of Equation 10 is the result of replacing the (k+1) -ten through the r -ten singular value with 0 (zero), respectively. Note that the singular values in "S" are arranged in the order of their size.The diagonal matrix on the right of Equation 10 is represented by "S" with the accent sign Tilde above. The accent sign Tild means that the "S" with the accent sign Tild is similar to the original "S" on the left side of Equation 10. Equation 10 represents the approximation in the SVD form.As shown in Equation 10, "k" is a positive integer less than the rank "r".The right side of Equation 10 may be deformed as shown by the following equation.Here, "true" in the lower right index on the right of Equation 11 derives from the word "true" (dt. Shortening) away. U trunc is a matrix of size m×k. S trunc is a diagonal matrix of size kxk. V trunc is a matrix of size n×k.In the image fetch apparatus according to Embodiment 2, V trunc of Equation 11 is used as the matrix calculated by the matrix calculator 120. Thus, in the image fetch apparatus according to Embodiment 2, the projection transformation section 130 and the second projection transformation section 210 perform projections using V trunc.The effect of using V trunc as projection is the same as the effect of using only the first k components of the low-dimensionallyized image feature (g target_x) to calculate the distance disclosed in Embodiment 1.The particular effect of the image retrieval apparatus according to Embodiment 2 is that the dimension of the image feature (f target), which relates to the image to be searched, can be reduced from n to k even when X d-1 has a full column rank and "r" is "n".By this measure, the image fetch apparatus of Embodiment 2 solves the problem in a manner described in Embodiment 1. That is, the image retrieval apparatus according to Embodiment 2 solves the problem of "variation of matrices suitable for dimensionality reduction" and can perform high-speed searches that target multiple databases relating to different time slots.Embodiment 3.An image fetch apparatus according to Embodiment 3 is a variant of the image fetch apparatus according to the disclosed technology.Unless otherwise specified, the same reference numerals are used in Embodiment 3 as in the aforementioned embodiments. In Embodiment 3, the description overlapping with the above-described embodiment is omitted as appropriate.As described above, the technical feature of the image fetch apparatus according to Embodiments 1 and 2 is that C d, the matrix used for the arithmetic processing with respect to the d-th time window, is calculated based on the data (X d-1) with respect to the d-1st time window. Thus, in the image fetch apparatus according to Embodiments 1 and 2, data (X d-1) relating to the (d-1) -te time window, which is the adjacent most recent past time window, is used.The image fetch apparatus according to Embodiment 3 generalizes the part "adjacent more recent past" to "empirically a past in which the characteristics of the image data can be predicted to be similar.". In other words, when viewed in a daily time window, the adjacent recent past is a concrete example of "a past in which the characteristics of the image data can be predicted to be similar according to experience.". From a technical point of view, it is not necessarily required that the image data be "adjacent to the immediate past", but it is important that "it can be empirically predicted that the characteristics of the image data will be similar".The database used by the image fetch apparatus is not limited to a database in which the time slot is a unit of "one day". For example, it may be necessary to create a database in a time window in units of 6 hours, such as 0 to 6 o'clock, 6 to 12 o'clock, 12 to 18 o'clock, 18 to 24 o'clock. It is assumed that the current time is included in the time window "12 to 18 o'clock". At this time, when it is known from experience that the characteristics of the image data relating to this time window are similar to those of "12 to 18 o'clock" of gester, the image fetch apparatus according to Embodiment 3 can calculate a C d using the data "12 to 18 o'clock" of gester.As shown in the above example, the characteristics of image data related to humans may periodically change according to the human life behavior.There are some typical periods in which the characteristics of the image data change, such as units of one day, units of one week, and units of one year.For example, the properties of the image data may differ between days of the week and weekends of the week.It is assumed that the current time is included in the time window Suntag. At this time, when it is known from experience that the characteristics of the image data at this time window are similar to those of the last sunday, the image fetch apparatus according to Embodiment 3 may calculate a C d using the data at the last sunday. It is believed that this cycle of the wander in units of one week is also derived from human life behavior.For example, the properties of the image data can also differ between summer and winter. It is assumed that the current time falls within the summer time window.At this time, when it is known from experience that the characteristics of the image data at this time window are similar to those of the last summer, the image fetch apparatus according to Embodiment 3 may calculate a C d using the data of the last summer. It is believed that this cycle of the wall in units of one year is also derived from differences in human life behavior, especially clothing.It is also possible that the characteristics of the image data are different between a day with a particular event and a normal day. Particular events include festivals, concerts of known artists, international conferences, international sports events, etc.For example, the characteristics of the image data at the curative evening may be different from those in a normal day.It is assumed that the current time is included in the time window for curative evening. At this time, when it is known from experience that the characteristics of the image data relating to this time window are similar to those of the last curative, the image fetch apparatus according to Embodiment 3 can calculate a C d using the data relating to the last curative. As shown in this example, changes on a particular day with particular events are also due to human life behavior.Thus, it can be determined that the characteristics of image data relating to humans are closely related to human life behavior. Human life behavior is also strongly influenced by catastrophes such as wars, pandemias and natural catastrophes.For example, assume that the current time is included in the Curative Evening time window 2022. In view of the effects of the propagation of the new coronavirus infection, the image pickup apparatus according to Embodiment 3 can calculate a C d using the data relating to Helibaend not of the last year but 2019 or 2018 before the propagation of the new coronavirus infection.The image fetch apparatus according to Embodiment 3 calculates the projection matrix (C d) based on data related to a time window of "past in which features of the image data can be predicted to be similar from experience.".By generalizing the questions concerning the calculation of the projection matrix (C d) the image retrieval apparatus according to Embodiment 3 can perform high-speed search that aims at a plurality of databases that relate to different time slots in consideration of human life behavior.[INDUSTRIAL APPLICABILITY]The disclosed technology can be used, for example, as a system for finding lost or suspect persons from video images captured by security cameras in facilities such as airports, and is industrially applicable.[REFERENCE LIST]100 Feature reduction processing circuit, 110 Data storage unit, 120 Matrix calculator, 130 Projection transformation section, 200 Retrieval processing unit, 210 Second projection transformation section, 220 Search execution section.References included in the specificationThis list of documents cited by the applicant has been produced in an automated manner and is only included for the better information of the reader. The list is not part of the German patent application or utility model application. The DPMA does not take any adhesion for any faults or omissions.Patent Literature citedWO 2014 / 167880 A1

[0004]

Claims

An image retrieval apparatus comprising a feature reduction processing circuit, and a retrieval processing unit, wherein a matrix calculator included in the feature reduction processing circuit calculates a matrix (C d) based on stored data (X d-1) a projection transformation section included in the feature reduction processing circuit multiplies the matrix (C d) by a sequentially acquired image feature amount {f i} a second projection transformation section included in the retrieval processing unit calculates a low-dimensionallyized image feature (g target_x) from an input image feature (f target) of a search target with the matrix (C d), a search execution section included in the retrieval processing unit, calculates distances between vectors calculated by the projection transformation section and the low-dimensionallyized image feature (g target_x) in the feature space, and performs a search for plausible image features using the distances, wherein the vectors calculated by the projection transformation section 130 belong to a time-window-divided database.The image fetch apparatus of claim 1, wherein the matrix (C d) is calculated using singular value decomposition of the data (X d-1).The image fetch apparatus according to claim 1, wherein the matrix (C d) is calculated using approximate singular vectors of the data (X d-1).The image retrieval apparatus according to any one of claims 1, 2 and 3, wherein the stored data relates to images captured in an adjacent most recent past time window.The image retrieval apparatus of any of claims 1, 2 and 3, wherein the stored data relates to images that have been captured in past time slots and can be empirically predicted to have similar features of image data.

Images