Image search device

Abstract

The inventors of the disclosed technology have discovered a problem in that when the image search device exemplified in Patent Document 1 is applied to a query system, even if the images are obtained from the same security camera, when viewed over a time window of, for example, one day, the matrix suitable for use in dimensionality reduction changes, albeit slowly. The image search device according to the present disclosure includes a feature reduction processing circuit (100) including a matrix calculator (120) and a projective transformation unit (130), and a search processing circuit (200) including a second projective transformation unit (210) and a search execution unit (220). The matrix calculator (120) calculates a feature reduction process from stored data (X d-1 ) based on the feature reduction matrix (C d ), and the projective transformation unit (130) calculates the image feature quantity f i For (i=1, 2, ..., m), the matrix (C d ), and the second projective transformation unit (210) multiplies the image feature quantity (f target ), the reduced-dimensional image features (g target_x ), and the search execution unit (220) calculates the vectors calculated by the projective transformation unit (130) and divided for each time window in the feature space and the reduced-dimensional image feature (g target_x ) and searches for plausible image features.

Description

[Technical field]

[0001] The present disclosure relates to an image search device. [Background technology]

[0002] In the technical field of image search, a technique is known that uses a matrix obtained from the results of known statistical analyses such as principal component analysis and discriminant analysis to reduce the dimensions of image features so that they are suitable for high-speed searches. Here, the image feature is a concept used in the technical field of machine learning, and is a vector quantity in a feature space that is generated for an image. As the name suggests, the image feature represents the characteristics of an image.

[0003] For example, Patent Document 1 discloses a technique for simplifying a classification problem by using a first linear transformation parameter as a matrix obtained from the results of statistical analysis and transforming image features using the first linear transformation parameter. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2014 / 167880 Summary of the Invention [Problem to be solved by the invention]

[0005] The image search device exemplified in Patent Document 1 can be applied as a system for finding lost children or suspicious persons from video images captured by security cameras in facilities such as airports. In this specification, a system for finding lost children or suspicious persons from video images captured by security cameras in facilities such as airports is referred to as a "query system."

[0006] The inventors of the disclosed technology have discovered a problem in that when the image search device exemplified in Patent Document 1 is applied to a query system, even if the images are obtained from the same security camera, when viewed over a time window of, for example, one day, the matrix suitable for use in dimensionality reduction changes, albeit slowly. The disclosed technique aims to provide an image search device that overcomes the problem that matrices suitable for use in dimensional reduction change, albeit slowly. [Means for solving the problem]

[0007] The image search device according to the disclosed technique includes a feature amount reduction processing circuit including a matrix calculator and a projective transformation unit, and a search processing circuit including a second projective transformation unit and a search execution unit. The matrix calculator calculates stored data (X d-1 ) based on the feature reduction matrix (C d ), and the projective transformation unit calculates the image feature f i For (i=1,2,…,m), the matrix (C d ), and the second projective transformation unit multiplies the image feature (f target ), the reduced-dimensional image features (g target_x ) and the search execution unit calculates the vectors calculated by the projective transformation unit and divided for each time window in the feature space and the reduced-dimensional image feature (g target_x ) and searches for plausible image features. Effect of the Invention

[0008] Since the image search device according to the present disclosure has the above-described configuration, it is possible to overcome the problem that matrices suitable for use in dimensionality reduction change, albeit slowly. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing the functional configuration of a feature amount reduction processing circuit 100 according to the disclosed technique. [Diagram 2] FIG. 2 is a time chart showing the process of the feature amount reduction processing circuit 100 according to the disclosed technique in chronological order. [Diagram 3] FIG. 3 is a block diagram showing the functional configuration of the search processing circuit 200 according to the disclosed technique. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] As described above, the image feature amount on which the present disclosure is based is a concept used in the technical field of machine learning, and is a vector amount in a feature amount space that is generated for an image. The present disclosure is not limited by the matter of the type of learning model by which the image feature amount is generated. However, it is easy to understand the image feature amount as an intermediate product when the learning model is, for example, a convolutional neural network (CNN). CNN is a type of artificial neural network that is recognized as an effective means, particularly in the technical field of image analysis.

[0011] Embodiment 1 The image search device according to the disclosed technique includes a feature amount reduction processing circuit 100 and a search processing circuit 200. The feature amount reduction processing circuit 100 is a processing circuit that acquires image feature amounts and performs feature amount reduction processing such as dimensionality reduction. The search processing circuit 200 is a processing circuit that, when image features to be searched for are input, searches for plausible image features from a database divided into time spans (hereinafter referred to as "time windows").

[0012] 1 is a block diagram showing the functional configuration of a feature reduction processing circuit 100 according to the present disclosure. As shown in FIG. 1, the feature reduction processing circuit 100 includes a data storage unit 110, a matrix calculator 120, and a projective transformation unit 130.

[0013] Fig. 2 is a time chart showing the time series of the processing of the feature reduction processing circuit 100 according to the disclosed technique. In Fig. 2, the portion indicated as ST110 represents the timing at which the processing is performed by the data storage unit 110. Similarly, the portion indicated as ST120 represents the timing at which the processing is performed by the matrix calculator 120. Similarly, the portion indicated as ST130 represents the timing at which the processing is performed by the projective transformation unit 130.

[0014] <<Data storage unit 110 constituting feature quantity reduction processing circuit 100>> The data storage unit 110 constituting the feature amount reduction processing circuit 100 is a component that stores image feature amounts obtained sequentially as data. i When (i=1,2,...,m), the data to be stored (X) is given by the following formula. TIFF0007661632000001.tif31160 As shown in formula (1), the image features (f i ) is a real vector of size 1×n. In this specification, the image feature (f i ) is defined as a row vector. As shown in formula (1), image features are generally real vectors whose components are real numbers, but the definition can also be extended to complex vectors whose components are complex numbers. Image features (f i ), the data (X) stored in the data storage unit 110 is a matrix of size m×n. In formula (1), the data (X) is shown as a real matrix, but if the definition of the image feature amount is extended to a complex vector, the data (X) becomes a complex matrix. In this specification, when emphasizing that the data stored in the data storage unit 110 is data relating to the d-1th time window, for example, d-1 is added in the form of a subscript to the lower right, and "X d-1 " shall be expressed as follows.

[0015] <<Matrix calculator 120 constituting feature quantity reduction processing circuit 100>> The matrix calculator 120 constituting the feature quantity reduction processing circuit 100 calculates the data (X d-1 ) based on the feature reduction matrix (C d ) is calculated by the matrix calculator 120. d ) is the image feature (f i The technical feature of the image search device according to the present disclosure is that the feature amount of the data (X d-1 ) using the matrix (C d ) is used in the calculation process related to the d-th time window. Here, the letter "C" is used as the matrix for reducing the feature amount. d The C in "Compression" comes from the first letter of "Compression."

[0016] The matrix (C d ) may be a matrix obtained from the results of known statistical analysis such as principal component analysis or discriminant analysis, as shown in Patent Document 1. d ) may be a matrix obtained based on singular value decomposition. d-1 ) is given as shown in the following formula: TIFF0007661632000002.tif30160 U in formula (2) is the left singular vector {u i}, i = 1, 2, ..., r is a matrix with columns. Also, V in formula (2) is a right singular vector {v i},i=1,2,…,r is a matrix with columns. The upper right subscript T indicates transposition. S on the right side of formula (2) is a diagonal matrix in which the singular values ​​are arranged in descending order. Here, the data for the d-1th time window (X d-1) is the total number of rows in the image, i.e., the number of image features (f i The number of columns (m) is the total number of columns, n, that is, the image features (f i The subscript r in formula (2) is the first letter of rank, and d-1 It represents the rank of X. d-1 If X has full column rank, then r is equal to n. d-1 The rank of is r, which means that the vector (f1, f2, … f m ) has dimension r.

[0017] The matrices U and V constructed from the singular vectors have the following properties: TIFF0007661632000003.tif13160 Here, I(r) appearing on the right side of equation (3) is a unit matrix whose size is r × r.

[0018] X d-1 If is not a zero matrix, then, using the result of the singular value decomposition shown in equation (2) and the properties of equation (3), X d-1 According to the Moore-Penrose type definition of the general inverse matrix, X d-1 The generalized inverse matrix of is given by the following formula: TIFF0007661632000004.tif26160 X d-1 If is a regular matrix, the generalized inverse matrix given by equation (4) is X d-1 It is equal to the inverse matrix of .

[0019] X is a matrix d-1 m row vectors (f1,f2,…f m ) is stretched by R n The subspace of X d-1 The right singular vectors {v i},i=1,2,…,r,transposed to create an orthonormal basis for row space. One aspect of the image search device according to the disclosed technique is to calculate the projection matrix onto this row space and apply it to a query system. v ) is given by the following formula: TIFF0007661632000005.tif11160 P shown in formula (5) v is a projection matrix of size n × n and rank r. Here, VV T is V shown in formula (3) T Note that this is different from V. In the query system, the image feature (f target ) is given, the right projection matrix (P v ) to obtain the image feature (f target ) into the row space, which is a subspace, and the search range can be limited to the row space. The letter P, which stands for projection matrix, comes from the first letter of Projection.

[0020] If we multiply both sides of equation (2) by V from the right, due to the properties of equation (3), equation (2) can be transformed as follows. TIFF0007661632000006.tif62160 Here, g1, g2, … g m is a reduced-dimensional image feature (hereinafter referred to as a "reduced-dimensional image feature"). Equation (6) is a row vector (f x ) belongs to the row space, i.e., f1, f2, … f m If it can be expressed as a linear combination of the row vector (f x ) by V on the right, the row vector obtained is g1, g2, … g m This shows that it can be expressed as a linear combination of TIFF0007661632000007.tif18160 Here, g1, g2, … gm are all row vectors of size 1×r. That is, g1, g2, … g m f can be expressed as a linear combination of x V is also a row vector of size 1×r. m are coefficients, respectively.

[0021] Another row vector (f y ) is given, and f x Unlike,f y does not belong to the row space. f does not belong to the row space. y First, the projection matrix (P v ) from the right, project it to row space, and then multiply it by V from the right. y P v Since f is a row vector that belongs to the row space, x The result of this attempt is given by the following formula: TIFF0007661632000008.tif18160 After all, f does not belong to the row space. y For example, if we simply multiply V from the right, the row vector (f y V) must be g1, g2, … g m (See Equation (8)).

[0022] As described above, the matrix (C d ) may be a matrix of singular vectors, V. Note that X d-1 If is full column rank and r is equal to n, then the image features (f i ) by V from the right, the size of the vector remains 1×n. In such a case, a method can be considered in which singular values ​​that can be approximated to 0 as a result of singular value decomposition are set to 0, and the dimension is reduced by approximation. The details of the method of setting singular values ​​that can be approximated to 0 to 0 will become clear from the second embodiment.

[0023] 《Projection Conversion Unit 130 Constituting Feature Quantity Reduction Processing Circuit 100》 The projection conversion unit 130 constituting the feature quantity reduction processing circuit 100 is a component that multiplies the matrix (C i ) calculated by the matrix calculator 120 for the image feature quantities f d (i = 1, 2,..., m) obtained sequentially. In this specification, since the image feature quantities (f i ) are defined as horizontal vectors, the multiplication of the matrix (C d ) in the projection conversion unit 130 is a right multiplication (f i C i ) with respect to the image feature quantities (f d ). Considering C d as V, the size of C d is n×r (r < n). Thus, the projection conversion unit 130 is a component that plays a role of converting the image feature quantities (f i ) into the low-dimensional image feature quantities (g i ). The low-dimensional image feature quantities (g i ) are shown to be calculated using the matrix V consisting of eigenvectors in Equation (6), but the technology of the present disclosure is not limited thereto. In the technology of the present disclosure, even when the projection conversion unit 130 uses a matrix (C d、 ≠ V) different from V, the low-dimensionalized image feature quantities are regarded as the low-dimensional image feature quantities and are represented by g i .

[0024] As shown in FIG. 1, the low-dimensional image feature quantities (g i ) generated by the projection conversion unit 130 are stored in a storage device accessible by the image search device as a database for each time window. Also, since the matrix (C d ) used by the projection conversion unit 130 for multiplication is also used in the search processing circuit 200 of the image search device, it is shared by the feature quantity reduction processing circuit 100 and the search processing circuit 200.

[0025] 3 is a block diagram showing a functional configuration of the search processing circuit 200 according to the disclosed technique. The feature amount reduction processing circuit 100 of the image search device is a component used in the phase of storing image data, whereas the search processing circuit 200 of the image search device is a component used in the phase of searching for images. In explaining the processing contents of the image search device according to the disclosed technique, the "phase of storing image data" and the "phase of searching for images" are defined separately, but in a scene where the image search device is used, both phases may proceed simultaneously. In other words, it may happen that image data is stored by the feature amount reduction processing circuit 100 and images are searched for by the search processing circuit 200 in parallel. The f shown in Figure 3 target represents the image feature quantity related to the image to be searched. When the image search device is a query system, f target are image features related to the query target image.

[0026] As shown in FIG. 3, the search processing circuit 200 includes a second projective transformation unit 210 (210-0, 210-1, ..., 210-x, ...) and a search execution unit 220 (220-0, 220-1, ..., 220-x, ...). As shown in FIG. 3, the functional block with the reference number "-0" is a functional block that handles a database related to the d-th time window. Similarly, the functional block with the reference number "-1" is a functional block that handles a database related to the d-1-th time window. In addition, as a generalized expression, the functional block with the reference number "-x" is a functional block that handles a database related to the dx-th time window. The time range of the databases that the search processing circuit 200 of the image search device searches is a design item of the image search device, and may be determined based on the purpose of use and specifications of the image search device.

[0027] <<Second projective transformation unit 210 constituting the search processing circuit 200>> The second projective transformation unit 210 constituting the search processing circuit 200 converts the input f targetIt can be said that the second projective transformation unit 210 in the search processing circuit 200 is a component corresponding to the projective transformation unit 130 in the feature amount reduction processing circuit 100. As described above, the matrix (C d ,C d-1 ,C d-2 ,…,C d-x , . . . ) are shared in search processing circuit 200, specifically, shared in second projective transformation section 210 of search processing circuit 200. The reduced-dimensional image feature quantity (g target_x ) is given by the following formula: TIFF0007661632000009.tif13160 The matrix (C d ,C d-1 ,C d-2 ,…,C d-x , . . .) is a singular vector V, the second projective transformation unit 210 uses it for multiplication (C d ,C d-1 ,C d-2 ,…,C d-x ,…) is also a V of singular vectors.

[0028] Search execution unit 220 constituting search processing circuit 200 The search execution unit 220 constituting the search processing circuit 200 calculates the reduced-dimensional image feature (g target_x ) to search for plausible image features from a database divided into time windows. Note that the term "search" used here is the same as in the computer technology field, and means the process of retrieving desired data or determining where it is stored. Specifically, the search execution unit 220 calculates the distance in the feature space and extracts the most likely image feature based on the magnitude of the calculated distance. The distance calculated by the search execution unit 220 is a distance used as an index for evaluating similarity, such as Euclidean distance, Mahalanobis distance, Bhattacharyya distance, etc.

[0029] As shown in Equation (2) and Equation (6), by using singular value decomposition in which the singular values ​​are arranged in descending order, the reduced-dimensional image feature (g target_x ) are generated such that the most dominant element becomes the first component, the second dominant element becomes the second component, and so on. Therefore, the search execution unit 220 constituting the search processing circuit 200 may, for example, calculate the distance using only the first component, or using only the first component to the k-th component (k is a natural number equal to or greater than 2 and less than r), and extract plausible image feature candidates. Alternatively, the search execution unit 220 constituting the search processing circuit 200 may, for example, calculate the distance using only the first component, or using only the first component to the k-th component, and perform filtering to separate those that should be searched from those that should not be searched. The idea of ​​calculating the distance using only the first component to the k-th component is common to the dimensional reduction method of approximating singular values ​​that can be approximated to 0 in singular value decomposition by setting them to 0. As described above, the dimensional reduction method of approximating singular values ​​that can be approximated to 0 in singular value decomposition by setting them to 0 will be made clear in the second embodiment.

[0030] The search performed by the search execution unit 220 is based on the reduced-dimensional image feature quantity (g target_x ) and the database of reduced-dimensional image features (g i ) is calculated by calculating the distance between all the reduced image features (g i ) there is no need to calculate the distance. The search performed by the search execution unit 220 is, for example, a search target reduced-dimensional image feature quantity (g target_x ) and the databased reduced-dimensional image features (g i) and each of the components can be expressed in binary. Expressing in binary is useful for narrowing down search candidates (known as filtering). Specifically, the search performed by the search execution unit 220 checks whether two pieces of data expressed in binary match, starting from the most significant bit. The reduced-dimensional image feature (g target_x ) and a certain reduced-dimensional image feature of interest (g i When comparing the first component of with and, if the most significant bits of each do not match, then it is unlikely that they are the pair with the shortest distance. In this way, data where even the most significant bit does not match can be excluded from the search candidates. After filtering by the most significant bit, a similar filtering is performed using the second most significant bit. This search method is similar in concept to a search method known as the bisection method. The method of checking whether two pieces of data match starting from the most significant digit is not limited to those expressed in binary notation. A method of checking whether two pieces of data match starting from the most significant digit for those expressed in general N-ary notation can also be considered as filtering in a search.

[0031] As described above, the search processing performed by the search execution unit 220 is performed in parallel with the accumulation of image data by the feature amount reduction processing circuit 100. For example, if the current time belongs to the d-th time window, it is conceivable that the search execution unit 220 searches the d-th database in parallel with the accumulation of image data in the d-th database by the feature amount reduction processing circuit 100. 2, the process (ST120) in which the matrix calculator 120 calculates a matrix based on the data of the database relating to the d-th time window is performed after the process (ST110) in which the data storage unit 110 acquires all image feature amounts in the d-th time window. Therefore, according to the conventional thinking, the image retrieval device cannot perform the projective transformation process (ST130) by the projective transformation unit 130 until after the data storage unit 110 has completed all the processes (ST110) in which the image feature amounts are acquired. In other words, according to the conventional thinking, the search execution unit 220 cannot search the d-th database in parallel with the feature amount reduction processing circuit 100 storing image data in the d-th database. It is possible to generate reduced-dimensional image features using a single fixed matrix (C). However, as mentioned above, even if the images are taken from the same security camera, the matrix suitable for dimensionality reduction will change when viewed over a time window of, for example, one day. In other words, as time passes, dimensionality reduction using a single fixed matrix (C) will no longer be suitable for a query system.

[0032] One of the advantageous effects of the image search device of embodiment 1 is that the projective transformation unit 130 can sequentially perform the projective transformation process (ST130) even before the data storage unit 110 has completed all of the processes (ST110) for acquiring image features. Another excellent effect of the image retrieval device of embodiment 1 is that the search execution unit 220 can perform a search of the d-th database in parallel with the feature reduction processing circuit 100 storing image data in the d-th database. In summary, the outstanding effect of the image retrieval device of embodiment 1 is that it overcomes the problem of "changes in the matrix suitable for dimensionality reduction" and enables high-speed searches of multiple databases relating to different time domains. These effects are achieved by actively taking advantage of the property that matrices suitable for dimensionality reduction change slowly when a time window is set, for example, on a daily basis. Taking advantage of the nature that the change is gradual, the image retrieval device according to the first embodiment uses the data related to the d-1th time window (X d-1 ) using the matrix (C d ) is used in the calculation process related to the d-th time window. Due to this technical feature, the image retrieval device according to the first embodiment achieves the above-mentioned excellent effects.

[0033] Embodiment 2 The image retrieval device according to the second embodiment is a modified example of the image retrieval device according to the disclosed technique. In terms of functional configuration, the image retrieval device according to the second embodiment is the same as the image retrieval device according to the first embodiment. Therefore, unless otherwise specified, the same symbols as those used in the first embodiment are used in the second embodiment. Furthermore, in the second embodiment, explanations that overlap with those in the first embodiment are omitted as appropriate.

[0034] It can be said that the image retrieval device according to the second embodiment is an embodiment of the matrix calculated by the matrix calculator 120. In the image retrieval device according to the second embodiment, the matrix calculated by the matrix calculator 120 is based on a technique for reducing dimensions by approximation, in which singular values ​​that can be approximated to 0 in singular value decomposition are set to 0.

[0035] The singular value decomposition shown in equation (2) can be approximated by the following equation: TIFF0007661632000010.tif42160 The right-hand side of formula (10) replaces the k+1th to rth singular values, counting from largest to smallest, with 0 (zero). The diagonal matrix on the right-hand side of formula (10) is represented by adding a tilde accent symbol to S, indicating that it is similar to the diagonal matrix (S) on the left-hand side of formula (10). Formula (10) represents an "approximation in which singular values ​​that can be approximated to 0 are set to 0." As shown in Equation (10), k is a positive integer smaller than r, which indicates the rank.

[0036] The right hand side of equation (10) can be transformed as shown in the following equation. TIFF0007661632000011.tif41160 Here, the subscript trunc on the right side of the formula (11) comes from truncation. trunc is a matrix of size m×k. S trunc is a diagonal matrix of size k×k. V trunc is a matrix of size n × k.

[0037] The image retrieval device according to the second embodiment is configured as follows: trunc is the matrix calculated by the matrix calculator 120. Therefore, in the image search device according to the second embodiment, the projective transformation unit 130 constituting the feature amount reduction processing circuit 100 and the second projective transformation unit 210 constituting the search processing circuit 200 respectively calculate V trunc A projective transformation is performed using V trunc The effect of using the matrix calculated by the matrix calculator 120 is that the reduced-dimensional image feature (g target_x ) has the same effect as using only the first through kth components.

[0038] The advantages unique to the image search device according to the second embodiment are as follows: d-1 Even if is full column rank and r is equal to n, the image features (f target ) from n to k. Due to this effect, the image retrieval device of embodiment 2, like the image retrieval device of embodiment 1, overcomes the problem that "the matrix suitable for dimensionality reduction changes" and can perform high-speed searches of multiple databases related to different time domains.

[0039] Embodiment 3 The image search device according to the third embodiment is a modified example of the image search device according to the disclosed technique. Unless otherwise specified, the same reference symbols as those used in the previous embodiments are used in the third embodiment. Furthermore, in the third embodiment, descriptions that overlap with those in the previous embodiments are omitted as appropriate.

[0040] As described above, the technical feature of the image retrieval device according to the first and second embodiments is that the data related to the d-1th time window (X d-1 ) using the matrix (C d ) is used for the d-th time window calculation process. In this way, in the image search device according to the first and second embodiments, when viewed on the time axis, the data (X d-1 ) is used. The image search device according to the third embodiment generalizes the phrase "the most recent adjacent past" to "the past when it can be predicted from experience that the image data has similar characteristics." In other words, when viewed over a one-day time window, the most recent adjacent past is a specific example of "the past when it can be predicted from experience that the image data has similar characteristics." From a technical standpoint, it is not essential that it is "the most recent adjacent past," but what is important is that "it can be predicted from experience that the image data has similar characteristics."

[0041] There are cases where data used by the image retrieval device is required to be stored in a database not on a daily basis, but on a six-hour time window basis, such as from 0:00 to 6:00, 6:00 to 12:00, 12:00 to 18:00, and 18:00 to 24:00. Assume that the current time is included in the time window "12:00 to 18:00". In this case, if it is known from experience that the characteristics of image data related to this time window are similar to those of yesterday's "12:00 to 18:00", the image retrieval device according to the third embodiment will compile a matrix (C d ) can be calculated. In this way, the characteristics of image data relating to humans may change periodically in accordance with the daily activities of humans.

[0042] The period during which the characteristics of image data change may vary, for example, on a daily basis, a weekly basis, or a yearly basis. For example, the characteristics of image data may differ between weekdays and weekends. Assume that the current time is included in the time window for Sunday. In this case, if it is known from experience that the characteristics of image data related to this time window are similar to those of last Sunday, the image retrieval device according to the third embodiment uses the data related to last Sunday to search for a matrix (C d ) can be calculated. This weekly cycle of change is also thought to result from human daily activities. Furthermore, for example, the characteristics of image data may differ between summer and winter. Assume that the current time is included in the summer time window. In this case, if it is known from experience that the characteristics of image data related to this time window are similar to those of last summer, the image retrieval device according to the third embodiment uses data related to last summer to search for a matrix (C d This annual cycle of change is also thought to be due to differences in human behavior, especially clothing.

[0043] The characteristics of the image data may be different on a day when there is a special event, such as a festival, a concert by a popular artist, an international conference, an international sporting event, etc. For example, the characteristics of image data on Christmas Eve may differ from those of image data on a normal day. Assume that the current time is included in the time window for Christmas Eve. If it is known from experience that the characteristics of image data related to this time window are similar to those of last year's Christmas Eve, the image retrieval device according to the third embodiment can use the data related to last year's Christmas Eve to search for a matrix (C d In this way, changes on days when special events occur can also be considered to result from human daily activities.

[0044] In this way, the characteristics of image data related to humans are closely related to human daily activities, which are also greatly affected by disasters such as wars, pandemics, and natural disasters. For example, assume that the current time is included in the time window for Christmas Eve in 2022. In consideration of the impact of the spread of the novel coronavirus, the image search device according to the third embodiment calculates the matrix (C d ) may be calculated.

[0045] The image retrieval device according to the third embodiment calculates a projection matrix (C d ) is calculated. This projection matrix (C d By generalizing matters related to the calculation of , the image retrieval device according to the third embodiment can perform high-speed searches across multiple databases relating to different time domains, taking into account human daily activities. [Industrial Applicability]

[0046] The disclosed technology can be applied, for example, to a system for finding lost children or suspicious people from video images captured by security cameras in facilities such as airports, and has industrial applicability. [Explanation of symbols]

[0047] 100 feature amount reduction processing circuit, 110 data storage unit, 120 matrix calculator, 130 projective transformation unit, 200 search processing circuit, 210 second projective transformation unit, 220 search execution unit.

Claims

1. a feature reduction processing circuit including a matrix calculator and a projective transformation unit; a search processing circuit including a second projective transformation unit and a search execution unit; The matrix calculator calculates the stored data (X ｄ－１ ) based on which a matrix (C ｄ ) is calculated, The projective transformation unit sequentially acquires image features f ｉ For (i=1, 2, ..., m), the matrix (C ｄ ) and multiply it by The second projective transformation unit converts the image feature quantity (f ｔａｒｇｅｔ ), the reduced-dimensional image feature (g ｔａｒｇｅｔ＿ｘ ) is calculated, The search execution unit compares the vectors calculated by the projective transformation unit and divided for each time window with the reduced-dimensional image feature (g ｔａｒｇｅｔ＿ｘ ) and search for plausible image features. Image search device.

2. The matrix (C ｄ )teeth, The stored data (X ｄ－１ ) is a matrix of singular vectors obtained by singular value decomposition of The image search device according to claim 1 .

3. The matrix (C ｄ )teeth, The stored data (X ｄ－１ ) is singular value decomposed and reduced in dimension to obtain a matrix of approximate singular vectors. The image search device according to claim 1 .

4. the stored data relates to images taken in an adjacent and immediately past time window; The image search device according to claim 1 .

5. the stored data relates to images taken in a past time window where empirically the characteristics of the image data can be expected to be similar; The image search device according to claim 1 .

Images