Method, device, image storage device and storage medium for retrieving face images

Abstract

The application relates to the technical field of image processing, and discloses a method and device for retrieving a face image, an image storage device and a storage medium. In the application, if it is detected that an image feature extraction algorithm version corresponding to a user-input face image to be retrieved does not match an image feature extraction algorithm version corresponding to an image in a face image library, it is indicated that the image feature extraction algorithm used by the image storage device may have been changed, and the user-input face image to be retrieved is calculated by using an original algorithm to obtain feature values, so that an operation of directly performing feature value comparison cannot be executed. At this time, the image storage device searches for a first optimal face image which has the same face identifier as the face identifier of the face image to be retrieved and has completed feature value updating, reads feature values of the first optimal face image as substitutes, and finally performs face retrieval on the face image to be retrieved according to the read feature values of the first optimal face image to output, thereby realizing the function of similar face retrieval.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, image storage device, and storage medium for retrieving face images. Background Technology

[0002] Currently, many image storage devices (such as network video recorders, video surveillance management devices, and storage management service devices) have similar face retrieval functions. When a user inputs a captured image of a face, the image storage device can retrieve similar face images from all the saved face images and output them to the display interface.

[0003] However, when the image feature extraction algorithm used by the image storage device is changed, the subsequent captured images use the feature values ​​calculated by the changed algorithm, while the original captured images still use the feature values ​​calculated by the unchanged algorithm. The two feature values ​​are not comparable, so the image storage device cannot use the similar face retrieval function normally at this time.

[0004] To address this issue, the current conventional approach is to recalculate feature values ​​for all captured images stored on the image storage device using the updated algorithm after a version change. However, due to the large number of captured images stored on the device, updating the feature values ​​for all images is time-consuming. Summary of the Invention

[0005] In view of this, embodiments of this application provide a method, apparatus, image storage device, and storage medium for retrieving face images, which enables the image storage device to still provide similar face retrieval function normally after the image feature extraction algorithm version is changed, and does not require a lot of time to perform image feature value update operations.

[0006] A first aspect of this application provides a method for retrieving face images, including:

[0007] Obtain the face image to be searched;

[0008] Determine whether the version of the first algorithm for extracting image features from the face image to be retrieved matches the version of the second algorithm for extracting image features from images in the face image database;

[0009] If the version of the first algorithm does not match the version of the second algorithm, then the feature value of the first optimal face image stored in the face image database is read; wherein, the face identifier of the first optimal face image is the same as the face identifier of the face image to be retrieved, and the version of the algorithm for extracting image features from the first optimal face image matches the version of the second algorithm.

[0010] Based on the feature values ​​of the first optimal face image, perform face retrieval output on the face image to be retrieved.

[0011] In this embodiment, if the image feature extraction algorithm version corresponding to the user-inputted face image to be retrieved does not match the image feature extraction algorithm version corresponding to the image in the face image database, it indicates that the image feature extraction algorithm used by the image storage device may have changed, while the user-inputted face image to be retrieved still uses the original algorithm to calculate feature values, so the feature value comparison operation cannot be directly performed. In this case, the image storage device will search for the first optimal face image whose face identifier is the same as the face identifier of the face image to be retrieved (if the face identifier is the same, it means it is an image of the same face), and read the feature value of the first optimal face image as a substitute. Finally, based on the read feature value of the first optimal face image, the face retrieval output of the face image to be retrieved is performed. In the above process, since the image feature extraction algorithm version corresponding to the first optimal face image matches the image feature extraction algorithm version corresponding to the face image database, that is, the first optimal face image has completed the feature value update operation, the image feature value comparison operation can be performed normally. In addition, in this embodiment, feature value updates can be performed on each of the best face images after the image storage device completes the version change of the image feature extraction algorithm. Since the best face images stored in the image storage device are only a small part of all face images stored in the image storage device, the time spent performing image feature value update operations can be effectively reduced.

[0012] In one implementation of this application, the second algorithm is an image feature extraction algorithm used by an image storage device, the face image library is a database of the image storage device for storing face images, and the method may further include:

[0013] When the image storage device restarts, it is detected whether a flag file exists on the disk of the image storage device. The flag file is used to indicate whether to perform a feature value update operation for the face image.

[0014] If the flag file exists on the disk of the image storage device, the second algorithm is used to calculate and update the feature values ​​of each optimal face image stored in the image storage device; wherein, the image storage device stores the optimal face image corresponding to each known face identifier.

[0015] Furthermore, the flag file is written to the disk when the version of the second algorithm changes. After calculating and updating the feature values ​​of each optimal face image stored on the image storage device using the second algorithm, the process may further include:

[0016] Delete the flag file from the disk.

[0017] Furthermore, the feature values ​​of each optimal face image stored in the image storage device can be updated through the following steps:

[0018] The first thread reads the optimal face image, transcodes the optimal face image into image data of a specified format, and writes the image data of the specified format into the first buffer sequence;

[0019] The second thread reads the image data of the specified format from the first cache sequence, calculates the image data of the specified format using the second algorithm to obtain updated feature values, and writes the updated feature values ​​into the second cache sequence.

[0020] A third thread reads the updated feature value from the second cache sequence and writes the updated feature value into a specified database.

[0021] Furthermore, after detecting whether a flag file exists on the disk of the image storage device, the process may further include:

[0022] If the flag file exists on the disk of the image storage device, the second algorithm is used to calculate and update the feature values ​​of each face image stored in the user face database; wherein, the user face database is a set of face images consisting of a portion of face images selected by the user from all face images stored in the image storage device.

[0023] Furthermore, the method may also include:

[0024] Obtain the face images to be added to the database;

[0025] Determine whether the version of the third algorithm for extracting image features from the face image to be added to the database matches the version of the second algorithm;

[0026] If the version of the third algorithm does not match the version of the second algorithm, the second algorithm is used to calculate and update the feature values ​​of the face image to be added to the database, and then the face image to be added to the database is stored in the user face database.

[0027] In one implementation of this application, the step of performing a face retrieval output on the face image to be retrieved based on the feature values ​​of the first optimal face image may include:

[0028] The feature values ​​of the first optimal face image are matched with the feature values ​​of each optimal face image stored in the face image database to obtain the feature value matching results.

[0029] Based on the feature value matching result, a second optimal face image similar to the first optimal face image is determined from all the optimal face images stored in the face image database;

[0030] Output all face images whose face identifiers stored in the face image database are the same as the face identifier of the second optimal face image, and output all face images whose face identifiers stored in the face image database are the same as the face identifier of the face image to be retrieved.

[0031] A second aspect of this application provides an apparatus for retrieving face images, comprising:

[0032] The module for acquiring the face image to be retrieved is used to acquire the face image to be retrieved.

[0033] The first algorithm version judgment module is used to determine whether the version of the first algorithm for extracting image features from the face image to be retrieved matches the version of the second algorithm for extracting image features from the face image database.

[0034] The feature value reading module is used to read the feature value of the first optimal face image stored in the face image database if the version of the first algorithm does not match the version of the second algorithm; wherein the face identifier of the first optimal face image is the same as the face identifier of the face image to be retrieved, and the version of the algorithm for image feature extraction of the first optimal face image matches the version of the second algorithm.

[0035] The face retrieval module is used to perform face retrieval output on the face image to be retrieved based on the feature values ​​of the first optimal face image.

[0036] A third aspect of this application provides an image storage device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method for retrieving face images as provided in the first aspect of this application.

[0037] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method for retrieving face images as provided in the first aspect of this application.

[0038] A fifth aspect of this application provides a computer program product that, when run on an image storage device, causes the image storage device to execute the method for retrieving face images described in the first aspect of this application.

[0039] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating a method for retrieving face images provided in an embodiment of this application;

[0041] Figure 2 This is a schematic diagram illustrating an operation for updating facial image feature values ​​using a multi-threaded approach, as provided in an embodiment of this application.

[0042] Figure 3 This is a schematic diagram of a feature value update process provided in an embodiment of this application;

[0043] Figure 4 This is a schematic diagram of a face image database entry operation provided in an embodiment of this application;

[0044] Figure 5 This is a structural diagram of a device for retrieving face images provided in an embodiment of this application;

[0045] Figure 6 This is a schematic diagram of an image storage device provided in an embodiment of this application. Detailed Implementation

[0046] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail. Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0047] For image storage devices such as network video recorders, video surveillance management equipment, and storage management service equipment, the image feature extraction algorithms they use are changed from time to time. When the algorithm is changed (including algorithm version upgrades or rollbacks), the feature values ​​of subsequently generated snapshots are calculated using the changed algorithm, while the feature values ​​of the original snapshots are still calculated using the unchanged algorithm. The two types of feature values ​​cannot be directly compared, which will cause the image storage device to be unable to provide the function of similar face retrieval normally. The above problem is specifically manifested as follows: if the image to be retrieved by the user is an original snapshot, the image storage device will prompt that the algorithm version is different and similar face retrieval cannot be performed; if the image to be retrieved by the user is a subsequently generated snapshot, the original snapshot saved by the image storage device cannot be displayed as a search result.

[0048] To address the aforementioned issues, this application proposes a method for retrieving face images, which enables the image storage device to continue providing similar face retrieval functionality even after the image feature extraction algorithm version is upgraded, without requiring a significant amount of time to perform image feature value update operations.

[0049] It should be understood that the execution subject of the method for retrieving face images provided in this application embodiment can be a network video recorder, video surveillance management equipment, storage management service equipment, or other image storage devices with similar face retrieval functions. This application embodiment does not impose any restrictions on the specific type of image storage device.

[0050] Please see Figure 1 This application illustrates a method for retrieving face images according to an embodiment of the present application, comprising:

[0051] 101. Obtain the face image to be searched;

[0052] First, the image of the face to be searched is acquired. This image is the one selected by the user for the similarity search; that is, it retrieves face images already stored on the image storage device that are similar to the image of the face to be searched. Specifically, the user can operate on the image storage device, selecting either a newly generated snapshot or an existing snapshot already stored on the device.

[0053] 102. Determine whether the version of the first algorithm for extracting image features from the face image to be retrieved matches the version of the second algorithm for extracting image features from the face image database;

[0054] After acquiring the image of the face to be retrieved, it is determined whether the version of the first algorithm for extracting image features from the face image to be retrieved matches the version of the second algorithm for extracting image features from images in the face image database. Here, the face image database can be a database on the image storage device used to store face images; therefore, the second algorithm can be the image feature extraction algorithm used by the image storage device. That is, it checks whether the algorithm used to calculate the feature values ​​of the face image to be retrieved is the same as the algorithm currently used by the image storage device to calculate the feature values ​​of the face image. For example, if the image storage device has changed its image feature extraction algorithm, and the face image to be retrieved is a snapshot taken before the algorithm change, a mismatch in algorithm versions will occur; if the image storage device has changed its image feature extraction algorithm, and the face image to be retrieved is a snapshot taken after the algorithm change, a matching algorithm version will occur; if the image storage device has not changed its image feature extraction algorithm, a matching algorithm version will occur.

[0055] If the first algorithm version does not match the second algorithm version, proceed to steps 103-104; otherwise, proceed to step 105.

[0056] 103. Read the feature values ​​of the first optimal face image stored in the face image database;

[0057] If the version of the image feature extraction algorithm corresponding to the face image to be retrieved does not match the version of the image feature extraction algorithm used by the image storage device, it is not possible to directly use the feature values ​​of the face image to be retrieved for comparison. In this case, the image storage device will search for the first best face image with the same face identifier as the face image to be retrieved from all the best face images already stored in the face image library, and read the feature values ​​of the first best face image to replace it.

[0058] The image storage device can save the optimal face image corresponding to each known face identifier, and ensures that the image feature extraction algorithm version corresponding to each optimal face image matches the image feature extraction algorithm version used by the image storage device. Each face image (including the optimal face image) saved by the image storage device is associated with a face identifier (face_id) to represent the corresponding face. There may be multiple face images under the same face identifier. For example, there may be many face images with the face identifier "Zhang San", and all of these face images represent the identity of "Zhang San". For each face identifier, the image storage device can select the image with the best image quality and the most representative facial features as the optimal face image under that face identifier. For example, if an image storage device stores 100 facial images of "Zhang San" and 200 facial images of "Li Si", it will select the image with the best image quality and the one that best represents Zhang San's facial features from the 100 facial images of "Zhang San" as the optimal facial image for identifying "Zhang San"; and select the image with the best image quality and the one that best represents Li Si's facial features from the 200 facial images of "Li Si" as the optimal facial image for identifying "Li Si", and so on.

[0059] This application embodiment requires that the image feature extraction algorithm version corresponding to all optimal face images stored in the image storage device matches the image feature extraction algorithm version used by the image storage device. In actual operation, it is only necessary to update the feature values ​​of all stored optimal face images after the image storage device completes the change of the image feature extraction algorithm version. Since the optimal face images stored in the image storage device are only a small part of all face images stored in the image storage device, assuming that each face identifier has an average of 100 face images (of which 1 is the optimal face image), the number of optimal face images only accounts for 1% of all face images stored in the image storage device. Therefore, updating the feature values ​​of all optimal face images stored in the image storage device does not require a lot of time.

[0060] The image storage device searches for the first best face image whose face identifier is the same as the face image to be retrieved from all the best face images that have been saved. For example, if the face identifier of the face image to be retrieved is "Li Si", the image storage device will search for the best face image with the face identifier "Li Si", which is referred to here as the first best face image. Subsequently, the feature values ​​of the best face image of "Li Si" can be compared with the feature values ​​of the best face images of other face identifiers to find other best face images similar to "Li Si".

[0061] In one implementation of this application, the method may further include:

[0062] (1) When the image storage device restarts, detect whether there is a flag file in the disk of the image storage device. The flag file is used to indicate whether to perform a feature value update operation for the face image.

[0063] (2) If the flag file exists in the disk of the image storage device, the second algorithm is used to calculate and update the feature value of each optimal face image stored in the image storage device; wherein, the image storage device stores the optimal face image corresponding to each known face identifier.

[0064] To ensure that the image feature extraction algorithm version corresponding to all the best face images stored on the image storage device matches the image feature extraction algorithm version used by the image storage device, a flag file can be checked on the disk of the image storage device each time it restarts. If the flag file exists, it means that the version of the image feature extraction algorithm (i.e., the second algorithm) used by the image storage device has been changed and the feature value update operation of the face image has not been completed. Therefore, it is necessary to use the second algorithm (here referring to the second algorithm after the version change) to calculate and update the feature value of each best face image stored on the image storage device.

[0065] Furthermore, the flag file is written to the disk when the version of the second algorithm changes. After calculating and updating the feature values ​​of each optimal face image stored on the image storage device using the second algorithm, the process may further include:

[0066] Delete the flag file from the disk.

[0067] Image storage devices can write a flag file to the disk after each change to the image feature extraction algorithm to determine whether an optimal face image feature value update operation is needed. Each time the image storage device boots up, it checks if the flag file exists on the disk. If it exists, it indicates that the image feature extraction algorithm version used by the image storage device has changed, and an optimal face image feature value update operation is required. If it does not exist, it means that the image feature extraction algorithm version used by the image storage device has not changed, or the optimal face image feature value update operation has been completed (the flag file is deleted after the update operation), and in this case, there is no need to perform the optimal face image feature value update operation.

[0068] In practice, restarting an image storage device typically involves the following three scenarios:

[0069] (A) Automatic restart of the image storage device after upgrading the image feature extraction algorithm;

[0070] (B) The feature value update operation of the optimal face image has not been completed. The image storage device has been powered off and restarted or manually restarted by the user.

[0071] (C) Restarting the image storage device under normal conditions.

[0072] In scenarios (A) and (B), after restarting, the image storage device will detect the presence of a flag file on the disk (meaning the image feature extraction algorithm of the image storage device has changed, but the feature values ​​of the optimal face image have not yet been updated). Therefore, it will enter the feature value update process for the optimal face image. After this update process is completed, the flag file on the disk will be deleted. In scenario (C), the feature value update operation for the optimal face image has been completed, and the flag file on the disk has been deleted. Therefore, the image storage device will not enter the feature value update process for the optimal face image.

[0073] In one implementation of this application, the feature values ​​of each optimal face image stored in the image storage device can be updated through the following steps:

[0074] (1) Use the first thread to read the optimal face image, transcode the optimal face image into image data of a specified format, and write the image data of the specified format into the first buffer sequence;

[0075] (2) Use the second thread to read the image data of the specified format from the first cache sequence, use the second algorithm to calculate the image data of the specified format to obtain the updated feature value, and write the updated feature value into the second cache sequence;

[0076] (3) Use a third thread to read the updated feature value from the second cache sequence and write the updated feature value into the specified database.

[0077] Updating the feature values ​​of a face image can be divided into three stages: (1) reading the face image and converting it into image data of a specified format, such as RGB data or YUV data, to facilitate subsequent image feature extraction algorithms; (2) using the updated image feature extraction algorithm (i.e., the second algorithm after version change) to calculate the specified format image data and obtain the updated feature values; (3) writing the updated feature values ​​into a specified database and updating the feature value version in the index. To improve processing efficiency, this embodiment of the application divides the above three stages into three threads for processing. Specifically, it requires the use of three threads: the first thread, the second thread, and the third thread, as well as two cache sequences: the first cache sequence and the second cache sequence. First, the first thread (which can be called the image reading and transcoding thread) reads the face image to be processed and transcodes it into image data in a specified format such as YUV. This image data is then written to the first buffer sequence. Next, the second thread (which can be called the algorithm thread) reads the specified format image data from the first buffer sequence, calculates the updated feature values ​​using the image feature extraction algorithm adapted to the changed image storage device, and writes these updated feature values ​​to the second buffer sequence. Finally, the third thread (which can be called the feature value writing thread) reads the updated feature values ​​from the second buffer sequence and writes them to a specified database for storage. A schematic diagram of this process is shown below. Figure 2 As shown. Compared with the traditional single-threaded processing method, this three-threaded processing method can effectively improve data processing efficiency and shorten the time of the entire feature value update operation.

[0078] In addition, to further improve data processing efficiency, both cache sequences can allocate a fixed amount of memory at once during initialization. During data read / write operations, a circular read / write approach is used to avoid the additional time consumption caused by repeatedly allocating and releasing memory. Both cache sequences require locking during data read / write operations to ensure data synchronization. Furthermore, some global variables are needed for information exchange between the three threads (for example, the image reading and transcoding thread needs to inform the other two threads of the number of face images whose feature values ​​need to be updated, so that the other two threads can exit promptly after updating the feature values). In addition, it is important to ensure that the first cache sequence is not empty and the second cache sequence is not full as much as possible. This maximizes the workload of the algorithm threads, reducing the total time spent on feature value update operations.

[0079] In one implementation of this application, after detecting whether a flag file exists on the disk of the image storage device, the method may further include:

[0080] If the flag file exists on the disk of the image storage device, the second algorithm is used to calculate and update the feature values ​​of each face image stored in the user face database; wherein, the user face database is a set of face images consisting of a portion of face images selected by the user from all face images stored in the image storage device.

[0081] In addition to similar face retrieval, this image storage device typically also features real-time comparison with a user's face database. The user's face database is a collection of face images manually added by the user; it can be understood as a subset of faces the user is interested in. When the real-time comparison function of the user's face database is enabled, for newly captured images, the image storage device can retrieve similar face images from the user's face database and display them on the monitoring page in real time. If a similar face image belongs to a face added to the user's blacklist, the image storage device can trigger an alarm. All face images stored on the image storage device can be placed in a general database (such as the face image database mentioned above). Users can select a subset of face images they are particularly interested in from these image databases and transfer them to the user's face database; this operation can be called the face image import operation. Conversely, users can also remove face images from the user's face database; this operation can be called the face image export operation.

[0082] When the image feature extraction algorithm used by the image storage device changes, in order to ensure that the real-time comparison function of the user's face database can function normally, it is also necessary to perform feature value updates on each face image stored in the user's face database. The specific implementation method can be the same as the feature value update method of the optimal face image mentioned above, and will not be repeated here. In addition, since the face images stored in the user's face database are only a small part of all the face images stored in the image storage device, performing feature value updates on the face images in the user's face database will not take a long time.

[0083] In summary, the feature value update process of this application embodiment can be as follows: Figure 3 As shown, when a change in the image feature extraction algorithm version used by the image storage device is detected, the feature values ​​of each face image stored in the user's face database are updated, as are the feature values ​​of each optimal face image. With this configuration, the image storage device can normally provide real-time comparison functionality with the user's face database and similar face retrieval functionality.

[0084] On the other hand, taking a network video recorder (NVR) as an example, in order to ensure the efficiency of feature value updates and the integrity of various NVR functions, some functions of the NVR need to be turned off during the stages of updating the user face database and updating the optimal face image, as shown in Table 1 below:

[0085] Table 1

[0086]

[0087] In Table 1, "prohibited" indicates that the corresponding function needs to be turned off, while "allowed" indicates that the corresponding function does not need to be turned off. When updating the user face database, disabling face capture is to allocate all AI computing power to updating the user face database (specifically, updating the feature values ​​of each face image in the database), minimizing the update time. Face capture requires significant AI computing power. Disabling real-time comparison of the user face database is because face capture is disabled, and no new capture images are generated, so the real-time comparison function is naturally unusable. Disabling the face database entry and exit functions is to avoid data asynchrony. Since the user face database is currently being processed, if the user performs entry and exit operations at this time, it is difficult to guarantee the synchronization of these face images. For example, it is possible that a user has just exited a portion of face images, but after the user face database is updated, these face images reappear in the user face database.

[0088] The three functions—similar face retrieval, image search, and reading captured images from the face database—actually use the same control logic at the underlying level, which can be collectively referred to as the "similar retrieval" function. Because the "similar retrieval" function requires comparing the feature values ​​of face images, and the feature value versions of various face images cannot be guaranteed to be consistent during the user face database update phase and the optimal face image update phase, executing the "similar retrieval" function at this time may cause errors. Therefore, to ensure the correctness and completeness of the image retrieval results, the "similar retrieval" function needs to be disabled during these two phases. Afterwards, once the two feature value update phases are completed, all other functions of the NVR can be enabled normally.

[0089] Furthermore, the method may also include:

[0090] (1) Obtain the face image to be added to the database;

[0091] (2) Determine whether the version of the third algorithm for extracting image features from the face image to be added to the database matches the version of the second algorithm;

[0092] (3) If the version of the third algorithm does not match the version of the second algorithm, the second algorithm is used to calculate and update the feature values ​​of the face image to be added to the database, and then the face image to be added to the database is stored in the user face database.

[0093] When performing a face image import operation, i.e., saving the face image to be imported into the user's face database, the system first checks whether the image feature extraction algorithm version corresponding to the face image to be imported matches the image feature extraction algorithm version used by the image storage device. If the versions match, the face image to be imported can be directly transferred to the user's face database, completing the import operation. If the versions do not match, a feature value update process needs to be performed on the face image to be imported. Specifically, the image feature extraction algorithm used by the image storage device is used to calculate and update the feature values ​​of the face image to be imported, and then the face image to be imported is transferred to the user's face database. A flowchart of the above operation is shown below. Figure 4 As shown.

[0094] 104. Based on the feature values ​​of the first optimal face image, perform face retrieval output on the face image to be retrieved;

[0095] Since the algorithm version for extracting image features from the first optimal face image is the same as the version of the second algorithm used by the image storage device, the image storage device can normally perform feature value comparison based on the feature values ​​of the first optimal face image to perform face retrieval output of the face image to be retrieved.

[0096] In one implementation of this application embodiment, step 104 may include:

[0097] (1) Match the feature values ​​of the first optimal face image with the feature values ​​of each optimal face image stored in the face image database to obtain the feature value matching result;

[0098] (2) Based on the feature value matching result, determine a second optimal face image that is similar to the first optimal face image from all the optimal face images stored in the face image database;

[0099] (3) Output all face images whose face identifiers are the same as those of the second optimal face image stored in the face image database, and output all face images whose face identifiers are the same as those of the face image to be searched stored in the face image database.

[0100] The feature values ​​of the first optimal face image can be matched with the feature values ​​of each optimal face image stored in the face image database of the image storage device (all optimal face images have already undergone feature value updates) to obtain the feature value matching results. Specifically, the feature value matching results here can be similarity scores. For example, if the image storage device stores 30 optimal face images, one of which is the first optimal face image, then the feature values ​​of the first optimal face image are matched with the feature values ​​of the other 29 optimal face images to obtain the similarity scores between each of the other optimal face images and the first optimal face image.

[0101] Then, based on the feature value matching result, each optimal face image similar to the first optimal face image is determined from all the optimal face images stored in the face image database; these are referred to here as the second optimal face images. For example, a similarity threshold can be set; if the similarity between a certain optimal face image and the first optimal face image is higher than this threshold, then the optimal face image is determined to be similar to the first optimal face image. Obviously, the number of second optimal face images found here could be 0, 1, or more.

[0102] Finally, all face images whose face identifiers are the same as those of the second optimal face image stored in the face image database are output. This is the first part of the search results. Since the second optimal face image is a face image similar to the first optimal face image, and the first optimal face image and the face image to be searched belong to the same face, outputting all face images whose face identifiers are the same as those of the second optimal face image is equivalent to outputting all face images similar to the face image to be searched, thus realizing the function of similar face retrieval. For example, if the face identifiers of the face image to be searched and the first optimal face image are both "Zhang San", and there are two second optimal face images found, namely the optimal face image with the face identifier "Li Si" and the optimal face image with the face identifier "Wang Wu" (indicating that the facial features of "Li Si" and "Wang Wu" are similar to "Zhang San"), then the image storage device will output all the saved face images with the face identifier "Li Si" and the face image with the face identifier "Wang Wu". These output face images can be displayed on the display screen of the image storage device or other display devices connected to the image storage device.

[0103] On the other hand, it can also output all face images whose face identifiers are the same as those of the face image to be searched, which is the second part of the search results.

[0104] Face images whose face identifiers are the same as those of the face image to be retrieved (that is, the same as those of the first optimal face image) represent the same face. Therefore, these face images also fall within the range of similarity to the face image to be retrieved. Thus, the image storage device can also output these saved face images as search results.

[0105] 105. Based on the feature values ​​of the face image to be retrieved, perform face retrieval output on the face image to be retrieved.

[0106] The image feature extraction algorithm version corresponding to the face image to be retrieved matches the image feature extraction algorithm version used by the image storage device. At this point, the feature values ​​of the face image to be retrieved can be directly compared to perform a similar face retrieval output. Specifically, since the image feature extraction algorithm versions of all the best face images in the image storage device can be ensured to be consistent with the image feature extraction algorithm version used by the image storage device, the feature values ​​of the face image to be retrieved can be matched with the feature values ​​of each best face image stored in the image storage device to find the portion of best face images similar to the face image to be retrieved. Finally, the image storage device outputs all face images under the face labels of these best face images, as well as all face images whose face labels are the same as the face labels of the face image to be retrieved, thereby realizing the similar face retrieval function.

[0107] In this embodiment, if the image feature extraction algorithm version corresponding to the user-inputted face image to be retrieved is found to be mismatched with the image feature extraction algorithm version used by the image storage device, it indicates that the image feature extraction algorithm used by the image storage device may have been changed, while the user-inputted face image to be retrieved still uses the original algorithm to calculate feature values, so the feature value comparison operation cannot be directly performed. In this case, the image storage device will search for the first optimal face image whose face identifier is the same as the face identifier of the face image to be retrieved (if the face identifier is the same, it means it is an image of the same face), and read the feature value of the first optimal face image as a substitute. Finally, based on the read feature value of the first optimal face image, a similar face retrieval output is performed for the face image to be retrieved. In the above process, since the image feature extraction algorithm version corresponding to the first optimal face image matches the image feature extraction algorithm version used by the image storage device, that is, the first optimal face image has completed the feature value update operation, the image feature value comparison operation can be performed normally. In addition, in this embodiment, feature value updates can be performed on each of the best face images after the image storage device completes the version change of the image feature extraction algorithm. Since the best face images stored in the image storage device are only a small part of all face images stored in the image storage device, the time spent performing image feature value update operations can be effectively reduced.

[0108] In summary, the image storage device of this application embodiment only needs to update the feature values ​​of a subset of face images related to the AI ​​face (i.e., face images in the user's face database and the optimal face image), which significantly reduces the time required compared to updating all face images. Moreover, multi-threaded processing can be used when updating feature values ​​to further improve the efficiency of feature value updates.

[0109] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0110] The above mainly describes a method for retrieving face images. The following will describe a device for retrieving face images.

[0111] Please see Figure 5 This application illustrates an apparatus for retrieving face images, comprising:

[0112] The image acquisition module 501 is used to acquire the image of the face to be retrieved.

[0113] The first algorithm version judgment module 502 is used to determine whether the version of the first algorithm for extracting image features from the face image to be retrieved matches the version of the second algorithm for extracting image features from the face image database.

[0114] The feature value reading module 503 is used to read the feature value of the first optimal face image stored in the face image library if the version of the first algorithm does not match the version of the second algorithm; wherein the face identifier of the first optimal face image is the same as the face identifier of the face image to be retrieved, and the version of the algorithm for image feature extraction of the first optimal face image matches the version of the second algorithm.

[0115] The face retrieval module 504 is used to perform face retrieval output on the face image to be retrieved based on the feature values ​​of the first optimal face image.

[0116] In one implementation of this application, the second algorithm is an image feature extraction algorithm used by an image storage device, the face image library is a database of face images stored in the image storage device, and the apparatus may further include:

[0117] The update detection module is used to detect whether a flag file exists on the disk of the image storage device when the image storage device restarts. The flag file is used to indicate whether to perform a feature value update operation for the face image.

[0118] The first feature value update module is used to calculate and update the feature values ​​of each optimal face image stored in the image storage device using the second algorithm if the flag file exists in the disk of the image storage device; wherein the image storage device stores the optimal face image corresponding to each known face identifier.

[0119] Furthermore, the flag file is written to the disk when the version of the second algorithm changes, and the apparatus may further include:

[0120] The flag file deletion module is used to delete the flag file in the disk.

[0121] Furthermore, the first feature value update module may include:

[0122] An image transcoding unit is used to read the optimal face image using a first thread, transcode the optimal face image into image data of a specified format, and write the image data of the specified format into a first buffer sequence;

[0123] The feature value calculation unit is used to read the image data of the specified format from the first cache sequence using a second thread, calculate the image data of the specified format using the second algorithm to obtain updated feature values, and write the updated feature values ​​into the second cache sequence;

[0124] The feature value writing unit is used to read the updated feature value from the second cache sequence using a third thread and write the updated feature value into a specified database.

[0125] In one implementation of this application, the apparatus may further include:

[0126] The second feature value update module is used to calculate and update the feature values ​​of each face image stored in the user face database using the second algorithm if the flag file exists in the disk of the image storage device; wherein, the user face database is a set of face images composed of a portion of the face images selected by the user from all the face images stored in the image storage device.

[0127] In one implementation of this application, the apparatus may further include:

[0128] The module for acquiring face images to be added to the database is used to acquire face images to be added to the database.

[0129] The second algorithm version judgment module is used to determine whether the version of the third algorithm for extracting image features from the face image to be added to the database matches the version of the second algorithm.

[0130] The face image storage module is used to calculate and update the feature values ​​of the face image to be stored in the database using the second algorithm if the version of the third algorithm does not match the version of the second algorithm, and then store the face image to be stored in the user face database.

[0131] In one implementation of this application, the face retrieval module may include:

[0132] The feature value matching unit is used to match the feature value of the first optimal face image with the feature value of each optimal face image stored in the face image database to obtain the feature value matching result;

[0133] A similar optimal face image determination unit is used to determine a second optimal face image that is similar to the first optimal face image from all the optimal face images stored in the face image database based on the feature value matching result.

[0134] The retrieval result output unit is used to output all face images whose face identifiers stored in the face image database are the same as the face identifier of the second optimal face image, and to output all face images whose face identifiers stored in the face image database are the same as the face identifier of the face image to be retrieved.

[0135] This application embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements... Figure 1 This represents any method for retrieving face images.

[0136] This application also provides a computer program product that, when run on an image storage device, causes the image storage device to execute an implementation such as... Figure 1 This represents any method for retrieving face images.

[0137] Figure 6 This is a schematic diagram of an image storage device provided in an embodiment of this application. For example... Figure 6 As shown, the image storage device 6 of this embodiment includes: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60. When the processor 60 executes the computer program 62, it implements the steps in the embodiments of the various methods for retrieving face images described above, for example... Figure 1 Steps 101 to 105 are shown. Alternatively, when the processor 60 executes the computer program 62, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 5 The functions of modules 501 to 504 are shown.

[0138] The computer program 62 can be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 62 in the image storage device 6.

[0139] The processor 60 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0140] The memory 61 can be an internal storage unit of the image storage device 6, such as a hard disk or RAM of the image storage device 6. The memory 61 can also be an external storage device of the image storage device 6, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the image storage device 6. Furthermore, the memory 61 can include both internal and external storage units of the image storage device 6. The memory 61 is used to store the computer program and other programs and data required by the image storage device. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0141] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0142] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0143] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0144] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0145] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0146] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0147] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0148] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0149] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for retrieving a face image, characterized in that, The method comprises: acquiring a face image to be searched; judging whether a version of a first algorithm for image feature extraction of the face image to be searched matches a version of a second algorithm for image feature extraction of images in a face image library; if the version of the first algorithm does not match the version of the second algorithm, reading feature values of a first optimal face image saved by the face image library; wherein a face identifier of the first optimal face image is the same as a face identifier of the face image to be searched, the same face identifier means the same face image, and the version of the algorithm for image feature extraction of the first optimal face image matches the version of the second algorithm; performing face searching and outputting of the face image to be searched according to the feature values of the first optimal face image; the second algorithm is an image feature extraction algorithm used by an image storage device, and the face image library is a database for saving face images possessed by the image storage device, and the method further comprises: when the image storage device is restarted, detecting whether a flag file exists in a disk of the image storage device, the flag file being used to indicate whether a feature value updating operation of a face image is performed; if the flag file exists in the disk of the image storage device, calculating and updating feature values of each optimal face image saved by the image storage device respectively by using the second algorithm; wherein the image storage device saves an optimal face image corresponding to each known face identifier respectively; the performing of the face searching and outputting of the face image to be searched according to the feature values of the first optimal face image comprises: matching the feature values of the first optimal face image with feature values of each optimal face image saved by the face image library respectively to obtain a feature value matching result; determining a second optimal face image similar to the first optimal face image from all optimal face images saved by the face image library according to the feature value matching result; outputting all face images saved by the face image library and having the same face identifier as the second optimal face image, and outputting all face images saved by the face image library and having the same face identifier as the face image to be searched.

2. The method of claim 1, wherein, the flag file is written into the disk when the version of the second algorithm is changed, and after the calculating and updating of the feature values of each optimal face image saved by the image storage device respectively by using the second algorithm, the method further comprises: deleting the flag file in the disk.

3. The method of claim 1, wherein, the feature values of each optimal face image saved by the image storage device are updated by the following steps: reading the optimal face image by using a first thread, transcoding the optimal face image into image data in a specified format, and writing the image data in the specified format into a first cache sequence; reading the image data in the specified format from the first cache sequence by using a second thread, calculating the image data in the specified format by using the second algorithm to obtain updated feature values, and writing the updated feature values into a second cache sequence; reading the updated feature value from the second cache sequence using a third thread and writing the updated feature value to a designated database.

4. The method of claim 1, wherein, After detecting whether the flag file exists in the disk of the image storage device, further comprising: If the flag file exists in the disk of the image storage device, using the second algorithm to calculate and update the feature value of each face image saved in the user face library respectively; wherein the user face library is a face image set composed of part of face images selected by a user from all face images saved by the image storage device.

5. The method of claim 4, wherein, Further comprising: Obtaining a face image to be stored in the library; Judging whether the version of a third algorithm for image feature extraction of the face image to be stored in the library matches the version of the second algorithm; If the version of the third algorithm does not match the version of the second algorithm, using the second algorithm to calculate and update the feature value of the face image to be stored in the library, and then storing the face image to be stored in the library into the user face library.

6. An apparatus for retrieving a face image, the apparatus comprising: Comprising: A face image to be searched obtaining module for obtaining a face image to be searched; A first algorithm version judging module for judging whether the version of a first algorithm for image feature extraction of the face image to be searched matches the version of a second algorithm for image feature extraction of images in a face image library; A feature value reading module for reading the feature value of a first optimal face image saved by the face image library if the version of the first algorithm does not match the version of the second algorithm; wherein the face identifier of the first optimal face image is the same as the face identifier of the face image to be searched, the same face identifier means the same face image, and the version of the algorithm for image feature extraction of the first optimal face image matches the version of the second algorithm; A face searching module for performing face searching output of the face image to be searched according to the feature value of the first optimal face image The second algorithm is an image feature extraction algorithm used by an image storage device, the face image library is a database for saving face images possessed by the image storage device, and the device further comprises: An update detecting module for detecting whether a flag file exists in the disk of the image storage device when the image storage device is restarted, the flag file being used to indicate whether a feature value updating operation of a face image is performed; A first feature value updating module for calculating and updating the feature value of each optimal face image saved by the image storage device using the second algorithm respectively if the flag file exists in the disk of the image storage device; wherein the image storage device saves an optimal face image corresponding to each known face identifier respectively; The face searching module comprises: A feature value matching unit for matching the feature value of the first optimal face image with the feature value of each optimal face image saved by the face image library respectively to obtain a feature value matching result. The similar optimal face image determining unit is configured to determine a second optimal face image similar to the first optimal face image from all optimal face images stored in the face image library according to the feature value matching result. The search result output unit is configured to output all face images stored in the face image library and having the same face identity as the second optimal face image, and output all face images stored in the face image library and having the same face identity as the face image to be searched.

7. An image storage device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor implements the method for searching a face image according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising: The computer program implements the method for searching a face image according to any one of claims 1 to 5 when executed by the processor.

Images