Image search method, device, storage medium and electronic device

Abstract

Embodiments of the present invention provide an image search method, apparatus, storage medium, and electronic device. The method includes: upon determining that an image similar to a first image needs to be searched, sending the first image to a first device to instruct the first device to extract and return first feature information of the first image; comparing the first feature information with feature information pre-stored in a target database to obtain a comparison result, the comparison result being used to indicate the similarity between the first feature information and the feature information pre-stored in the target database, the feature information pre-stored in the target database being feature information reported after multiple devices extract features from captured images; and displaying a target image corresponding to feature information whose similarity to the first feature information exceeds a predetermined similarity threshold on a target display interface. The present invention solves the problem in the related art that image search requires a large number of devices, resulting in high costs.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of communications, and in particular, to an image search method, device, storage medium, and electronic device. Background Art

[0002] With the development of science and technology and the progress of society, "fast, convenient, and safe" have become synonymous with contemporary society. In the rapidly developing modern society, people increasingly hope to live in a safe environment, especially their personal safety and property safety. Therefore, the development and application of computer technology have become a top priority. In recent years, with the development of applications such as e-commerce, people's technical requirements for safe, fast, effective, and automatic face recognition have become increasingly urgent.

[0003] There are two main image search methods in the related art. One method is to search across multiple IVSS devices. In this method, the FR / FD camera needs to be connected to the IVSS device. The platform simultaneously initiates image search requests to multiple IVSS devices and then merges the search results from multiple IVSS devices. The image search of the FR / FD camera is strongly dependent on the IVSS device. The FR / FD camera needs to be connected to the IVSS device and the platform then requests the image search from the IVSS device. Therefore, users who have already purchased the FR / FD camera need to purchase an additional IVSS device to support image search of the FR / FD camera's captured images. In addition, the platform simultaneously initiates image search requests to multiple IVSS devices and needs to wait until all IVSS devices have completed the search before the search results can be correctly sorted. Waiting for all searches to complete can take several minutes or even longer, resulting in a poor user experience. Another method is for the software platform to deploy an intelligent server at the center dedicated to extracting facial feature information and image search services. This requires users to purchase an additional intelligent server and is wasteful for small scenarios.

[0004] Regarding the existing FR / FD cameras in the related technologies, image search can only be realized by connecting to IVSS equipment, or an additional intelligent server needs to be deployed so that the intelligent server can realize image search. As a result, image search requires more equipment, which leads to the problem of high cost. No effective solution has been proposed yet. Summary of the Invention

[0005] Embodiments of the present invention provide an image search method, device, storage medium, and electronic device to at least solve the problem in the related art that image search requires a large number of devices, resulting in high costs.

[0006] According to one embodiment of the present invention, an image search method is provided, comprising: in a case where it is determined that an image similar to a first image needs to be searched, sending the first image to a first device to instruct the first device to extract and return first feature information of the first image; comparing the first feature information with feature information pre-stored in a target database to obtain a comparison result, wherein the comparison result is used to indicate the similarity between the first feature information and the feature information pre-stored in the target database, the feature information pre-stored in the target database being feature information reported after multiple devices perform feature extraction on captured images; and displaying a target image corresponding to feature information whose similarity with the first feature information exceeds a predetermined similarity threshold in a target display interface.

[0007] In an exemplary embodiment, sending the first image to a first device to instruct the first device to extract and return first feature information of the first image includes: receiving an input search instruction, wherein the search instruction is used to instruct a search for an image similar to the first image from images taken by a second device included in a plurality of the devices; determining a target algorithm adopted by the second device when performing feature extraction on the image taken by the second device; and sending the first image to the first device for feature extraction using the target algorithm to instruct the first device to extract and return the first feature information of the first image using the target algorithm.

[0008] In an exemplary embodiment, sending the first image to the first device for feature extraction using the target algorithm to instruct the first device to extract and return the first feature information of the first image using the target algorithm includes: when there are multiple second devices and the target algorithms adopted by multiple second devices when performing feature extraction are multiple versions of target algorithms, sending the first image to the devices included in the first device for using each version of the target algorithm included in the multiple versions, respectively, to instruct each device included in the first device to extract and return the first feature information of the first image according to the target algorithm used by itself.

[0009] In an exemplary embodiment, before receiving the input search instruction, the method also includes: receiving a device information acquisition request; and displaying the device information of multiple devices in a target display manner under the triggering of the acquisition request to indicate that the search instruction is input on the interface for displaying the device information of multiple devices.

[0010] In an exemplary embodiment, before comparing the first feature information with the feature information pre-stored in the target database to obtain a comparison result, the method also includes: obtaining target data reported by multiple devices, wherein the target data includes the feature information; and storing the target data in blocks according to the type of data included in the target data.

[0011] In an exemplary embodiment, displaying the target image corresponding to the feature information whose similarity with the first feature information exceeds a predetermined similarity threshold in the target display interface includes at least one of the following: displaying the target images in sequence from large to small similarity; displaying the target images in sequence from near to far search time.

[0012] In an exemplary embodiment, when it is determined that an image similar to the first image needs to be searched, the method further includes: when it is determined that there is a predetermined device that does not support reporting of the extracted feature information, and the predetermined device has an image search function, sending the first image to the predetermined device to instruct the predetermined device to search for and return images in the images taken by itself whose similarity with the first image exceeds the predetermined similarity threshold.

[0013] According to another embodiment of the present invention, an image search device is provided, including: a sending module for sending the first image to a first device when it is determined that an image similar to the first image needs to be searched, so as to instruct the first device to extract and return first feature information of the first image; a comparison module for comparing the first feature information with feature information pre-stored in a target database to obtain a comparison result, wherein the comparison result is used to indicate the similarity between the first feature information and the feature information pre-stored in the target database, and the feature information pre-stored in the target database is feature information reported by multiple devices after feature extraction on the captured images; a display module for displaying a target image corresponding to feature information whose similarity with the first feature information exceeds a predetermined similarity threshold in a target display interface.

[0014] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is configured to execute the steps of any one of the above method embodiments when run.

[0015] According to another embodiment of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any one of the above method embodiments.

[0016] According to the present invention, feature information of an image can be stored in advance in a target database. When searching for an image similar to the first image, the first image can be sent to a first device, thereby instructing the first device to extract and return first feature information of the first image and compare the first feature information with the feature information of the image pre-stored in the target database. The comparison result can be the similarity between the first feature information and the feature information of the image pre-stored in the target database, and the target image corresponding to the feature information whose similarity to the first feature information exceeds a predetermined similarity threshold is displayed on the target interface. The feature information of the image pre-stored in the target database can be information reported by multiple devices after feature extraction on the captured images. This achieves the situation where, when the camera device itself does not support image search by image, other existing devices that support feature extraction can be used to extract features, and then determine images similar to the image to be searched through feature comparison. This avoids the need to add additional auxiliary equipment when the camera itself does not support the image search by image function. This effectively solves the problem in the related art that image search by image requires a large number of devices, resulting in high costs, and thus achieves the effect of supporting cross-device image search, improving the efficiency of image search by image, and reducing search costs, further enhancing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a hardware structure block diagram of a mobile terminal for an image search method according to an embodiment of the present invention;

[0018] Figure 2 is a flowchart of an image search method according to an embodiment of the present invention;

[0019] Figure 3 This is a diagram of the overall system architecture consisting of a front-end FR / FD camera, a back-end IVSS smart device, and a software platform according to an embodiment of the present invention;

[0020] Figure 4 is a flowchart of an image search method according to a specific embodiment of the present invention;

[0021] Figure 5 is a structural block diagram of an image search device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0022] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with embodiments.

[0023] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0024] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG is a hardware structure block diagram of a mobile terminal of an image search method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown) a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA and other processing devices) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0025] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the image search method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above-mentioned method. The memory 104 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0026] The transmission device 106 is used to receive or send data via a network. A specific example of the aforementioned network may include a wireless network provided by the mobile terminal's communications provider. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In one embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0027] In this embodiment, an image search method is provided. Figure 2 is a flow chart of an image search method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0028] S202: If it is determined that an image similar to the first image needs to be searched, send the first image to a first device to instruct the first device to extract and return first feature information of the first image;

[0029] S204: Compare the first feature information with feature information pre-stored in a target database to obtain a comparison result, wherein the comparison result indicates a similarity between the first feature information and the feature information pre-stored in the target database, where the feature information pre-stored in the target database is feature information reported by multiple devices after feature extraction on captured images;

[0030] S206: Displaying a target image corresponding to feature information whose similarity with the first feature information exceeds a predetermined similarity threshold in the target display interface.

[0031] The operations described above may be performed by a platform, such as a software platform, a processor, a server, or other processing devices or processing units with similar processing capabilities. The first device may be an intelligent device, such as an IVSS device that supports image feature extraction, a FR / FD camera, etc. The following uses the platform performing the operations described above as an example (this is merely an illustrative description; in actual operations, other devices or modules may also be used to perform the operations):

[0032] In the above embodiment, when it is necessary to search for an image similar to the first image, the platform sends the first image to the first device to instruct the first device to extract and return the first feature information of the first image. In actual applications, there are multiple devices connected to the platform, including some devices that support image feature extraction and some devices that do not support image feature extraction. When a device that does not support image feature extraction needs to perform an image search, it can request other devices that support image feature extraction through the platform to perform feature extraction on the image to be searched, thereby achieving the purpose of cross-device feature extraction, so that other images similar to the image to be searched can be searched through the features extracted across devices. In addition, in addition to searching for images similar to the first image, the platform can also continuously search for images similar to other images in a similar manner. In addition, the number of the above-mentioned first images can be one or more. When the number of the above-mentioned first images is multiple, images similar to each first image can be searched in sequence in a similar search method.

[0033] In the above embodiment, multiple devices supporting image feature extraction can be used in advance to extract feature information from the captured images. After the device performs feature extraction, the corresponding feature information can be reported to the platform at the same time as the device reports the captured image to the platform. Of course, the device can also extract feature information from images not captured by the device. After receiving this feature information, the platform will store it in the platform's database. Then, when it is determined that an image similar to the first image needs to be searched, the platform can directly call the feature information stored in the target database and compare it with the first feature information of the first image extracted and returned by the first device to obtain a comparison result. The platform can pre-store feature information of a large amount of images, and the device can implement the reporting of the features of the captured images. Of course, the features of the captured images can also be reported according to a fixed period, for example, extracting feature information of all images captured in the current period once every 24 hours, every 12 hours, or every 3 hours, and then reporting this feature information to the platform. In addition, if the platform detects that the newly received feature information is the same as the feature information historically stored in the database, it can overwrite the feature information historically stored in the database to update it to the latest feature information, and the remaining feature information received that is different from the feature information historically stored in the database can be directly stored in the database.

[0034] In the above embodiment, a similarity threshold can be set in advance. For example, the similarity threshold can be set to 90%. Then, the platform can compare the first feature information with the feature information pre-stored in the database, and the image corresponding to the feature information with a similarity greater than or equal to 90% can be sent to the display interface of the requester who requests a similar image search. Of course, the similarity threshold can also be set to 80%, 85%, 95%, etc. The similarity threshold can be flexibly adjusted according to the actual application scenario.

[0035] In the above embodiment, when the camera device itself does not support image search by image, it is possible to use other existing devices that support feature extraction to extract features, and then determine images similar to the image to be searched through feature comparison, thereby avoiding the need to add other auxiliary devices when the camera itself does not support the image search by image function, and effectively solving the problem of large number of devices required for image search by image in related technologies, which leads to high cost, thereby achieving the effect of supporting cross-device image search, improving the efficiency of image search by image, reducing search costs, and further enhancing the user experience.

[0036] In an exemplary embodiment, sending the first image to a first device to instruct the first device to extract and return first feature information of the first image includes: receiving an input search instruction, wherein the search instruction is used to instruct a search for an image similar to the first image from images captured by a second device included in the plurality of devices; determining a target algorithm used by the second device to extract features from the image captured by the second device; and sending the first image to the first device for feature extraction using the target algorithm to instruct the first device to extract and return the first feature information of the first image using the target algorithm. In this embodiment, when requesting a similar image search, it is possible to additionally indicate which devices' images to search for similar images. When searching for similar images in images captured by a second device, it is first necessary to extract features of the first image to be searched according to the algorithm used by the second device to extract image features. Generally, feature information extracted using similar algorithms can be compared, so it is necessary to extract feature information of the first image according to the algorithm used by the second device. In actual applications, the algorithms used by each device to extract features from the images it captures are also different. Each device may have its own algorithm, or all devices may use the same algorithm, and so on. In addition, the above-mentioned search instructions can be instructions entered by the user (or the requester) on the search interface displayed on a mobile terminal or a computer terminal.

[0037] In an exemplary embodiment, sending the first image to the first device for feature extraction using the target algorithm to instruct the first device to extract and return the first feature information of the first image using the target algorithm includes: when there are multiple second devices and the target algorithms adopted by multiple second devices when performing feature extraction are multiple versions of target algorithms, sending the first image to the devices included in the first device for using each version of the target algorithm included in the multiple versions, respectively, to instruct each device included in the first device to extract and return the first feature information of the first image according to the target algorithm used by itself. In this embodiment, when there are multiple second devices and the target algorithms used by the multiple second devices when performing feature extraction are multiple versions of the target algorithm, the platform can send the first image to the devices included in the first device for using each version of the target algorithm included in the multiple versions, and instruct these devices to extract and return the first feature information of the first image according to the target algorithm they use. For example, when there are two versions of the algorithm used by the multiple second devices, that is, the first version and the second version, a device that supports the first version of the algorithm and a device that supports the second version of the algorithm (these two devices are the above-mentioned first devices) can be determined, and then the first image can be sent to the two devices respectively, so that the two devices can extract the corresponding feature information according to the algorithms they support. Of course, there may also be a situation where the same device supports multiple versions of the algorithm. When there is a device that supports both the first version and the second version of the algorithm, the first image can be sent to the device so that the device can extract two types of feature information. When multiple second devices use the same version of the target algorithm when performing feature extraction, the platform can send the first image to any one of the second devices (and use the device as the first device mentioned above), and instruct the device to extract and return the first feature information of the first image according to the target algorithm used by itself.

[0038] In an exemplary embodiment, before receiving the input search instruction, the method further includes: receiving a device information acquisition request; and displaying the device information of multiple devices in a target display manner under the triggering of the acquisition request, so as to indicate that the search instruction is input on the interface for displaying the device information of multiple devices. In this embodiment, when the platform receives the device information acquisition request, the device information of multiple or all of the devices is displayed on the interface, wherein the display manner can be multiple, for example, directly displaying the device information of each device in sequence, or displaying the device information of each device in the form of a device tree (i.e., displaying the device information hierarchically in the form of nodes) on the interface where the user or operator inputs the search instruction, so that the user or operator can select the above-mentioned second device, wherein the displayed device information can include the device name, device IP address, device channel, device intelligent capability, etc. In addition, the user or operator can also input the image to be searched on the interface for inputting the search instruction, so as to indicate that an image similar to the image to be searched is searched from the images taken by the above-mentioned second device.

[0039] In an exemplary embodiment, before comparing the first feature information with feature information pre-stored in a target database to obtain a comparison result, the method further includes: obtaining target data reported by multiple devices, wherein the target data includes the feature information; and storing the target data in blocks according to the type of data included in the target data. In this embodiment, the target data may include basic data (e.g., device code, image URL, recognized facial attributes, etc.) and facial feature information (e.g., facial feature values, facial feature algorithm version, etc.). In actual applications, the basic data and facial feature information can be stored separately, thereby avoiding the leakage of stored data when the database is out of the database, further enhancing the security of information storage.

[0040] In an exemplary embodiment, displaying the target image corresponding to the feature information whose similarity with the first feature information exceeds a predetermined similarity threshold in the target display interface includes at least one of the following: displaying the target images in sequence from large to small similarity; displaying the target images in sequence from near to far search time. In this embodiment, there may be multiple target images corresponding to feature information whose similarity with the first feature information exceeds a predetermined similarity threshold, so the target images need to be displayed in batches to the display interface of the user or operator, wherein the number of target images that the display interface can accommodate can be pre-set. For example, the first 1000 (of course, it can also be the first 1100, the first 900, the first 800, etc.) target images arranged in order of similarity from large to small can be preferentially paged and displayed to the display interface of the user or operator. For example, the first 100 target images among the first 1000 target images are preferentially displayed to the first page of the user display interface. In addition, the first 1000 (of course, it can also be the first 1100, the first 900, the first 800, etc.) target images with the earliest search time can also be preferentially paged and displayed to the display interface of the user or operator in order of search time from near to far. For example, the first 100 target images among the first 1000 target images are preferentially displayed to the first page of the user display interface, etc.

[0041] In an exemplary embodiment, when it is determined that a search for images similar to the first image is required, the method further includes: when it is determined that a predetermined device exists that does not support reporting the extracted feature information, and the predetermined device has a search-by-image function, sending the first image to the predetermined device to instruct the predetermined device to search for and return images whose similarity to the first image exceeds a predetermined similarity threshold in its own captured images. In this embodiment, a user or operator can select multiple devices from a device tree for search-by-image, and some of the selected devices may not support reporting the extracted feature information to the platform, but the predetermined device has a search-by-image function. Then, the platform can send the first image to these devices, thereby instructing them to search for and return images whose similarity to the first image exceeds a predetermined similarity threshold in their own captured images. The devices can then extract and compare the feature information of the first image and their own captured images according to their own algorithms, thereby determining whether there are images similar to the first image in their own captured images.

[0042] Obviously, the embodiments described above are only part of the embodiments of the present invention, rather than all the embodiments.

[0043] The present invention will be described in detail below with reference to specific embodiments:

[0044] Figure 3FIG is a diagram showing the overall system architecture consisting of a front-end FR / FD camera, a back-end IVSS intelligent device, and a software platform according to an embodiment of the present invention. Figure 3 As shown, the front-end FR / FD camera and the IVSS back-end intelligent device are both connected to the software platform. Both the FR / FD camera and the IVSS back-end intelligent device can capture and recognize faces, and then extract the feature values ​​of the captured images according to their own algorithms, and report the extracted facial feature values ​​(corresponding to the above-mentioned feature information) and algorithm versions to the software platform. The software platform saves the received facial feature values ​​and algorithm versions to the database, and integrates the facial feature comparison algorithm for facial feature value comparison and integrated retrieval of comprehensive data.

[0045] Figure 4 is a flow chart of an image search method according to a specific embodiment of the present invention. Figure 4 As shown, the process includes the following steps:

[0046] S402: A terminal (e.g., a mobile terminal or a computer terminal) inputs the original image and image search conditions (corresponding to the device information selected by the operator or user and the similarity threshold, etc.);

[0047] The device (i.e., any device connected to the platform with camera and feature extraction capabilities) captures a face and identifies facial features. The device then reports the captured and recognized facial information, including facial feature values ​​and the facial algorithm version (only feature values ​​with the same algorithm version are compared), to the software platform via a TCP over TLS long connection (which ensures the security of the transmission of facial sensitive data, which are facial feature values ​​and facial algorithm version). To avoid network congestion or data loss caused by network latency or excessive data concurrency during data transmission, the software platform and device must implement a negotiated confirmation mechanism to ensure data transmission reliability. After receiving processed data from a device, the platform immediately sends an ACK to the device to confirm the data. If the device does not receive an ACK from the platform within a few seconds, the data is placed in a retransmission queue, awaiting reupload to the software platform. To avoid network storms caused by data retransmission, the retransmission interval can be gradually increased based on the number of upload failures.

[0048] The platform receives data reported by the device in real time and stores it persistently. The software platform can receive facial data (i.e., facial feature values ​​and facial algorithm versions) reported by the device in real time, and save the facial data in persistent partitioned tables in the software's platform database. The partitioned tables are used to ensure the security of facial data, and the basic data (e.g., device code, image URL (uniform resource locator), recognized facial attributes, etc.) and facial feature information (e.g., facial feature values, facial algorithm versions) are stored separately, thereby avoiding facial data leakage when the database is out of storage. The facial feature values ​​are encrypted and stored using the AES algorithm, further avoiding the leakage of facial features.

[0049] The user performs an image search operation. The user enters the user name and password to log in to the software platform and obtains the information of all facial intelligent devices (for example, device name, device IP address, device channel, device intelligent capabilities, etc.) from the platform service through the HTTPS (Hyper Text Transfer Protocol over Secure Socket Layer) protocol. The platform will display all device information on the user's display interface in the form of a device tree. The user uploads the original image to be searched (corresponding to the first image above) and selects the device to be searched from the device tree (corresponding to the second device above, multiple selections can be supported). The format of the original image uploaded by the user is required to be JPEG or PNG format, and multiple original images can be supported. Figure 1 The maximum face library capacity for a single search (i.e., a search of a single original image) is 20 million, and cross-database search is supported.

[0050] S404: The platform instructs a specific device (corresponding to the first device) to extract feature values ​​of the original image uploaded by the user;

[0051] S406, if the algorithm versions of the second devices are all the same, only one specific device needs to upload the successfully extracted original image feature values ​​to the platform;

[0052] After the software platform receives the user's image search request, it will retrieve the intelligent algorithm version corresponding to the device selected by the user (corresponding to the above-mentioned target algorithm version) from the MySQL database (corresponding to the above-mentioned target database) based on the device tree information selected by the user. If the intelligent algorithm versions of all devices in the device tree are consistent, the software platform will request any device selected by the user to extract the feature values ​​of the original image. If the intelligent algorithm versions of devices in the device tree are inconsistent, the software platform will extract the original image feature values ​​once from devices with different algorithm versions (the results obtained after comparing the facial feature values ​​of devices with consistent algorithm versions are more accurate). In addition, to ensure the security of facial data, the software platform batch sends requests to extract the original image feature values ​​to the corresponding devices through the TCP over TLS encrypted long link channel.

[0053] S408: If the second device has a device with an inconsistent algorithm version, obtain a stretching table (i.e., a similarity stretching table) of original image feature values ​​extracted by devices with different algorithm versions in the specific device;

[0054] S410, uploading the above-mentioned original image feature value stretching table (similarity stretching table) of different algorithm versions successfully extracted to the platform;

[0055] S412: After receiving the original image feature value stretching table (similarity stretching table), the platform searches for historical facial feature values ​​in the database and facial feature values ​​in the face library in batches according to the table;

[0056] The specific device extracts the feature values ​​of the original image based on the platform's instructions. After receiving the feature values ​​of the original image returned by the specific device, the platform decrypts the historical facial feature values ​​in the database and the facial feature values ​​in the facial base library in batches according to the predetermined device information checked in the device tree and in reverse chronological order through AES and loads them into the cache. Then the platform service runs the facial feature comparison algorithm, and compares the feature values ​​of the original image with the historical facial feature values ​​loaded into the cache and the facial feature values ​​of the face library one by one through the facial feature comparison algorithm (that is, the feature values ​​of the original image are compared with the facial feature values ​​in the cache, and the operation results are used as the index in the facial feature comparison vector table, so as to retrieve the similarity corresponding to the index in the facial feature comparison vector table), and compare the similarity percentage results.

[0057] S414, the platform obtains historical facial feature values ​​and facial feature values ​​in the database and the face base database in batches that conform to the original image feature value stretching table (similarity stretching table);

[0058] In step S416, the platform compares the feature values ​​of the original image with the historical facial feature values ​​and the facial feature values ​​in the face database obtained in step S414, and preferentially obtains the top 1,000 results that meet the image search criteria.

[0059] S418, returning the first 1000 comparison results obtained in the above step S416 to the user in batches.

[0060] Taking into account the actual application scenarios and improving the search speed, the platform compares the feature values ​​of the original image with the historical face feature values ​​loaded into the cache and the face feature values ​​of the face library in reverse chronological order by image search, and the first 1,000 records with the most recent return time that meet the similarity conditions (corresponding to the above-mentioned similarity threshold) are displayed on the user's display interface first. If the user's first page display interface cannot accommodate 1,000 records, the search results can be returned to the client in pages (that is, the user can see the first batch of search results as soon as possible). The maximum face library capacity for one search is 20 million.

[0061] It should also be noted that, considering the compatibility between the platform and the device, there may be cases where some devices do not report facial feature values ​​to the platform. If the platform detects that a device has not reported feature values ​​(that is, the device does not have a record of corresponding facial historical features in the database), but the device supports the image search function, the platform will automatically send a request to the device to search by image (that is, concurrently search instructions to multiple devices), and then merge and sort the search results of all devices that have completed the search, and return them to the client for display.

[0062] It can be seen from the aforementioned embodiments that the present invention integrates the device support for feature extraction with the platform (supporting cross-device search) and runs the feature value comparison and fusion retrieval algorithms. Without increasing the hardware performance requirements, it fully utilizes current resources, and the platform and device work together to support cross-device image search. At the same time, it solves the problem that FR / FD cameras themselves do not support the image search function, and can also be applied to miniaturized scenarios.

[0063] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0064] In this embodiment, an image search device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments. Details that have already been described will not be repeated. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0065] Figure 5 is a structural block diagram of an image search device according to an embodiment of the present invention. Figure 5 As shown, the device includes:

[0066] a sending module 52 configured to, when determining that an image similar to the first image needs to be searched, send the first image to the first device to instruct the first device to extract and return first feature information of the first image;

[0067] a comparison module 54 configured to compare the first feature information with feature information pre-stored in a target database to obtain a comparison result, wherein the comparison result indicates a degree of similarity between the first feature information and the feature information pre-stored in the target database, wherein the feature information pre-stored in the target database is feature information reported by multiple devices after feature extraction on captured images;

[0068] The display module 56 is configured to display, in a target display interface, a target image corresponding to feature information whose similarity with the first feature information exceeds a predetermined similarity threshold.

[0069] In an exemplary embodiment, the sending module 52 includes:

[0070] a receiving unit, configured to receive an input search instruction, wherein the search instruction is used to instruct to search for an image similar to the first image from images captured by a second device included in the plurality of devices;

[0071] a determining unit, configured to determine a target algorithm used by the second device when performing feature extraction on an image captured by the second device;

[0072] A sending unit is used to send the first image to the first device used for feature extraction using the target algorithm, so as to instruct the first device to extract and return the first feature information of the first image using the target algorithm.

[0073] In an exemplary embodiment, the sending unit includes:

[0074] A sending subunit is used to send the first image to the devices included in the first device for using each version of the target algorithm included in the multiple versions, when there are multiple second devices and the target algorithms used by the multiple second devices when performing feature extraction are multiple versions of the target algorithm, so as to instruct each device included in the first device to extract and return the first feature information of the first image according to the target algorithm used by itself.

[0075] In an exemplary embodiment, the apparatus further comprises:

[0076] A receiving module, configured to receive a device information acquisition request before receiving an input search instruction;

[0077] A display module is used to display the device information of the multiple devices in a target display manner under the triggering of the acquisition request, so as to instruct the input of the search instruction on the interface used to display the device information of the multiple devices.

[0078] In an exemplary embodiment, the apparatus further comprises:

[0079] an acquisition module, configured to acquire target data reported by a plurality of devices before comparing the first feature information with feature information pre-stored in a target database to obtain a comparison result, wherein the target data includes the feature information;

[0080] The storage module is used to store the target data in blocks according to the type of data included in the target data.

[0081] In an exemplary embodiment, the display module 56 includes at least one of the following:

[0082] A first display unit is used to display the target images in descending order of similarity;

[0083] The second display unit is used to display the target images in order from near to far according to the search time.

[0084] In an exemplary embodiment, the apparatus further comprises:

[0085] A determination module is used to send the first image to the predetermined device when it is determined that an image similar to the first image needs to be searched, and when it is determined that there is a predetermined device that does not support reporting the extracted feature information, and the predetermined device has an image search function, to instruct the predetermined device to search for and return an image whose similarity with the first image exceeds the predetermined similarity threshold among the images taken by the predetermined device.

[0086] It should be noted that the above modules can be implemented through software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0087] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any one of the above method embodiments when running.

[0088] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0089] An embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0090] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0091] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail here.

[0092] Obviously, those skilled in the art will appreciate that the various modules or steps of the present invention described above can be implemented using a general-purpose computing device, can be centralized on a single computing device, or can be distributed across a network of multiple computing devices. They can be implemented using program code executable by the computing device, and thus, can be stored in a storage device and executed by the computing device. In some cases, the steps shown or described herein can be performed in a different order than that shown, or can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0093] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. An image search method, characterized in that: include: If it is determined that an image similar to the first image needs to be searched, the first image is sent to a first device to instruct the first device to extract and return first feature information of the first image. If it is determined that there is a predetermined device that does not support reporting the extracted feature information, and the predetermined device has an image search function, the first image is sent to the predetermined device to instruct the predetermined device to search for images captured by the predetermined device and return images whose similarity to the first image exceeds a predetermined similarity threshold. The predetermined device is configured to extract feature information of the first image and the image captured by the predetermined device according to an algorithm used by the predetermined device, and compare the feature information to determine whether there is an image similar to the first image in the image captured by the predetermined device. Comparing the first feature information with feature information pre-stored in a target database to obtain a comparison result, wherein the comparison result indicates a similarity between the first feature information and the feature information pre-stored in the target database, where the feature information pre-stored in the target database is feature information reported by multiple devices after feature extraction on captured images; Displaying a target image corresponding to feature information whose similarity with the first feature information exceeds a predetermined similarity threshold in a target display interface; The method also includes: displaying the device information in the form of a device tree on the user's display interface; after receiving the user's image search request, retrieving the intelligent algorithm version corresponding to the second device from the target database according to the device tree information selected by the user, and when the intelligent algorithm versions of all devices in the device tree are consistent, requesting any one of the second devices to extract the first feature information; when the intelligent algorithm versions of devices in the device tree are inconsistent, the first feature information is extracted respectively by devices with different intelligent algorithm versions.

2. The method according to claim 1, characterized in that Sending the first image to the first device to instruct the first device to extract and return first feature information of the first image includes: receiving an input search instruction, wherein the search instruction is used to instruct searching for an image similar to the first image from images captured by a second device included in the plurality of devices; determining a target algorithm used by the second device when performing feature extraction on an image captured by the second device; The first image is sent to the first device for extracting features using the target algorithm, so as to instruct the first device to extract and return the first feature information of the first image using the target algorithm.

3. The method according to claim 2, characterized in that Sending the first image to the first device for performing feature extraction using the target algorithm to instruct the first device to extract and return the first feature information of the first image using the target algorithm includes: When there are multiple second devices and the target algorithms adopted by the multiple second devices when performing feature extraction are multiple versions of the target algorithm, the first image is sent separately to the devices included in the first device for using each version of the target algorithm included in the multiple versions, so as to instruct each device included in the first device to extract and return the first feature information of the first image according to the target algorithm used by itself.

4. The method according to claim 2, characterized in that Before receiving the input search instruction, the method further includes: Receive a request to obtain device information; Under the triggering of the acquisition request, the device information of the plurality of devices is displayed in a target display manner to indicate inputting the search instruction on the interface for displaying the device information of the plurality of devices.

5. The method according to claim 1, wherein Before comparing the first feature information with feature information pre-stored in a target database to obtain a comparison result, the method further includes: Acquire target data reported by multiple devices, wherein the target data includes the feature information; The target data is stored in blocks according to the type of data included in the target data.

6. The method according to claim 1, characterized in that Displaying a target image corresponding to feature information whose similarity with the first feature information exceeds a predetermined similarity threshold in the target display interface includes at least one of the following: Display the target images in descending order of similarity; The target images are displayed in sequence according to the search time from near to far.

7. An image search device, characterized in that: include: a sending module for, if it is determined that an image similar to the first image needs to be searched, sending the first image to a first device to instruct the first device to extract and return first feature information of the first image; and, if it is determined that there is a predetermined device that does not support reporting the extracted feature information and the predetermined device has an image search function, sending the first image to the predetermined device to instruct the predetermined device to search for and return images with a similarity to the first image exceeding a predetermined similarity threshold among images captured by the predetermined device, the predetermined device being configured to extract and compare feature information of the first image and images captured by the predetermined device according to an algorithm used by the predetermined device to determine whether there is an image similar to the first image among the images captured by the predetermined device; a comparison module, configured to compare the first feature information with feature information pre-stored in a target database to obtain a comparison result, wherein the comparison result indicates a similarity between the first feature information and the feature information pre-stored in the target database, wherein the feature information pre-stored in the target database is feature information reported after feature extraction is performed on captured images by multiple devices; A display module, configured to display, in a target display interface, a target image corresponding to feature information whose similarity with the first feature information exceeds a predetermined similarity threshold; The device is also used to: display device information in the form of a device tree on the user's display interface; after receiving the user's image search request, retrieve the intelligent algorithm version corresponding to the second device from the target database according to the device tree information selected by the user; when the intelligent algorithm versions of all devices in the device tree are consistent, request any one of the second devices to extract the first feature information; when the intelligent algorithm versions of devices in the device tree are inconsistent, the first feature information is extracted respectively by devices with different intelligent algorithm versions.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

Images