Multi-platform image search method, device, electronic device and storage medium

Abstract

The present application relates to a multi-platform image search method, device, electronic device and storage medium, wherein the method includes: obtaining a feature vector of a to-be-searched image based on an image search platform; performing a similarity comparison between the feature vector of the to-be-searched image and the feature vectors of all captured and recorded images on an edge intelligent platform, screening out images that meet preset conditions from the captured and recorded images as preliminary screening images, and obtaining the first detection frame of the preliminary screening images in the edge intelligent platform; based on the image search platform, searching the to-be-searched image according to the preliminary screening images and the corresponding first detection frame to obtain the search results. Through this application, the problem that the search results fail to meet the expected similarity requirements, resulting in the need to re-examine features, causing waste of resources and reducing search efficiency is solved; preliminary screening images that meet the conditions are initially found through preliminary screening, avoiding the re-examination of a large number of irrelevant data algorithm features, reducing resource waste and improving search efficiency.

Description

Technical Field

[0001] The present application relates to the field of image search technology, and in particular to a multi-platform image search method, device, system, electronic device, and storage medium. Background Art

[0002] With the continuous advancement of network technology, image searches can now be performed through search platforms. Currently, image search services are generally performed on the platform side, often involving edge intelligence. This means that some object detection and feature extraction may be handled on the edge intelligence algorithm detection platform, resulting in switching between multiple algorithm detection platforms. For example, while object detection and feature extraction are performed on platform A for the image to be searched, the feature vectors of the captured image to be searched may come from edge intelligence platforms such as algorithm detection platform B, algorithm detection platform C, or algorithm detection platform D. Although the algorithm versions of these platforms are the same, differences in hardware and chipsets can lead to differences in feature vectors across platforms, causing search results to fall short of the expected similarity. For example, when searching for the same face image, due to differences in detection on different platforms, the feature vectors detected on the edge intelligence platform (NVIDIA TX1) and the feature vectors re-detected and extracted on the image search platform (HiSilicon 3559A platform) may not meet the expected similarity requirements. This also requires re-extracting features from captured images, wasting resources and reducing search efficiency. Furthermore, due to the large number of captured images, image search cannot be performed during the feature extraction phase, impacting the user experience.

[0003] There is a problem in related technologies that search results fail to meet the expected similarity requirements, resulting in the need to re-extract features, which wastes resources and reduces search efficiency. Currently, no effective solution has been proposed. Summary of the Invention

[0004] In this embodiment, a multi-platform image search method, device, electronic device and storage medium are provided to solve the problem in the related art that each search result has differences, resulting in the need to re-extract features, causing waste of resources and reducing search efficiency.

[0005] First, in this embodiment, a multi-platform image search method is provided, including:

[0006] Obtain the feature vector of the image to be searched based on the image search platform;

[0007] Performing a similarity comparison between the feature vector of the image to be searched and the feature vectors of all captured images on the edge intelligent platform, selecting images that meet preset conditions from the captured images as preliminary screening images, and obtaining a first detection frame of the preliminary screening images in the edge intelligent platform;

[0008] Based on the image search platform, the image to be searched is searched according to the preliminary screening image and the corresponding first detection frame to obtain an image search result.

[0009] In some embodiments, obtaining a feature vector of a to-be-searched image based on an image search platform includes:

[0010] Based on the image search platform, a CNN neural network model is used to perform target detection on the image to be searched, and a feature vector of the image to be searched is extracted from the target detection result.

[0011] In some embodiments, the image search platform is used to perform target detection on the image to be searched using a CNN neural network model, and the feature vector of the image to be searched is extracted from the target detection result, including:

[0012] Based on the image search platform, using the CNN neural network model to perform target detection on the image to be searched;

[0013] If the target detection result is a face image, extract the feature vector corresponding to the face image;

[0014] If the target detection result is a human body picture, the feature vector corresponding to the human body picture is extracted.

[0015] In some embodiments, the similarity comparison between the feature vector of the image to be searched and the feature vectors of all captured images on the edge intelligent platform is performed, and images that meet preset conditions are selected from the captured images as preliminary screening images, including:

[0016] Using a similarity formula, the feature vector of the image to be searched is compared with the feature vectors of all captured and recorded images of the edge intelligent platform for similarity;

[0017] Based on the similarity comparison results, sorting rules and quantity thresholds, pictures are selected from the captured and recorded pictures as preliminary screening pictures.

[0018] In some embodiments, the image search platform searches the image to be searched based on the preliminary screening image and the corresponding first detection frame to obtain the image search result, including:

[0019] Based on the image search platform, obtaining the feature vector of the primary screening image and the corresponding second detection frame;

[0020] Optimizing the second detection frame and the first detection frame using a detection frame optimization algorithm, locating a target image in the primary screening image, and determining a feature vector corresponding to the target image;

[0021] The image search result is obtained by comparing and sorting the feature vectors of the image to be searched and the feature vectors of the determined target image.

[0022] In some embodiments, obtaining the feature vector of the primary screening image and the corresponding second detection frame based on the image search platform includes:

[0023] Based on the image search platform, the CNN neural network model is used to perform target detection on the preliminary screening image, the feature vector of the preliminary screening image is extracted from the target detection result, and the second detection frame of the preliminary screening image in the image search platform is obtained.

[0024] In some embodiments, the detection frame optimization algorithm is a center distance optimization algorithm or an intersection-over-union optimization algorithm.

[0025] In some embodiments, comparing and sorting the feature vectors of the image to be searched and the determined feature vectors of the target image to obtain the image search results includes:

[0026] Using a similarity formula, a similarity comparison is performed between the feature vector of the image to be searched and the feature vector of the determined target image;

[0027] The target images are sorted according to the similarity comparison results, and the sorted target images are used as image search results.

[0028] In some embodiments, the multi-platform image search method provided by this embodiment further includes:

[0029] The image search results are displayed according to a preset display strategy.

[0030] In a second aspect, a multi-platform image search device is provided in this embodiment, including an acquisition module, a preliminary screening module, and an image search module;

[0031] The acquisition module is used to obtain the feature vector of the image to be searched based on the image search platform;

[0032] The primary screening module is configured to perform a similarity comparison between the feature vectors of the image to be searched and the feature vectors of all captured images on the edge intelligent platform, select images that meet preset conditions from the captured images as primary screening images, and obtain a first detection frame of the primary screening image on the edge intelligent platform;

[0033] The image search module is used to search the image to be searched based on the image search platform and according to the preliminary screening image and the corresponding first detection frame to obtain the image search result.

[0034] In a third aspect, an electronic device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the multi-platform image search method described in the first aspect is implemented.

[0035] In a fourth aspect, a storage medium is provided in this embodiment, on which a computer program is stored. When the program is executed by a processor, the multi-platform image search method described in the first aspect is implemented.

[0036] Compared with the related art, the multi-platform image search method, device, electronic device and storage medium provided in this embodiment obtain the feature vector of the image to be searched based on the image search platform; perform a similarity comparison between the feature vector of the image to be searched and the feature vectors of all the captured and recorded images of the edge intelligent platform, and screen out images that meet preset conditions from the captured and recorded images as preliminary screening images, and obtain the first detection frame of the preliminary screening images in the edge intelligent platform; based on the image search platform, the image to be searched is searched according to the preliminary screening images and the corresponding first detection frame to obtain the search results, which solves the problem that the search results do not meet the expected similarity requirements, resulting in the need to re-extract features, causing resource waste and reducing search efficiency; preliminary screening images that meet the conditions are initially found through preliminary screening, avoiding the re-extraction of a large number of irrelevant data algorithm features, reducing resource waste and improving search efficiency.

[0037] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0039] Figure 1 This is a hardware structure block diagram of a terminal device for a multi-platform image search method provided by an embodiment of the present application;

[0040] Figure 2 This is a flowchart of a multi-platform image search method provided by an embodiment of the present application;

[0041] Figure 3 yes Figure 2 Flowchart of step S230;

[0042] Figure 4 1 is a schematic diagram of an intersection-over-union optimization algorithm provided in one embodiment of the present application;

[0043] Figure 5 This is a structural block diagram of a multi-platform image search device provided in one embodiment of the present application.

[0044] In the figure: 210, acquisition module; 220, initial screening module; 230, image search module. DETAILED DESCRIPTION

[0045] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0046] Unless otherwise defined, the technical terms or scientific terms involved in this application should have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "an", "a", "the", "these" and the like in this application do not indicate quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "plurality" involved in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. Generally, the character " / " indicates that the related objects are in an "or" relationship. The terms "first," "second," "third," etc. used in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0047] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG is a block diagram of the hardware structure of the terminal of the multi-platform image search method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1Only one is shown) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0048] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the multi-platform image search method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above-mentioned method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0049] The transmission device 106 is used to receive or send data via a network. The network may include a wireless network provided by the terminal's telecommunications provider. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0050] In this embodiment, a multi-platform image search method is provided. Figure 2 is a flowchart of the multi-platform image search method of this embodiment. Figure 2 As shown, the process includes the following steps:

[0051] Step S210: obtaining a feature vector of the image to be searched based on the image search platform;

[0052] Step S220: Compare the feature vector of the image to be searched with the feature vectors of all captured images on the edge intelligent platform for similarity. Select images that meet preset conditions from the captured images as preliminary screening images, and obtain a first detection frame for the preliminary screening images on the edge intelligent platform.

[0053] Step S230: Based on the image search platform, the image to be searched is searched according to the preliminary screening image and the corresponding first detection frame to obtain the image search result.

[0054] It's important to note that with the advent of the Internet of Everything (IoE) era, the amount of data generated by network edge devices is rapidly increasing, leading to higher data transmission bandwidth requirements. This also places higher demands on the real-time nature of data processing, which traditional cloud computing models are no longer able to effectively address. Consequently, edge intelligence has emerged. The fundamental concept of edge intelligence is to run computing tasks on computing resources close to the data source. This effectively reduces computing system latency, reduces data transmission bandwidth, alleviates pressure on cloud computing centers, improves availability, and protects data security and privacy. Thanks to these advantages, edge intelligence is also present in multi-platform image search. This involves placing some object detection and feature extraction processing on an edge intelligence algorithm detection platform (hereinafter referred to as the platform), and switching between multiple algorithm detection platforms.

[0055] The image search platform refers to the platform currently used by the user to search for images, which can be the HiSilicon 3559A platform, NVIDIA TX1 platform, or Cambricon platform. The image search platform can utilize a neural network model or other fully trained model to obtain feature vectors for the image being searched. During the feature vector acquisition process, corresponding detection frames are generated. Generally speaking, although the algorithm versions used in these platforms are consistent, differences in hardware and chipsets can lead to differences in the detection frames used to calculate feature vectors on each platform, resulting in differences in feature vectors and the need for feature re-extraction. Feature re-extraction refers to the need to re-detect on another edge intelligence platform because the comparison results between the two platforms do not meet the expected similarity requirements. The edge intelligence platform refers to an algorithm detection platform with edge intelligence, which can also be the HiSilicon 3559A platform, NVIDIA TX1 platform, or Cambricon platform. The edge intelligence platform uses a preliminary screening process to select images that meet preset criteria from the captured images recorded by the edge intelligence platform as preliminary screening images, and obtains the first detection frame for the preliminary screening images on the edge intelligence platform. This avoids the re-extraction of a large amount of irrelevant data algorithm features, reduces resource waste, and improves search efficiency. Finally, based on the image search platform, the image to be searched is searched according to the preliminary screening image and the corresponding first detection frame to obtain the search results. Based on the image search platform, the image search is performed according to the preliminary screening image and the corresponding first detection frame. This can ensure that the target image in the image to be searched is locked, so that the search results meet the expected similarity requirements. In other words, there is no need to go to another edge intelligent platform for re-testing. This solves the problem of search results not meeting the expected similarity requirements, resulting in the need to re-extract features, resulting in resource waste and reduced search efficiency.

[0056] The following is a detailed description of the above steps:

[0057] In some embodiments, step S210 includes the following steps:

[0058] Based on the image search platform, the CNN neural network model is used to perform target detection on the image to be searched, and the feature vector of the image to be searched is extracted from the target detection results.

[0059] The CNN neural network model can be used to quickly complete target detection and feature extraction of the image to be searched. In other embodiments, other well-trained models can also be used, which are not limited here.

[0060] Specifically, based on the image search platform, a CNN neural network model is used to perform target detection on the image to be searched, and a feature vector of the image to be searched is extracted from the target detection result, including the following steps:

[0061] Based on the image search platform, the CNN neural network model is used to detect objects in the searched images;

[0062] If the target detection result is a face image, extract the feature vector corresponding to the face image;

[0063] If the target detection result is a human body picture, the feature vector corresponding to the human body picture is extracted.

[0064] Since there are many types of images to be searched, in order to reduce the amount of computation and improve efficiency, the target detection result is first judged. If the target detection result is a face image, the feature vector corresponding to the face image is extracted; if the target detection result is a face image, the feature vector corresponding to the face image is extracted. In other embodiments, the target detection result may also be a vehicle image, a toy image, a plant image, etc., and the corresponding feature vector is extracted.

[0065] In some embodiments, step S220 of performing a similarity comparison between the feature vector of the image to be searched and the feature vectors of all captured images on the edge intelligence platform, and screening out images that meet preset conditions from the captured images as preliminary screening images, includes the following steps:

[0066] Step S221: Using a similarity formula, perform a similarity comparison between the feature vector of the image to be searched and the feature vectors of all captured and recorded images on the edge intelligence platform.

[0067] Step S222: Filter out pictures from the captured pictures as preliminary screening pictures based on the similarity comparison results, sorting rules and quantity thresholds.

[0068] Specifically, the similarity formula is the Euclidean distance calculation formula: Where a i represents the feature vector of the i-th image to be searched; b i represents the feature vector of the i-th captured image; d represents the distance between the feature vector A of the image to be searched and the feature vector B of the captured image. Using the Euclidean distance formula above, we can determine the distance between the feature vector of each image to be searched and the feature vector of the captured image, thereby obtaining a similarity comparison result.

[0069] Among them, the preset conditions are sorting rules and quantity thresholds, wherein the sorting rules can be sequential or reverse order; if it is sequential, then the similarity comparison results of the quantity threshold N are screened from front to back, and then the corresponding pictures are obtained as preliminary screening pictures. If it is reverse order, then the similarity comparison results of the quantity threshold N are screened from back to front, and then the corresponding pictures are obtained as preliminary screening pictures. That is to say, N pictures with the highest similarity are screened as preliminary screening pictures, and N can be 1000. The N preliminary screening pictures and the corresponding first detection frames are put into the cache. Since the same picture has only a small difference when detected on different platforms under the same algorithm version, all the preliminary screening pictures that pass the preliminary screening meet the basic requirements of similarity for image search. In other embodiments, the similarity comparison results that meet the preset similarity threshold can also be directly screened and then sorted, and there is no restriction on this.

[0070] If all the hardware and software of the edge intelligence platform and the image search platform are the same, then the pre-screened image is the target image. It can also be considered that in step S230, the feature vector of the image to be searched is compared with the feature vector of the pre-screened image for similarity to perform the image search.

[0071] If the hardware and software of the edge intelligence platform and the image search platform used are different, then in some embodiments, such as Figure 3 As shown, step S230 includes the following steps:

[0072] Step S231: Obtain the feature vector of the primary screening image and the corresponding second detection frame based on the image search platform;

[0073] Specifically, based on the image search platform, a CNN neural network model is used to perform target detection on the initial screening images. Feature vectors for the initial screening images are extracted from the target detection results, and the secondary detection boxes for the initial screening images on the image search platform are obtained. In other words, target detection and feature extraction are re-performed on the initial screening images on the image search platform to obtain the feature vectors and corresponding secondary detection boxes for each initial screening image. To eliminate the impact of the algorithm on the results, the target detection and feature extraction algorithms are the same as those used in the edge intelligence platform. For example, if the edge intelligence platform uses version 1.0 of the CNN neural network model, the image search platform will also use version 1.0 of the CNN neural network model. Since target detection is performed on each initial screening image, multiple targets may be present in each image. When platforms differ, the detected targets only have slightly different detection boxes. That is, for the same initial screening image, the second detection box on the image search platform differs from the first detection box on the edge intelligence platform. In this embodiment, by remodeling (target detection and feature extraction) the initial screening images that meet the preset conditions, only the initial screening images that meet the preset conditions are modeled, avoiding the need to remodel all captured and recorded images, and increasing the similarity of image search. Since only a small amount of image data needs to be modeled, it does not affect other services such as real-time detection after switching platforms, and solves the differences in hardware inconsistencies between cross-image search platforms and edge intelligence platforms.

[0074] Step S232: Optimize the second detection frame and the first detection frame using a detection frame optimization algorithm to locate the target image in the initial screening image to determine a feature vector corresponding to the target image;

[0075] Specifically, in order to solve the differences in the above-mentioned detection frames, the second detection frame and the first detection frame are optimized using a detection frame optimization algorithm to locate the target image in the initial screening image to determine the corresponding feature vector of the target image, thereby solving the problem of differences in detection frames on different platforms. Among them, the detection frame optimization algorithm can be a center distance optimization algorithm or an intersection-in-union optimization algorithm. The position of the target image detected by the edge intelligent platform in the image search platform is determined by the center distance optimization algorithm or the intersection-in-union optimization algorithm, which effectively eliminates the situation where there are many interference items when there are multiple targets in the image.

[0076] In one embodiment, Figure 4 The figure shows a schematic diagram of the intersection-in-union optimization algorithm. The formula for calculating the intersection-in-union ratio is Here, A∩B represents the intersection of the two detection frame areas, and A∪B represents the union of all areas within the two detection frame areas. The larger the IOU (Intersection-over-Union) ratio (IOU), the more overlap there is between the two detection frames. Therefore, the detection frame with the largest IOU is selected to locate the target image in the initial screening image. This method eliminates unnecessary captured objects from the captured images.

[0077] Step S233, comparing and sorting the feature vectors of the image to be searched and the feature vectors of the determined target image to obtain the image search results;

[0078] Specifically, a similarity formula is used to compare the feature vector of the image to be searched with the feature vector of the determined target image. The target images are sorted according to the similarity comparison results, and the sorted target images are used as the image search results.

[0079] In this embodiment, the similarity formula is the Euclidean distance calculation formula: Where a i represents the feature vector of the i-th image to be searched; c i represents the feature vector of the i-th target image; d′ represents the distance between the feature vector A of the image to be searched and the feature vector C of the target image. The above Euclidean distance calculation formula can be used to determine the distance between the feature vector of each pair of search images and the feature vector of the target image, thereby obtaining the similarity comparison result.

[0080] The feature vector of the searched image is compared with the feature vector of the determined target image to obtain a new similarity ranking. The ranked target image is used as the search result. Since both the searched image and the target image undergo object detection and feature extraction on the image search platform, there are no platform differences. The new similarity ranking is the final searched similarity. If a re-extracted image is the same as the searched image, the similarity is close to 100%. Similarity ranking refers to sorting in order.

[0081] Step S234: Display the image search results according to a preset display strategy.

[0082] Specifically, the display strategy is to select a preset number of search results and sort them by similarity from high to low for display. The display number is less than or equal to N.

[0083] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0084] In this embodiment, a multi-platform image search device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the details that have been described will not be repeated. The terms "module", "unit", "sub-unit", etc. used below can implement a combination of software and / or hardware for predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, implementation by hardware, or a combination of software and hardware, is also possible and conceivable.

[0085] Figure 5 This is a structural block diagram of the multi-platform image search device of this embodiment. Figure 5 As shown, the device includes: an acquisition module 210, a primary screening module 220 and an image search module 230;

[0086] An acquisition module 210 is used to acquire a feature vector of a to-be-searched image based on an image search platform;

[0087] The initial screening module 220 is used to compare the feature vectors of the searched image with the feature vectors of all captured images on the edge intelligent platform. It selects images that meet preset conditions from the captured images as initial screening images and obtains the first detection frame of the initial screening images on the edge intelligent platform.

[0088] The image search module 230 is used to search the image to be searched based on the image search platform according to the preliminary screening image and the corresponding first detection frame to obtain the image search result.

[0089] Through the above device, the problem of search results failing to meet the expected similarity requirements, resulting in the need to re-examine features, causing waste of resources and reducing search efficiency, is solved; by initially finding the preliminary screening images that meet the conditions through preliminary screening, a large number of irrelevant data algorithm features are avoided from being re-examined, thus reducing resource waste and improving search efficiency.

[0090] In some embodiments, the acquisition module 210 is further configured to perform target detection on the image to be searched using a CNN neural network model based on the image search platform, and extract a feature vector of the image to be searched from the target detection result.

[0091] In some embodiments, the acquisition module 210 includes a target detection unit, a first extraction unit and a second extraction unit; the target detection unit is used to perform target detection on the search image based on the image search platform using a CNN neural network model; the first extraction unit is used to extract the feature vector corresponding to the face image if the target detection result is a face image; the second extraction unit is used to extract the feature vector corresponding to the body image if the target detection result is a body image.

[0092] In some embodiments, the initial screening module 220 is further configured to use a similarity formula to compare the feature vector of the image to be searched with the feature vectors of all captured and recorded images of the edge intelligence platform for similarity;

[0093] Based on the similarity comparison results, sorting rules and quantity thresholds, pictures are selected from the captured pictures as preliminary screening pictures.

[0094] In some of the embodiments, the image search module 230 includes an acquisition unit, a positioning unit, and an image search unit;

[0095] An acquisition unit, configured to acquire a feature vector of the primary screening image and a corresponding second detection frame based on the image search platform;

[0096] a positioning unit, configured to optimize the second detection frame and the first detection frame using a detection frame optimization algorithm, and locate the target image in the primary screening image to determine a feature vector corresponding to the target image;

[0097] The image search unit is used to compare and sort the feature vectors of the image to be searched and the feature vectors of the determined target image to obtain the image search results.

[0098] In some of the embodiments, the acquisition unit is also used to perform target detection on the preliminary screening image based on the image search platform using a CNN neural network model, extract the feature vector of the preliminary screening image from the target detection result, and obtain a second detection frame of the preliminary screening image in the image search platform.

[0099] In some embodiments, the detection box optimization algorithm is a center distance optimization algorithm or an intersection-over-union optimization algorithm.

[0100] In some embodiments, the image search unit is further configured to compare the feature vector of the image to be searched with the feature vector of the determined target image using a similarity formula;

[0101] Sort the target images according to the similarity comparison results, and use the sorted target images as the search results.

[0102] In some of the embodiments, the image search module 230 further includes a display unit;

[0103] The display unit is used to display the image search results according to a preset display strategy.

[0104] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0105] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0106] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0107] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:

[0108] S1, obtain the feature vector of the image to be searched based on the image search platform;

[0109] S2: Compare the feature vector of the image to be searched with the feature vectors of all captured images on the edge intelligence platform for similarity. Select images that meet preset conditions from the captured images as preliminary screening images, and obtain the first detection frame of the preliminary screening images on the edge intelligence platform.

[0110] S3, based on the image search platform, searches the image to be searched according to the preliminary screening image and the corresponding first detection frame to obtain the search result.

[0111] It should be noted that, for specific examples in this embodiment, reference may be made to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0112] In addition, in conjunction with the multi-platform image search method provided in the above embodiments, a storage medium may also be provided in this embodiment to implement the multi-platform image search method. The storage medium stores a computer program; when the computer program is executed by a processor, any of the multi-platform image search methods in the above embodiments is implemented.

[0113] It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0114] Obviously, the accompanying drawings are merely examples or embodiments of the present application. A person skilled in the art can also apply the present application to other similar situations based on these drawings without inventive effort. Furthermore, it is understandable that, although the work involved in this development process may be complex and lengthy, certain design, manufacturing, or production changes based on the technical content disclosed in this application are merely routine technical means for a person skilled in the art and should not be considered to constitute a deficiency in the disclosure of the present application.

[0115] The term "embodiment" as used in this application refers to specific features, structures, or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily mean that the embodiment is the same, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is understood, either explicitly or implicitly, by those skilled in the art that the embodiments described in this application can be combined with other embodiments when there is no conflict.

[0116] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A multi-platform image search method, characterized in that: include: Obtain the feature vector of the image to be searched based on the image search platform; Performing a similarity comparison between the feature vector of the image to be searched and the feature vectors of all captured images on the edge intelligent platform, selecting images that meet preset conditions from the captured images as preliminary screening images, and obtaining a first detection frame of the preliminary screening images in the edge intelligent platform; Based on the image search platform, searching the image to be searched according to the preliminary screening image and the corresponding first detection frame to obtain an image search result; The image search platform searches the image to be searched based on the preliminary screening image and the corresponding first detection frame to obtain the image search result, including: On the image search platform, re-performing target detection and feature extraction on the primary screening image to obtain a feature vector of the primary screening image and a corresponding second detection frame; Optimizing the second detection frame and the first detection frame using a detection frame optimization algorithm, locating a target image in the primary screening image, and determining a feature vector corresponding to the target image; The image search result is obtained by comparing and sorting the feature vectors of the image to be searched and the feature vectors of the determined target image.

2. The multi-platform image search method according to claim 1, characterized in that: The step of obtaining a feature vector of a picture to be searched based on the picture search platform includes: Based on the image search platform, a CNN neural network model is used to perform target detection on the image to be searched, and a feature vector of the image to be searched is extracted from the target detection result.

3. The multi-platform image search method according to claim 2, characterized in that: The method of performing target detection on the image to be searched using a CNN neural network model based on the image search platform and extracting a feature vector of the image to be searched from the target detection result includes: Based on the image search platform, using the CNN neural network model to perform target detection on the image to be searched; If the target detection result is a face image, extract the feature vector corresponding to the face image; If the target detection result is a human body picture, the feature vector corresponding to the human body picture is extracted.

4. The multi-platform image search method according to claim 1, characterized in that: The similarity comparison between the feature vector of the image to be searched and the feature vectors of all captured and recorded images on the edge intelligent platform is performed, and images that meet preset conditions are selected from the captured and recorded images as preliminary screening images, including: Using a similarity formula, the feature vector of the image to be searched is compared with the feature vectors of all captured and recorded images of the edge intelligent platform for similarity; Based on the similarity comparison results, sorting rules and quantity thresholds, pictures are selected from the captured and recorded pictures as preliminary screening pictures.

5. The multi-platform image search method according to claim 1, characterized in that: The re-performing target detection and feature extraction on the primary screening image on the image search platform to obtain a feature vector of the primary screening image and a corresponding second detection frame includes: On the image search platform, the CNN neural network model is used to re-perform target detection on the preliminary screening image, the feature vector of the preliminary screening image is extracted from the target detection result, and the second detection frame of the preliminary screening image in the image search platform is obtained.

6. The multi-platform image search method according to claim 1, characterized in that: The detection frame optimization algorithm is a center distance optimization algorithm or an intersection-over-union optimization algorithm.

7. The multi-platform image search method according to claim 1, characterized in that: The step of comparing and sorting the feature vectors of the image to be searched and the determined feature vectors of the target image to obtain the image search result includes: Using a similarity formula, a similarity comparison is performed between the feature vector of the image to be searched and the feature vector of the determined target image; The target images are sorted according to the similarity comparison results, and the sorted target images are used as image search results.

8. The multi-platform image search method according to claim 7, characterized in that: Also includes: The image search results are displayed according to a preset display strategy.

9. A multi-platform image search device, characterized in that: Including acquisition module, initial screening module and image search module; The acquisition module is used to obtain the feature vector of the image to be searched based on the image search platform; The primary screening module is configured to perform a similarity comparison between the feature vectors of the image to be searched and the feature vectors of all captured images on the edge intelligent platform, select images that meet preset conditions from the captured images as primary screening images, and obtain a first detection frame of the primary screening image on the edge intelligent platform; The image search module is configured to search the image to be searched based on the image search platform and the primary screening image and the corresponding first detection frame to obtain an image search result; The image search module is further configured to re-perform target detection and feature extraction on the primary screening image on the image search platform to obtain a feature vector of the primary screening image and a corresponding second detection frame; Optimizing the second detection frame and the first detection frame using a detection frame optimization algorithm, locating a target image in the primary screening image, and determining a feature vector corresponding to the target image; The image search result is obtained by comparing and sorting the feature vectors of the image to be searched and the feature vectors of the determined target image.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the multi-platform image search method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the multi-platform image search method according to any one of claims 1 to 8 are implemented.

Images