Picture storage method, picture retrieval method and related devices

Abstract

This application discloses an image storage method, an image retrieval method, and related apparatus, relating to the field of image processing. The image storage method includes: acquiring an image set; clustering the images in the image set into buckets according to image features to obtain a bucket set; performing video encoding on the images in each bucket of the bucket set to obtain a video sequence for each bucket in the bucket set; restoring and decoding the frames in the video sequence to obtain decoded frames; if the image feature similarity between the decoded frame and the original image is less than a first threshold, then after adjusting the video encoding parameters, returning to video encoding the images in each bucket of the bucket set, until the loop termination condition is met, so that the target video sequence of each bucket in the bucket set is obtained and stored at the end of the loop. This application considers image query needs when storing the image set. While encoding the image set into video after clustering, it achieves high-fidelity compressed storage through frame decoding and similarity comparison, reducing storage costs and management burden.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image storage method, an image retrieval method, and related apparatus. Background Technology

[0002] With the increasing popularity of large-scale visual retrieval and retrieval-augmented generation (RAG) applications, systems often need to store images in order to retrieve the most relevant images with low latency during queries.

[0003] Currently, each image is typically stored individually as text or a Blob (Binary Large Object). However, as the size of the image database expands, storing massive amounts of images one by one incurs significant storage costs and management burdens. Summary of the Invention

[0004] In view of the above problems, this application provides an image storage method, an image retrieval method, and related apparatus to reduce storage costs and management burden by storing massive amounts of images in a video format. The specific solution is as follows:

[0005] The first aspect of this application provides an image storage method, including:

[0006] Obtain an image set, and cluster the images in the image set into buckets according to image features to obtain a bucket set;

[0007] Video encoding is performed on the images in each bucket of the bucket set to obtain a video sequence for each bucket in the bucket set;

[0008] The frames in the video sequence are restored and decoded to obtain decoded frames. If the image feature similarity between the decoded frame and the original image is less than a first threshold, the video encoding parameters are adjusted and the process of video encoding the images in each bucket of the bucket set is returned until the loop ends. The original image is the image in the image set that corresponds to the decoded frame.

[0009] At the end of the loop, the target video sequence of each bucket in the bucket set is obtained and stored.

[0010] In one possible implementation, the step of clustering and binning the images within the image set according to image features to obtain a bucket set includes:

[0011] Iterate through the images in the image set and use the currently iterated image as the target image;

[0012] Determine the first distance between the image features of the target image and the cluster centers of the candidate buckets;

[0013] If there is a first candidate bucket in the candidate bucket whose first distance is less than or equal to a preset distance threshold and whose current bucket capacity is less than a preset bucket capacity threshold, then the target image is added to the first candidate bucket; otherwise, the target image is added to a new bucket.

[0014] Update the cluster centers of the buckets to which the target image is added based on the image features of the target image, and continue traversing until the traversal is complete.

[0015] One possible implementation also includes:

[0016] If a second candidate bucket is already full before all images in the image set have been traversed, then the images in the second candidate bucket are clustered according to image features to obtain a first clustering result. Based on the first clustering result, the second candidate bucket is split into multiple candidate buckets.

[0017] And / or, after updating the cluster centers of the first candidate buckets according to the image features of the target image, the average feature similarity of the image features of the images in the first candidate buckets relative to the updated cluster centers is calculated. If the average feature similarity is lower than a preset second threshold, the images in the first candidate buckets are clustered according to the image features to obtain a second clustering result. The first candidate bucket is then split into multiple candidate buckets according to the second clustering result.

[0018] In one possible implementation, calculating the average feature similarity of the image features of the images within the first candidate bucket relative to the updated cluster centers includes:

[0019] Calculate the similarity between the image features of each image in the first candidate bucket and the updated cluster center, and use it as the similarity for each image.

[0020] The average feature similarity is obtained by taking the average of the similarities of all images in the first candidate bucket.

[0021] In one possible implementation, the step of recovering and decoding the frames in the video sequence to obtain decoded frames includes:

[0022] A preset number of frames are sampled from the video sequence, and each sampled frame is restored and decoded to obtain the decoded frame corresponding to each frame; thus obtaining the preset number of decoded frames.

[0023] The process of determining the image feature similarity between the decoded frame and the original image includes:

[0024] Determine the similarity between the image features of each decoded frame and the corresponding original image to obtain a preset number of similarity scores;

[0025] The average of the similarities of the preset number of frames is taken as the image feature similarity between the decoded frame and the original image.

[0026] A second aspect of this application provides an image retrieval method, including:

[0027] Obtain the bucket summary and vector library for each bucket in the bucket set. The bucket summary represents the semantic distribution of the images within the bucket, and the vector library includes the image features and identity identifiers of each image within the bucket.

[0028] Calculate a first similarity between the query data and the bucket summary of each bucket in the bucket set, so as to determine the similar buckets of the query data from the bucket set based on the first similarity;

[0029] The query data is compared with the features of each image in the vector library of the similarity bucket to calculate a second similarity, so as to determine the identity of the similar image of the query data based on the second similarity, which is used as the target identity.

[0030] Obtain the target video sequence of the similarity bucket, decode the target video sequence of the similarity bucket to obtain the frame data of the similar image indicated by the target identity, and use it as the retrieval result of the query data.

[0031] In one possible implementation, the process of generating a bucket summary for any of the buckets includes:

[0032] Multiple representative images are selected from the images in the bucket according to a preset selection strategy. The selection strategy includes one or more of the following strategies: sampling based on the time the images are added to the bucket, uniform sampling, and a representative selection strategy based on the distance to the cluster center.

[0033] The image features of the multiple representative images are aggregated to obtain the bucket summary of the bucket.

[0034] In one possible implementation, obtaining the target video sequence of the similarity bucket, decoding the target video sequence of the similarity bucket to obtain the frame data of the similar image indicated by the target identity, includes:

[0035] Obtain metadata information containing the target identity identifier, the metadata information also includes the file identifier of the target video sequence of the similar bucket, and the first frame number of the decoding keyframe corresponding to the similar image and / or the second frame number of the similar image;

[0036] The target video sequence indicated by the file identifier is obtained as the target video sequence of the similarity bucket;

[0037] Based on the frame sequence number in the metadata information, determine the index table of the decoding keyframe, and determine the frame location data of the decoding keyframe in the target video sequence of the similar bucket from the index table;

[0038] Based on the frame positioning data, the target video sequence of the similar bucket is addressed to obtain the frame data of the decoded keyframe;

[0039] The frame data of the keyframes is decoded frame by frame until the frame data of the similar image is obtained.

[0040] In one possible implementation, if the metadata information includes the second frame sequence number, then determining the index table of the decoded keyframes based on the frame sequence number in the metadata information includes:

[0041] The index table containing the largest frame number not greater than the second frame number is determined from the pre-configured keyframe index table set and used as the index table for the decoded keyframe. The keyframe index table set includes the index tables of all decoded keyframes within the target video sequence of the similar bucket.

[0042] A third aspect of this application provides a computer program product, including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement the image storage method of the first aspect or any implementation thereof, or to implement the image retrieval method of the second aspect or any implementation thereof.

[0043] A fourth aspect of this application provides an electronic device, including at least one processor and a memory connected to the processor, wherein:

[0044] The memory is used to store computer programs;

[0045] The processor is used to execute the computer program so that the electronic device can implement the image storage method of the first aspect or any implementation thereof, or implement the image retrieval method of the second aspect or any implementation thereof.

[0046] The fifth aspect of this application provides a computer storage medium carrying one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device is able to implement the image storage method of the first aspect or any implementation thereof, or the image retrieval method of the second aspect or any implementation thereof.

[0047] By employing the above technical solution, the image storage method provided in this application obtains an image set. To avoid the images in the image set occupying a large amount of storage space, causing insufficient storage space, and to reduce the management burden, this application can cluster and bucket the images in the image set according to image features to obtain a bucket set. Then, video encoding is performed on the images in each bucket of the bucket set to obtain a video sequence for each bucket in the bucket set. Through clustering and bucketing, more similar images can be added to the same bucket, thereby making fuller use of the inter-frame redundancy between similar images for sufficient compression, greatly saving storage space and reducing storage costs. Furthermore, the method of encoding into video only requires managing a small number of video sequences, reducing the management burden.

[0048] Furthermore, considering the potential for future image retrieval needs after image set storage, to avoid image distortion caused by video encoding, which could prevent the search for authentic images, this application can recover and decode frames in the video sequence to obtain decoded frames. If the image feature similarity between the decoded frame and the corresponding original image is less than a first threshold, the video encoding parameters are adjusted, and the process of encoding the images in each bucket of the bucket set is repeated until the loop termination condition is met. At the end of the loop, the target video sequence for each bucket in the bucket set is obtained and stored. Therefore, this application considers image retrieval needs when storing the image set. While encoding the image set into video after clustering, it achieves high-fidelity compression of the image set through frame decoding and similarity comparison, improving the fidelity of subsequent image retrieval. Attached Figure Description

[0049] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0050] Figure 1 A schematic diagram of a system architecture provided for this application;

[0051] Figure 2 A flowchart illustrating an image storage method provided in this application;

[0052] Figure 3 A flowchart illustrating an image retrieval method provided in this application;

[0053] Figure 4 A schematic diagram of the structure of an image storage device provided in this application;

[0054] Figure 5 A schematic diagram of the structure of an image retrieval device provided in this application;

[0055] Figure 6 This is a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0056] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0057] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0058] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0059] Optionally, the image storage method and image retrieval method provided in this application can be applied to, for example... Figure 1 The system architecture shown includes a terminal 100 and a server 200. The server 200 may include one or more servers (…). Figure 1 (This example uses a server as an illustration).

[0060] Either terminal 100 or server 200 can be used independently to execute the image storage method and image retrieval method provided in the embodiments of this application. Alternatively, terminal 100 and server 200 can also be used collaboratively to execute the image storage method and image retrieval method provided in the embodiments of this application.

[0061] The following description Figure 1 The product form of the mid-terminal 100;

[0062] The terminal 100 in this application embodiment can be a mobile phone, tablet computer, wearable device, vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc., and this application embodiment does not impose any restrictions on it.

[0063] To enable those skilled in the art to better understand this application, the image storage method of the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0064] The image storage method described in this application is applicable to scenarios requiring image storage, especially those requiring the storage of massive amounts of images under limited storage space. Examples include access control systems storing facial images for facial recognition, social media platforms storing user-uploaded photos for users to view when needed, and RAG (Retrieval-Augmented Generation) image retrieval scenarios.

[0065] It should be noted that the above scenarios are merely examples and are not intended to limit the application scenarios of this application.

[0066] Reference Figure 2 , Figure 2 This is a flowchart illustrating an image storage method provided in an embodiment of this application, as follows: Figure 2 As shown, the image storage method may include:

[0067] Step S101: Obtain the image set, and cluster the images in the image set into buckets according to image features to obtain a bucket set.

[0068] Here, an image set is a collection of multiple images.

[0069] In order to store the image set with the smallest possible storage space, this embodiment can cluster the images in the image set into buckets according to image features to obtain a bucket set. The bucket set consists of multiple buckets, and each bucket includes a portion of the images in the image set.

[0070] Optionally, the image features can be image embedding features, i.e., image embedding features; they can also be image features encoded by the encoder, or other image features. This application does not impose any specific limitations.

[0071] Optionally, image embedding features can be obtained by extracting embedding features from images using at least one of Residual Network (ResNet), Swin Transformer, and Contrastive Language-Image Pre-training (CLIP). Of course, image embedding features can also be obtained through other network architectures, and this application does not impose any specific limitations.

[0072] This embodiment uses clustering and binning to group similar images into the same bin, thereby increasing inter-frame redundancy and facilitating video compression as described below.

[0073] Step S102: Perform video encoding on the images in each bucket of the bucket set to obtain the video sequence of each bucket in the bucket set.

[0074] In this embodiment, video encoding parameters can be preset to encode the images in each bucket into a video sequence.

[0075] Optionally, initially, the images in each bucket of the bucket set are video encoded using the same video encoding parameters. Of course, video encoding parameters can also be preset for each bucket, so that the images in each bucket are video encoded using different video encoding parameters.

[0076] Optional video encoding parameters may include, but are not limited to, the following: frame rate (e.g., 25 FPS), number of Group of Pictures (GOPs), constant rate factor (CRF), pixel format, and encoder type.

[0077] The pixel format can include yuv420p and yuv444p, and the encoder type can include H.264, H.265 and AV1.

[0078] It should be noted that the video encoding parameters mentioned above are just examples. In addition, there may be other parameters, such as the encoding strategy parameter for optimizing static image sequences, such as -tune stillimage.

[0079] It should also be noted that, optionally, the above video encoding parameters can be pre-processed with automated A / B verification before video encoding to obtain video encoding parameters that can encode better video quality.

[0080] Step S103: Recover and decode the frames in the video sequence to obtain decoded frames. If the image feature similarity between the decoded frame and the original image is less than the first threshold, then after adjusting the video encoding parameters, return to video encoding the images in each bucket of the bucket set until the loop termination condition is met.

[0081] The original images above are the images in the image set that correspond to the decoded frames.

[0082] Considering that in some scenarios, images are stored for later retrieval, and to avoid image distortion due to improper video encoding parameter configuration when using preset video encoding parameters, which could affect subsequent retrieval, this embodiment performs robustness checks on the video sequences of each bucket mentioned above.

[0083] Specifically, frames in a video sequence can be recovered and decoded to obtain decoded frames, and image features of the decoded frames, such as image embedding features, can be calculated. Here, the frames to be recovered and decoded in the video sequence can include all frames, some frames, or a single frame, which can be determined according to actual needs, and this application does not impose specific limitations.

[0084] To determine the recovery effect of the decoded frame, this embodiment can also determine the original image corresponding to the decoded frame from the image set and determine the image features of the original image. Then, the image features of the original image and the image features of the decoded frame are calculated to obtain the image feature similarity between the decoded frame and the original image. If the image feature similarity is less than a preset first threshold, it indicates that the robustness of the current video sequence is insufficient. In this case, the video encoding parameters need to be adjusted and the video encoding needs to be re-encoded. This process continues until the preset loop termination condition is reached.

[0085] As mentioned earlier, the frames to be recovered and decoded in a video sequence can include multiple frames, and therefore there can be multiple decoded frames. Based on this, the image feature similarity can be obtained by fusing the similarity between the image features of multiple decoded frames and the image features of the corresponding original images.

[0086] For example, in one possible implementation, a preset number of frames (i.e., frames to be recovered and decoded) can be sampled from the video sequence, and each sampled frame can be recovered and decoded to obtain the corresponding decoded frame, so as to obtain the preset number of decoded frames; then, the similarity between the image features of each decoded frame and the corresponding original image can be determined to obtain the preset number of similarities, and the average of the preset number of similarities can be taken as the image feature similarity between the decoded frame and the original image.

[0087] Optionally, a preset number of frames can be sampled from the video sequence using random sampling or regular sampling. Optionally, the preset number can be preset based on experience, or determined using the following method:

[0088] Formula (1);

[0089] in, Indicates the preset number, Indicates rounding up. This indicates the number of buckets in the bucket set.

[0090] Optionally, the similarity between the image features of each decoded frame and the corresponding original image can be cosine similarity. The image feature similarity can then be calculated as follows:

[0091] Formula (2);

[0092] in, Indicates the similarity of image features. This represents the image features of the k-th decoded frame. This represents the image features of the original image corresponding to the k-th decoded frame. This indicates the calculation of cosine similarity.

[0093] As mentioned earlier, if image feature similarity If the value is less than the first threshold, the video encoding parameters need to be adjusted. Optionally, one possible way to "adjust the video encoding parameters" is to reduce the CRF by a predetermined step size to improve the video encoding quality.

[0094] For example, adjust the CRF as follows:

[0095] Formula (3);

[0096] in, This indicates the adjusted CRF. This indicates the CRF before adjustment. Indicates the predetermined step size. This indicates the minimum allowed CRF value. Indicates taking and The maximum value in.

[0097] Optionally, the loop termination condition may include one or more of the following conditions: image feature similarity is greater than or equal to a first threshold, or the maximum number of recoding iterations is reached. (like (3).

[0098] Of course, the loop termination condition can be other than that, and this application does not impose any specific restrictions.

[0099] Step S104: Obtain the target video sequence for each bucket in the bucket set at the end of the loop, and store the target video sequence.

[0100] After the robustness detection and recoding described above, the target video sequence of each bucket in the bucket set can be obtained at the end of the loop. Then, the target video sequences of each bucket are stored. Since the target video sequence is compressed data compared to the original image, it can occupy less storage space and is easier to manage.

[0101] This application provides an image storage method for obtaining an image set. To avoid excessive storage space consumption and insufficient storage capacity, and to reduce management burden, this application can cluster and bucket the images in the image set according to their features, obtaining a bucket set. Then, video encoding is performed on the images in each bucket set to obtain a video sequence for each bucket. By clustering and bucketing, more similar images can be added to the same bucket, thus making fuller use of the inter-frame redundancy between similar images for compression, greatly saving storage space and reducing storage costs. Furthermore, the video encoding method only requires managing a small number of video sequences, further reducing the management burden.

[0102] Furthermore, considering the potential for future image retrieval needs after image set storage, to avoid image distortion caused by video encoding, which could prevent the search for authentic images, this application can recover and decode frames in the video sequence to obtain decoded frames. If the image feature similarity between the decoded frame and the corresponding original image is less than a first threshold, the video encoding parameters are adjusted, and the process of encoding the images in each bucket of the bucket set is repeated until the loop termination condition is met. At the end of the loop, the target video sequence for each bucket in the bucket set is obtained and stored. Therefore, this application considers image retrieval needs when storing the image set. While encoding the image set into video after clustering, it achieves high-fidelity compression of the image set through frame decoding and similarity comparison, improving the fidelity of subsequent image retrieval.

[0103] In some embodiments of this application, the process of step S101, "clustering and binning the images in the image set according to image features to obtain a bucket set," is described in detail.

[0104] In one alternative embodiment, clustering and binning can be achieved by binning each image in the image set one by one.

[0105] That is, in this embodiment, the images in the image set can be traversed. For ease of description below, the currently traversed image is taken as the target image. Then, the first distance between the image features of the target image and the cluster center of the candidate bucket can be determined. Here, images have been added to the candidate bucket, and the number of candidate buckets can be one or more.

[0106] Optionally, the first distance can be either cosine distance or Euclidean distance, or other distances; this application does not impose any specific limitations.

[0107] If a candidate bucket contains a first candidate bucket whose distance is less than or equal to a preset distance threshold T, and whose current bucket capacity is less than a preset bucket capacity threshold C, then the target image is added to the first candidate bucket. Optionally, the distance threshold T can range from [0.1, 0.5], and the bucket capacity threshold C can range from [100, 5000]. Optionally, the distance threshold T and bucket capacity threshold C can be pre-validated automatically using A / B testing before clustering and bucketing.

[0108] It is understandable that there may be multiple candidate buckets that satisfy the above distance threshold T and bucket capacity threshold C, thus there are multiple first candidate buckets. In this case, the target image can be added to any of the multiple first candidate buckets, or the target image can be added to the first candidate bucket with the smallest first distance.

[0109] Conversely, if none of the candidate buckets meet the above distance threshold and bucket capacity threshold conditions, then there is no first candidate bucket among the many candidate buckets, and the target image is added to the newly created bucket.

[0110] Since the addition of a target image may change the cluster center of the bucket, this embodiment can update the cluster center of the bucket to which the target image is added based on the image features of the target image. That is, if the target image is added to the first candidate bucket, the cluster center of the first candidate bucket is updated based on the image features of the target image; if the target image is added to a new bucket, the cluster center of the new bucket is updated based on the image features of the target image.

[0111] Optionally, the cluster centers of the buckets to which the target image is added can be updated using a weighted average or K-means method.

[0112] Optionally, after the target image is added to a new bucket, the cluster center of the new bucket is equal to the image features of the target image.

[0113] After the above processing is completed, you can continue to traverse the next target image until all images in the image set have been traversed.

[0114] It should be understood that in the above-described method of adding images one by one to the bucket, there may be situations where a candidate bucket is already full before all images in the image set have been traversed. For ease of description, a candidate bucket that is full before all images in the image set have been traversed is defined as a second candidate bucket. Considering that the untraversed images may be more similar to most of the images in the second candidate bucket, to avoid adding images to the wrong candidate bucket during subsequent traversal, optionally, this embodiment can cluster the images in the second candidate bucket according to image features to obtain a first clustering result, and then split the second candidate bucket into multiple candidate buckets based on the first clustering result.

[0115] For example, m cluster centers are pre-configured. Then, the image features of each image in the second candidate bucket are used to calculate the second distance with the m cluster centers. The cluster center with the smallest second distance is selected, and the image is added to the image cluster where the cluster center is located. The cluster center is then updated. In this way, the images in the second candidate bucket can be clustered into m image clusters, and the second candidate bucket can be split into m candidate buckets. At the same time, the cluster centers of each of the m candidate buckets can be obtained.

[0116] Considering a possible scenario where, although the target image and the first candidate bucket meet the aforementioned distance threshold and bucket capacity threshold, adding the target image to the first candidate bucket might cause the semantic distribution of the first candidate bucket to diverge, meaning the target image reduces the intra-cluster consistency of images within the first candidate bucket. To avoid severe semantic distribution divergence in the first candidate bucket, leading to poor clustering results, this embodiment can optionally update the cluster centers of the first candidate bucket based on the image features of the target image, then calculate the average feature similarity of the image features of the images within the first candidate bucket relative to the updated cluster centers. If the average feature similarity is lower than a preset second threshold, the images within the first candidate bucket are clustered according to their image features to obtain a second clustering result. Based on the second clustering result, the first candidate bucket is then split into multiple candidate buckets.

[0117] Optionally, the process of “calculating the average feature similarity between the image features of the images in the first candidate bucket and the updated cluster centers” may include: calculating the similarity between the image features of each image in the first candidate bucket and the updated cluster centers, using this as the similarity for each image, and taking the average of the similarities for all images in the first candidate bucket to obtain the average feature similarity.

[0118] For example, suppose the first candidate bucket already contains images 1-10, with features 1-10 respectively. The addition of images 1-10 updates the cluster center of the first candidate bucket to cluster center 1. In this embodiment, the similarity between feature 1 and cluster center 1 is calculated to obtain a similarity of 1 for image 1. The similarity between feature 2 and cluster center 1 is calculated to obtain a similarity of 2 for image 2, and so on, until a similarity of 10 for image 10 is obtained. Then, the average of similarities 1-10 is taken to obtain the average feature similarity.

[0119] The process of splitting the first candidate bucket can be referred to the process of splitting the second candidate bucket mentioned above, and will not be repeated here.

[0120] This embodiment calculates the similarity between the image features and cluster centers of the images in the image set, which can group more similar images into one bucket. In order to avoid bucketing errors, a bucket splitting method is also provided, which can achieve more accurate clustering and bucketing, and provide a data foundation for subsequent video encoding.

[0121] In an optional embodiment, in order to manage the target video sequences of each bucket more clearly, this embodiment of the application may also record metadata information for each frame in the target video sequence after obtaining the target video sequence of each bucket.

[0122] Optionally, the metadata information for each frame within the target video sequence may include the following fields: metadata identifier (denoted as vector ID), file identifier within the video sequence (video_file, optional, this identifier can be the name of the video file or the storage path of the video file), frame index within the video sequence (frame_index, this frame index is an integer, optional, the frame index starts from 0), timestamp within the video sequence (timestamp, optional, this timestamp can be seconds or ISO timecode), byte offset within the video sequence (byte_offset, optional field), bucket identifier (bucket_id), frame index of the corresponding decoded keyframe (keyframe_nearest, this decoded keyframe is in the same GOP as this frame), thumbnail path (thumbnail_uri, optional field, thumbnails are used when the requirements for search results are low during user query retrieval), checksum (such as a 256-bit secure hash algorithm (i.e., SHA256)), model name for extracting image features (embed_model), and image feature version number (embed_version).

[0123] It should be noted that the above fields are only examples. In addition, the metadata information of each frame may contain more or fewer fields than those mentioned above. For example, it may also contain the video encoding parameters (compression_params) of the video sequence in which it is located. This application does not impose any specific limitations.

[0124] Optionally, this embodiment may also record an index table (KeyFrameIndex) for the decoded keyframes (I-frames) in the target video sequence of each bucket.

[0125] Optionally, the index table for each decoded keyframe may include the following fields: the frame number (frame_index) in the video sequence, the timestamp (pts or timecode, in seconds or container PTS) in the video sequence, the byte offset (byte_offset, optional field, but preferred for accelerating file addressing) in the video sequence, the checksum (frame_checksum, such as SHA256, used for consistency checking) of the image frame data decoded using the decoded keyframe, the encoded parameter digest (encode_params, for example, the encoded parameter digest can be any of the following values: CRF value, GOP length, pixel format, etc.), and written in JSON format to a centralized metadata storage object (KeyFrameIndex, used for quick access to the retrieval service).

[0126] Of course, the index table of each decoded keyframe may also include other fields, and this application does not impose specific limitations on them.

[0127] Optionally, in this embodiment, after clustering and binning, a bucket summary can be calculated for each bucket to more quickly find the bucket most similar to the query data during subsequent query retrieval, thus accelerating the retrieval process. Here, the bucket summary represents the semantic distribution of images within the bucket.

[0128] Optionally, the bucket summary generation process for any bucket may include: selecting multiple representative images from the images in the bucket according to a preset selection strategy, the selection strategy including one or more of the following strategies: sampling based on the time the image enters the bucket, uniform sampling, and a representative selection strategy based on the distance to the cluster center (here, the cluster center refers to the cluster center updated based on the image features of the last image entering the bucket); and aggregating the image features of the multiple representative images to obtain the bucket summary.

[0129] Optionally, "aggregating the image features of multiple representative images to obtain a bucket summary" may include: calculating the arithmetic mean or weighted average of the image features of multiple representative images to obtain a bucket summary.

[0130] Of course, there are other ways to calculate bucket summaries. For example, the cluster center updated by the image features of the last image added to the bucket can be used as the bucket summary. Alternatively, the image features of all images in the bucket can be aggregated to obtain the bucket summary.

[0131] Optionally, for each bucket, this embodiment can configure a bucket summary table, which may include the following fields: bucket identifier (bucket_id), unique identifier of the target video sequence of the bucket (video_ids, for example, videos / bucket_000123_part0.mp4), bucket summary (bucket_summary_embedding), timestamp of the last update of the bucket summary table (last_updated), hot score (hot_acore, when the hot score is higher than a preset score threshold, the frame data after decoding of the images in the bucket can be pre-cached), average feature similarity (avg_embedding_distance), and video encoding parameters (encoding_params).

[0132] Optionally, this embodiment can also preset a vector library. Optionally, a vector library can be set up for the entire image set, or a vector library can be set up for each bucket.

[0133] In this embodiment, the vector library includes at least image features and image identifiers. For example, the vector library for each bucket includes image features and identifiers for each image within that bucket. Optionally, to correspond with the metadata information mentioned above, the identifier can be a metadata identifier.

[0134] When the vector library only stores the correspondence between image features and identity identifiers, the vector library can be used as a vector retrieval library for Faiss, enabling fast Faiss retrieval.

[0135] Optionally, the vector library may also store the aforementioned metadata information; this application does not impose specific limitations on this.

[0136] Corresponding to the image storage method described above, this application also provides an image retrieval method based on the image storage method. To enable those skilled in the art to better understand the image retrieval method of this application, a detailed description is provided through the following embodiments.

[0137] Please see Figure 3 The following is a flowchart illustrating an image retrieval method provided in an embodiment of this application, as shown below. Figure 3 The image retrieval method may include:

[0138] Step S201: Obtain the bucket summary and vector library for each bucket in the bucket set.

[0139] Here, the bucket summary represents the semantic distribution of images within the bucket, and the vector library includes the image features and identity identifiers of each image within the bucket.

[0140] Step S202: Calculate the first similarity between the query data and the bucket summary of each bucket in the bucket set, so as to determine the similar buckets of the query data from the bucket set based on the first similarity.

[0141] In this embodiment, the bucket summary of each bucket can reflect the semantic distribution of images within the bucket. Therefore, the bucket summary can be used as the basis for bucket queries. Thus, when a user needs to query a specific image or images from the target video sequences of multiple buckets stored above, they can input query data.

[0142] Optionally, the query data can include one or more of text, images, and voice. For example, the query data can be a complete descriptive text, such as "A red car is driving on the highway in sunny weather", or multiple keywords, such as "car, red, daytime, highway".

[0143] Of course, the data to be queried can be other types, and this application does not impose any specific limitations.

[0144] As mentioned earlier, bucket summaries can be calculated from the image features of the images within the bucket. Therefore, bucket summaries can be in vector form. In order to calculate the similarity between the query data and the bucket summaries, it is optional to first extract the features of the query data, such as the embedding feature. The extraction method of this feature is the same as the extraction method of the image features mentioned earlier. For details, please refer to the previous introduction, which will not be repeated here.

[0145] That is, in this embodiment, features of the query data can be extracted, and then the features of the query data can be used to calculate a first similarity with the bucket summary of each bucket in the bucket set. Optionally, the first similarity can be a cosine similarity.

[0146] In this embodiment, by calculating the first similarity of each bucket, the buckets can be sorted to determine the similar buckets that should be processed first.

[0147] For example, the L buckets with the highest similarity can be used as the similarity buckets for the query data, where L is a configurable parameter.

[0148] Step S203: Calculate the second similarity between the query data and each image feature in the vector library of the similar bucket, and determine the identity identifier of the similar image in the query data based on the second similarity, which is then used as the target identity identifier.

[0149] As described above, this embodiment can construct a vector library corresponding to each bucket. Based on this, this embodiment can obtain a vector library of similar buckets, extract features from the query data, and calculate a second similarity, such as cosine similarity or other similarities, with the image features in the vector library of similar buckets. Then, the identity identifier of similar images in the query data, i.e., the target identity identifier, can be determined based on the second similarity.

[0150] Optionally, the retrieval process in this step can be implemented using Faiss.

[0151] Optionally, the process of "determining the identity identifier of similar images in the query data based on the second similarity" may include: comparing the second similarity calculated based on each image feature with a preset third threshold; if the second similarity calculated based on an image feature is greater than or equal to the third threshold, then the image feature is determined to be highly similar to the feature in the query data, and the identity identifier corresponding to the image feature is the target identity identifier.

[0152] Optionally, the image features in the vector library of each similar bucket (or all similar buckets) can be sorted from largest to smallest according to the second similarity, and the top P image features can be selected. The identity identifiers corresponding to these P image features are the target identity identifiers. The P can also be configured.

[0153] Of course, there are other ways to identify the target, which will not be listed in this application.

[0154] Optionally, the above method of retrieving similar images using the third threshold may result in situations where no similar images are found within the L similarity buckets. In such cases, the size of L can be expanded as needed before continuing the retrieval in step S203.

[0155] Optionally, in this embodiment, the remaining buckets other than the L similar buckets can be placed in a secondary waiting queue to asynchronously retrieve similar images (i.e., the target video sequence of the L similar buckets is decoded in real time according to step S204, and the remaining buckets are delayed or skipped in real time decoding).

[0156] Step S204: Obtain the target video sequence of the similar bucket, decode the target video sequence of the similar bucket to obtain the frame data of similar images indicated by the target identity identifier, and use it as the retrieval result of the query data.

[0157] As mentioned above, this application stores image sets in the form of multiple target video sequences. Therefore, in order to obtain the required similar images, it is necessary to decode the target video sequences containing the similar images to obtain the frame data of the similar images, which can then be used as the retrieval results of the query data.

[0158] Optionally, a video decoder can be invoked (preferably using a decoding library interface, such as the application programming interface (API) of libav / FFmpeg) to decode the target video sequence of similar buckets.

[0159] The image retrieval method provided in this application adopts a two-stage retrieval strategy. The first stage is a coarse retrieval stage based on bucket retrieval, and the second stage is a fine-grained retrieval stage based on images. Through the retrieval in the first stage, images that are irrelevant or have low relevance to the query data can be quickly filtered out. Thus, the second stage can focus on images that are more relevant to the query data for similarity matching. At the same time, it can focus on similar buckets for subsequent video decoding, reducing the number of video files that need to be accessed, thereby significantly reducing the number of decoding times and reducing decoding overhead, and improving image query efficiency.

[0160] In one possible implementation, considering that users may have different search needs in different application scenarios, in order to quickly return the query results needed by the user, optionally, the user's needs can be determined before executing step S204. If the user has low demand for similar images, the thumbnails of the similar images that are pre-cached can be returned to the user. If the user needs full-resolution images, the frame data of the similar images can be decoded and returned according to step S204.

[0161] Optionally, user requirements can be input by the user, determined based on the current search scenario, or determined based on the specific user, etc., and this application does not impose specific limitations.

[0162] Considering that some images may be frequently accessed by users—for example, professionals in the automotive industry tend to search for car images—car images may be popular images for them. Therefore, optionally, this embodiment can record the frequency of queries for each frame in the target video sequence of each bucket, thereby pre-decoding and caching images with high query frequency. This allows for direct retrieval of decoded frame data from the cache when needed, enabling faster return of search results with lower latency.

[0163] Optionally, the frequency of the target video sequence being queried in each bucket, as well as the time of the most recent access, can be recorded. Based on the frequency of the query and the time of the most recent access, the hot score of each bucket can be determined. Then, the frequently accessed frames of buckets with higher hot scores are pre-decoded and the decoded frame data is cached, or all frames of buckets with higher hot scores are pre-decoded and the decoded frame data is cached, so as to quickly return the required search results.

[0164] To implement the above caching strategy, a two-level cache can be optionally adopted. The first-level cache caches the most frequently queried and retrieved pre-decoded frame data, and the second-level cache caches thumbnails of all or some of the images in the image set.

[0165] Of course, there are other caching methods, such as caching the pixel data of the most popular original image in the first level, and caching the relatively popular pre-decoded frame data and / or thumbnails in the second level. This application does not make any specific limitations.

[0166] This application provides a caching mechanism and a pre-decoding mechanism, which can save time in decoding frame data of similar images and improve query efficiency.

[0167] In some other embodiments of this application, the process of step S204 above, "obtaining the target video sequence of the similar bucket, decoding the target video sequence of the similar bucket, and obtaining the frame data of the similar image indicated by the target identity identifier", is described.

[0168] Having obtained the target identifier indicating similar images in the preceding steps, this embodiment can acquire metadata information containing this target identifier, which is the metadata information of the similar images. As described above, the metadata information of the similar images includes the file identifier (such as filename or storage address) of the target video sequence of the similar bucket, as well as the first frame sequence number of the decoding keyframe corresponding to the similar image and / or the second frame sequence number of the similar image itself.

[0169] Furthermore, the target video sequence indicated by the file identifier in the metadata information of similar images can be obtained, and this target video sequence is the target video sequence of the similarity bucket.

[0170] This embodiment can also determine the index table of decoding keyframes corresponding to similar images based on the frame sequence number in the metadata information of similar images. Specifically, if the metadata information includes a first frame sequence number, then the index table containing the first frame sequence number is obtained and used as the index table of decoding keyframes corresponding to the similar images. If the metadata information includes a second frame sequence number, then the index table containing the largest frame sequence number not greater than the second frame sequence number can be determined from the pre-configured keyframe index table set; this index table is the index table of decoding keyframes corresponding to the similar images. Here, the keyframe index table set includes the index tables of all decoding keyframes within the target video sequence of the similar bucket.

[0171] Taking the i-th similar bucket among L similar buckets as an example, assuming that the target video sequence of the i-th similar bucket includes 10 decoding keyframes, the index tables of the 10 decoding keyframes form a keyframe index table set. If the frame numbers of the 10 decoding keyframes are 0, 50, 100, 150, 200, 250, 300, 350, 400, and 450 respectively, and the frame number of the similar image (the second frame number) is 330, then the largest frame number not greater than the second frame number is 300. The decoding keyframe with frame number 300 is the decoding keyframe corresponding to the similar image. Therefore, the index table containing the largest frame number not greater than the second frame number is the index table of the decoding keyframe corresponding to the similar image.

[0172] As described above, each decoding keyframe's index table stores its frame location data within the target video sequence. This frame location data includes the timestamp of the video sequence containing the decoding keyframe and / or the byte offset within that sequence. Based on this, this embodiment can determine the frame location data of the decoding keyframe corresponding to a similar image within the target video sequence of similar buckets from the index table of decoding keyframes for similar images.

[0173] Furthermore, addressing within the target video sequence of similar buckets can be performed based on frame localization data to obtain the frame data of the decoding keyframes corresponding to similar images. Here, frame data refers to the encoded video frame data. Then, frame-by-frame decoding is performed based on the frame data of the decoding keyframes corresponding to similar images until the decoded frame data of the similar images is obtained. Here, frame data refers to the decoded frame data.

[0174] In other words, in this embodiment, the frame positioning data is used as the starting point of seek. The video encoder is called to seek the target video sequence of the similar bucket to the decoding key frame corresponding to the similar image. Starting from the decoding key frame, the decoding is performed sequentially until the frame number of the similar image is obtained, thereby obtaining the frame data of the similar image.

[0175] Optionally, consistency checks can be performed on the frame data of similar images obtained from decoding. For example, the checksums in the index table of the decoded keyframes corresponding to similar images can be compared, or image features can be extracted from the frame data of similar images and matched with the image features of the original images in the corresponding vector library. If the similarity is higher than the preset fourth threshold, the consistency check is considered to have passed.

[0176] Furthermore, the frame data that passes the consistency check can be returned to the upper-layer application so that the user can see the search results.

[0177] This embodiment provides a keyframe index-based positioning and partial decoding strategy, which can minimize the input / output interfaces (I / O) and decoding amount required for each retrieval, while taking into account the storage savings brought by video compression and the response latency requirements of the retrieval segment.

[0178] The above describes an image storage and retrieval method provided by an embodiment of this application. The following will describe the apparatus for performing the above image storage and retrieval method.

[0179] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an image storage device provided in an embodiment of this application. Figure 4 As shown, the image storage device may include:

[0180] The data acquisition and processing unit 401 is used to acquire an image set, and to cluster and bucket the images in the image set according to image features to obtain a bucket set.

[0181] Data encoding unit 402 is used to perform video encoding on the images in each bucket of the bucket set to obtain a video sequence for each bucket of the bucket set;

[0182] The data recoding unit 403 is used to recover and decode the frames in the video sequence to obtain a decoded frame. If the image feature similarity between the decoded frame and the original image is less than a first threshold, the unit returns to the method of video encoding the images in each bucket of the bucket set after adjusting the video encoding parameters, until the loop ends. The original image is the image in the image set that corresponds to the decoded frame.

[0183] The data output storage unit 404 is used to obtain the target video sequence of each bucket in the bucket set at the end of the loop and store the target video sequence.

[0184] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of an image retrieval device provided in an embodiment of this application. Figure 5 As shown, the image retrieval device may include:

[0185] The data input unit 501 is used to obtain the bucket summary and vector library of each bucket in the bucket set. The bucket summary represents the semantic distribution of the images in the bucket, and the vector library includes the image features and identity identifiers of each image in the bucket.

[0186] The first calculation unit 502 is used to calculate a first similarity between the query data and the bucket summary of each bucket in the bucket set, so as to determine the similar buckets of the query data from the bucket set based on the first similarity.

[0187] The second calculation unit 503 is used to calculate a second similarity between the query data and each image feature in the vector library of the similar bucket, so as to determine the identity identifier of the similar image of the query data based on the second similarity, and use it as the target identity identifier;

[0188] The data decoding output unit 504 is used to acquire the target video sequence of the similar bucket, decode the target video sequence of the similar bucket, and obtain the frame data of the similar image indicated by the target identity, as the retrieval result of the query data.

[0189] Each module in the aforementioned image storage device and image retrieval device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0190] This application also provides an electronic device, which may include at least one processor and a memory connected to the processor, wherein:

[0191] Memory is used to store computer programs;

[0192] The processor is used to execute computer programs to enable the electronic device to implement any of the image storage methods or any of the image retrieval methods provided in the embodiments of this application.

[0193] refer to Figure 6 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0194] like Figure 6 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. When the electronic device is powered on, the RAM 603 also stores various programs and data required for the operation of the electronic device. The processing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0195] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, memory cards, hard drives, etc.; and communication devices 609. Communication device 609 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0196] This application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device enables the electronic device to implement any of the image storage methods or any of the image retrieval methods provided in this application.

[0197] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the image storage methods or image retrieval methods provided in this application.

[0198] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0199] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0200] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0201] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A method for storing images, characterized in that, include: Obtain an image set, and cluster the images in the image set into buckets according to image features to obtain a bucket set; Video encoding is performed on the images in each bucket of the bucket set to obtain a video sequence for each bucket in the bucket set; The frames in the video sequence are restored and decoded to obtain decoded frames. If the image feature similarity between the decoded frame and the original image is less than a first threshold, the video encoding parameters are adjusted and the process of video encoding the images in each bucket of the bucket set is returned until the loop termination condition is met. The original image is the image in the image set that corresponds to the decoded frame; At the end of the loop, the target video sequence of each bucket in the bucket set is obtained and stored.

2. The image storage method according to claim 1, characterized in that, The process of clustering and binning the images within the image set according to image features to obtain a bucket set includes: Iterate through the images in the image set and use the currently iterated image as the target image; Determine the first distance between the image features of the target image and the cluster centers of the candidate buckets; If there is a first candidate bucket in the candidate bucket whose first distance is less than or equal to a preset distance threshold and whose current bucket capacity is less than a preset bucket capacity threshold, then the target image is added to the first candidate bucket; otherwise, the target image is added to a new bucket. Update the cluster centers of the buckets to which the target image is added based on the image features of the target image, and continue traversing until the traversal is complete.

3. The image storage method according to claim 2, characterized in that, Also includes: If a second candidate bucket is already full before all images in the image set have been traversed, then the images in the second candidate bucket are clustered according to image features to obtain a first clustering result. Based on the first clustering result, the second candidate bucket is split into multiple candidate buckets. And / or, after updating the cluster centers of the first candidate buckets according to the image features of the target image, the average feature similarity of the image features of the images in the first candidate buckets relative to the updated cluster centers is calculated. If the average feature similarity is lower than a preset second threshold, the images in the first candidate buckets are clustered according to the image features to obtain a second clustering result. The first candidate bucket is then split into multiple candidate buckets according to the second clustering result.

4. The image storage method according to claim 3, characterized in that, The calculation of the average feature similarity between the image features of the images in the first candidate bucket and the updated cluster centers includes: Calculate the similarity between the image features of each image in the first candidate bucket and the updated cluster center, and use it as the similarity for each image. The average feature similarity is obtained by taking the average of the similarities of all images in the first candidate bucket.

5. The image storage method according to claim 1, characterized in that, The step of recovering and decoding the frames in the video sequence to obtain decoded frames includes: A preset number of frames are sampled from the video sequence, and each sampled frame is restored and decoded to obtain the decoded frame corresponding to each frame; thus obtaining the preset number of decoded frames. The process of determining the image feature similarity between the decoded frame and the original image includes: Determine the similarity between the image features of each decoded frame and the corresponding original image to obtain a preset number of similarity scores; The average of the similarities of the preset number of frames is taken as the image feature similarity between the decoded frame and the original image.

6. An image retrieval method based on the image storage method according to any one of claims 1-5, characterized in that, include: Obtain the bucket summary and vector library for each bucket in the bucket set. The bucket summary represents the semantic distribution of the images in the bucket, and the vector library includes the image features and identity identifiers of each image in the bucket. Calculate a first similarity between the query data and the bucket summary of each bucket in the bucket set, so as to determine the similar buckets of the query data from the bucket set based on the first similarity; The query data is compared with the features of each image in the vector library of the similarity bucket to calculate a second similarity, so as to determine the identity of the similar image of the query data based on the second similarity, which is used as the target identity. Obtain the target video sequence of the similarity bucket, decode the target video sequence of the similarity bucket to obtain the frame data of the similar image indicated by the target identity, and use it as the retrieval result of the query data.

7. The image retrieval method according to claim 6, characterized in that, The process of generating a bucket summary for any of the buckets includes: Multiple representative images are selected from the images in the bucket according to a preset selection strategy. The selection strategy includes one or more of the following strategies: sampling based on the time the images are added to the bucket, uniform sampling, and a representative selection strategy based on the distance to the cluster center. The image features of the multiple representative images are aggregated to obtain the bucket summary of the bucket.

8. The image retrieval method according to claim 6, characterized in that, The step of obtaining the target video sequence of the similarity bucket, decoding the target video sequence of the similarity bucket, and obtaining the frame data of the similar image indicated by the target identity includes: Obtain metadata information containing the target identity identifier, the metadata information also includes the file identifier of the target video sequence of the similar bucket, and the first frame number of the decoding keyframe corresponding to the similar image and / or the second frame number of the similar image; The target video sequence indicated by the file identifier is obtained as the target video sequence of the similarity bucket; Based on the frame sequence number in the metadata information, determine the index table of the decoding keyframe, and determine the frame location data of the decoding keyframe in the target video sequence of the similar bucket from the index table; Based on the frame positioning data, the target video sequence of the similar bucket is addressed to obtain the frame data of the decoded keyframe; The frame data of the keyframes is decoded frame by frame until the frame data of the similar image is obtained.

9. The image retrieval method according to claim 8, characterized in that, If the metadata information includes the second frame sequence number, then determining the index table of the decoding keyframes based on the frame sequence number in the metadata information includes: The index table containing the largest frame number not greater than the second frame number is determined from the pre-configured keyframe index table set and used as the index table for the decoded keyframe. The keyframe index table set includes the index tables of all decoded keyframes within the target video sequence of the similar bucket.

10. A computer program product, characterized in that, It includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the image storage method as described in any one of claims 1 to 5, or the image retrieval method as described in any one of claims 6 to 9.

11. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the image storage method as described in any one of claims 1 to 5, or the image retrieval method as described in any one of claims 6 to 9.

12. A computer storage medium, characterized in that, The storage medium carries one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the image storage method as described in any one of claims 1 to 5, or the image retrieval method as described in any one of claims 6 to 9.

Images