Image feature extraction method and device, electronic equipment and readable storage medium

Abstract

The invention provides an image feature extraction method and device, electronic equipment and a readable storage medium, and the method comprises the steps: carrying out the preprocessing of obtained to-be-processed data, and obtaining the processed target data and the object type of a target object contained in the target data; determining at least one target feature extraction model based on an object type of a target object contained in the target data and a feature extraction requirement; inputting the target data into each target feature extraction model in sequence, so that each target feature extraction model outputs a feature vector after feature extraction of the target data; and storing the target data and the at least one corresponding feature vector into a vector database so as to determine target query data through the plurality of feature vectors stored in the vector database in a subsequent data query process. In this way, the accuracy of feature extraction of the specific target data can be improved, and then the accuracy of the subsequent query comparison process according to the extracted feature vectors is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to methods, apparatus, electronic devices and readable storage media for extracting image features. Background Technology

[0002] When extracting features from target objects in an image, manual annotation is usually required. After annotation, the operator analyzes the object information contained in the image and then performs subsequent analysis.

[0003] With the development of computer technology, features in images can be extracted and recognized through general AI models and related image processing techniques. However, the processing power of general AI models is generally related to the features of the training samples. If the training samples are not comprehensive enough, the processing accuracy of general AI models will be affected. At the same time, for fields with particularly high requirements for processing accuracy, such as criminal investigation, general AI models may not be able to accurately handle the specific object feature extraction process, which will lead to low accuracy in object feature extraction and subsequent query comparison. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method, apparatus, electronic device and readable storage medium for extracting image features. After preprocessing the acquired data to be processed to obtain target data, at least one target feature extraction model corresponding to the object type of the target object contained in the target data is selected. Based on the selected at least one target feature extraction model, the target data is selectively extracted for features, which can improve the accuracy of feature extraction for specific target data, and thus improve the accuracy of subsequent query and comparison processes based on the extracted feature vectors.

[0005] In a first aspect, embodiments of this application provide a method for extracting image features, the extraction method comprising: Acquire the data to be processed and preprocess the data to obtain the processed target data and the object type of the target object contained in the target data; wherein, the data to be processed includes the image to be processed and / or the video to be processed; Based on the object type of the target object contained in the target data and the feature extraction requirements, at least one target feature extraction model is determined for feature extraction of the target data; The target data is sequentially input into each of the target feature extraction models, so that each of the target feature extraction models outputs a feature vector after feature extraction of the target data; The target data and at least one corresponding feature vector are stored in a vector database so that the target query data can be determined through multiple feature vectors stored in the vector database during subsequent data querying.

[0006] In one possible implementation, storing the target data and at least one corresponding feature vector in a vector database includes: At least one feature vector corresponding to the target data is concatenated to obtain a target concatenated vector; the target data and the target concatenated vector are then stored in a vector database; or... The target data is weighted by summing at least one feature vector corresponding to the target data according to the weight coefficients corresponding to each feature vector to obtain the target weighted vector. The target data and the target weighted vector are then stored in a vector database; or... At least one feature vector corresponding to the target data is input into the vector fusion model to obtain the target fusion vector, and the target data and the target fusion vector are stored in the vector database.

[0007] In one possible implementation, determining at least one target feature extraction model for feature extraction of the target data based on the object type and feature extraction requirements includes: Based on the object type and feature extraction requirements, at least one candidate feature extraction model is determined; wherein, different object types correspond to different candidate feature extraction models, and the feature vectors extracted after processing by the feature extraction model are also different for different object types; For each candidate feature extraction model, a feature extraction evaluation score is determined based on at least one preset evaluation index. Based on the feature extraction evaluation score of each candidate feature extraction model, at least one target feature extraction model is determined from the candidate feature extraction models.

[0008] In one possible implementation, the extraction method further includes: The candidate feature extraction model is evaluated based on at least one preset evaluation index to determine at least one index score for the candidate feature extraction model. For each candidate feature extraction model, the score of at least one indicator corresponding to the candidate feature extraction model is weighted according to the preset indicator weight to determine the model indicator score corresponding to the candidate feature extraction model. The candidate feature extraction models whose model index scores are lower than the preset index scores are updated and optimized to obtain the updated and optimized feature extraction models.

[0009] In one possible implementation, the extraction method further includes: Retrieve the target query information input; Feature extraction is performed on the target query information to determine the target query vector; Determine at least one feature vector from the vector database that has a similarity greater than a preset similarity threshold with the target query vector; The image and / or video corresponding to the feature vector are determined as the target image and / or target video corresponding to the target query information.

[0010] In one possible implementation, the step of extracting features from the target query information to determine the target query vector includes: When the target query information contains query text, the query text is input into a pre-trained natural language processing model, which then performs vector transformation on the query text and outputs the target query vector; or... When the target query information includes a query image, the query image is input into a pre-trained image feature extraction model, so that the image feature extraction model extracts features from the query image and outputs the target query vector; or... When the target query information includes query text and query image, the query text and query image are input into a multimodal feature extraction model, so that the multimodal feature extraction model performs vector transformation on the query text to obtain a text feature vector, performs feature extraction on the query image to obtain an image feature vector, and then merges the text feature vector and the image feature vector to output the target query vector.

[0011] In one possible implementation, after determining the image and / or video corresponding to the feature vector as the target image and / or target video corresponding to the target query information, the extraction method further includes: Determine the similarity score and generated descriptive information for each target image / or target video; The target image / or target video is displayed in a preset display format, combined with the similarity score and the generated descriptive information.

[0012] Secondly, embodiments of this application also provide an image feature extraction device, the extraction device comprising: A data preprocessing module is used to acquire data to be processed and preprocess the data to obtain processed target data and the object type of the target object contained in the target data; wherein, the data to be processed includes images and / or videos to be processed; The feature extraction model selection module is used to determine at least one target feature extraction model for extracting features from the target data based on the object type of the target object contained in the target data and the feature extraction requirements. The feature extraction module is used to sequentially input the target data into each of the target feature extraction models, so that each of the target feature extraction models outputs a feature vector after feature extraction of the target data; The vector database storage module is used to store the target data and at least one corresponding feature vector in the vector database so as to determine the target query data through multiple feature vectors stored in the vector database during subsequent data querying.

[0013] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the image feature extraction method as described in any of the first aspects.

[0014] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the image feature extraction method as described in any of the first aspects.

[0015] The image feature extraction method, apparatus, electronic device, and readable storage medium provided in this application embodiment acquire data to be processed and preprocess the data to obtain processed target data and the object type of the target object contained in the target data; wherein, the data to be processed includes the image to be processed and / or the video to be processed; based on the object type of the target object contained in the target data and the feature extraction requirements, at least one target feature extraction model for feature extraction of the target data is determined; the target data is sequentially input into each target feature extraction model so that each target feature extraction model outputs a feature vector after feature extraction of the target data; the target data and the corresponding at least one feature vector are stored in a vector database so that the target query data can be determined through multiple feature vectors stored in the vector database in the subsequent data query process. In this way, after preprocessing the acquired data to be processed to obtain target data, at least one target feature extraction model corresponding to the object type of the target object contained in the target data is selected, and the target data is extracted in a targeted manner according to the selected at least one target feature extraction model, which can improve the accuracy of feature extraction for specific target data, and thus improve the accuracy of the subsequent query and comparison process based on the extracted feature vectors.

[0016] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating an image feature extraction method provided in an embodiment of this application; Figure 2 This is one of the structural schematic diagrams of an image feature extraction device provided in an embodiment of this application; Figure 3 This is a second schematic diagram of the structure of an image feature extraction device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0020] First, the applicable scenarios for this application will be introduced. This application can be applied to the field of image processing technology.

[0021] When extracting features from target objects in an image, manual annotation is usually required. After annotation, the operator analyzes the object information contained in the image and then performs subsequent analysis.

[0022] With the development of computer technology, some retrieval methods based on traditional image processing techniques have been applied to the field of criminal investigation. These methods typically rely on manually designed feature extraction algorithms, such as SIFT (Scale Invariant Feature Transform), SURF (Speed-Up Robust Feature Transform), and HOG (Histogram of Oriented Gradients), to extract local or global features of images. Then, image matching and retrieval are achieved by calculating the similarity between these feature vectors. For example, some studies have attempted to classify contraband in X-ray images using co-occurrence matrix features after Contourlet transform and Tamura texture features. However, these traditional methods have significant limitations: first, manually designed features are less robust to complex scenes (such as changes in lighting, viewpoint, occlusion, and blur), resulting in low recognition accuracy; second, these methods typically only handle specific types of images or tasks, have weak generalization ability, and are difficult to adapt to the ever-changing scenarios and needs in real-world criminal investigation.

[0023] With the maturity of deep learning technology, attempts have begun to introduce general-purpose AI models (such as CNN-based image classification or object detection models) to assist in processing crime images and videos. These models are pre-trained on large-scale public datasets such as ImageNet and possess a certain degree of general object recognition capability. However, these general-purpose models face numerous challenges when applied to the professional field of criminal investigation. First, there is the "domain gap" problem, meaning that the knowledge learned by the model on general datasets is difficult to directly transfer to crime images with distributions vastly different from the training data. Second, there is the accuracy issue; general-purpose models lack the ability to recognize fine-grained features of specific crime items, making it difficult to meet the high-precision requirements of criminal investigation evidence. Furthermore, these models typically require a large amount of labeled data for fine-tuning, while labeling crime images is costly and requires specialized knowledge, further limiting their application effectiveness.

[0024] Meanwhile, existing image retrieval systems for criminal cases generally have limited interaction methods, mostly restricted to keyword-based text queries. This approach requires users to know certain precise labels of the image beforehand and makes it difficult to express complex query intentions. For example, investigators might want to find "a man wearing a red shirt loitering near an ATM," and these complex query needs cannot be effectively supported by traditional keyword retrieval. Although "image search" technology has been applied in the civilian sector, in the criminal investigation field, how to deeply integrate it with case information and support "image-text hybrid" queries remains a pressing technical challenge. The limitations of the retrieval methods greatly restrict the thinking of investigators, making it difficult for them to quickly and flexibly extract deep-seated clues from massive amounts of data, thus significantly diminishing the value of the data.

[0025] Based on this, embodiments of this application provide a method for extracting image features to improve the accuracy of feature extraction for specific target data, thereby improving the accuracy of subsequent query and comparison processes based on the extracted feature vectors.

[0026] Please see Figure 1 , Figure 1 This is a flowchart illustrating an image feature extraction method provided in an embodiment of this application. Figure 1 As shown in the embodiments of this application, the image feature extraction method includes: S101. Obtain the data to be processed and preprocess the data to be processed to obtain the processed target data and the object type of the target object contained in the target data; wherein, the data to be processed includes the image to be processed and / or the video to be processed.

[0027] S102. Based on the object type of the target object contained in the target data and the feature extraction requirements, determine at least one target feature extraction model for extracting features from the target data.

[0028] S103. The target data is sequentially input into each of the target feature extraction models so that each of the target feature extraction models outputs a feature vector after feature extraction of the target data.

[0029] S104. Store the target data and at least one corresponding feature vector in a vector database so that the target query data can be determined through the multiple feature vectors stored in the vector database during subsequent data querying.

[0030] The image feature extraction method provided in this application preprocesses the acquired data to be processed to obtain target data. Based on the object type of the target object contained in the target data, at least one target feature extraction model corresponding to the object type is selected. Based on the selected at least one target feature extraction model, the target data is selectively extracted for features. This can improve the accuracy of feature extraction for specific target data, thereby improving the accuracy of subsequent query and comparison processes based on the extracted feature vectors.

[0031] The exemplary steps of the embodiments of this application are described below: S101. Obtain the data to be processed and preprocess the data to be processed to obtain the processed target data and the object type of the target object contained in the target data; wherein, the data to be processed includes the image to be processed and / or the video to be processed.

[0032] When extracting features from target objects in an image, manual annotation is usually required. After annotation, the operator analyzes the object information contained in the image and then performs subsequent analysis.

[0033] With the development of computer technology, features in images can be extracted and recognized through general AI models and related image processing techniques. However, the processing power of general AI models is generally related to the features of the training samples. If the training samples are not comprehensive enough, the processing accuracy of general AI models will be affected. At the same time, for fields with particularly high requirements for processing accuracy, such as criminal investigation, general AI models may not be able to accurately handle the specific object feature extraction process, which will lead to low accuracy in object feature extraction and subsequent query comparison.

[0034] Based on this, in the embodiments of this application, after preprocessing the acquired data to be processed to obtain target data, at least one target feature extraction model corresponding to the object type of the target object contained in the target data is selected, and the target data is selectively extracted based on the selected at least one target feature extraction model. This can improve the accuracy of feature extraction for specific target data, thereby improving the accuracy of subsequent query and comparison processes based on the extracted feature vectors.

[0035] The data to be processed here may include pictures and videos from mobile phones, computers, etc.; video streams or recorded files accessed from surveillance networks; and relevant multimedia data obtained from other criminal investigation system platforms.

[0036] Furthermore, in order to ensure the consistency and accuracy of subsequent data processing, the data to be processed needs to be preprocessed after it is obtained.

[0037] In one possible implementation, the preprocessing of the data to be processed may include format conversion and decoding. Specifically, images of different formats, such as JPG and PNG, and videos, such as MP4 and AVI, are uniformly converted into a standard data format for recognition and processing.

[0038] In another possible implementation, the preprocessing of the data to be processed may also include data cleaning to remove data that is not of great significance for processing. Specifically, duplicate, corrupt, or low-quality data to be processed are filtered out, for example, severely blurry or overexposed files, to ensure the quality of the target data obtained after processing.

[0039] In another possible implementation, the preprocessing of the data to be processed may also include basic metadata extraction. Specifically, basic information to be processed, such as shooting time, location information, device model, resolution, etc., is automatically extracted and used as auxiliary information for subsequent analysis.

[0040] In another possible implementation, the preprocessing of the data to be processed may also include preliminary content classification. Specifically, the data to be processed may be input into a content classification model to determine the initial classification information of the data to be processed.

[0041] Here, the content classification model can be a lightweight data processing model.

[0042] For example, inputting the data to be processed into a content classification model can determine whether the specific content contained in the data belongs to the categories of scenery, people, or documents. Here, the classification data can provide reference information for the selection of feature extraction models for subsequent feature extraction of target data, thereby improving the accuracy of feature extraction model selection and thus improving the accuracy of subsequent data feature extraction.

[0043] Furthermore, after preprocessing the data to be processed to obtain the target data, at least one target feature extraction model can be selected based on the object type of the target object contained in the target data.

[0044] S102. Based on the object type of the target object contained in the target data and the feature extraction requirements, determine at least one target feature extraction model for extracting features from the target data.

[0045] In one possible implementation, at least one target extraction model for extracting features from the target data is determined based on the object type of the target object contained in the determined target data and the feature extraction requirements.

[0046] To improve the accuracy of feature extraction for different types of data, it is necessary to match different feature extraction models to different object types.

[0047] For an image containing people, the engine may simultaneously utilize a face recognition model, a human attribute analysis model (such as clothing and posture), and a personal item detection model. The types of models integrated by this engine mainly include: 1) General object detection and classification models (such as the YOLO series and SSD), used for quickly locating and recognizing common objects in images. 2) Specialized item recognition models, which are specifically trained and optimized for specific items involved in cases (such as illegal machinery, knives, and illegal drugs) to achieve high-precision recognition. 3) Image description generation models (such as CLIP), used to understand the overall semantic content of an image and generate natural language descriptions to support text queries. 4) Feature extraction models (such as ResNet), used to convert images or video clips into high-dimensional feature vectors, providing a foundation for subsequent similarity matching.

[0048] Specifically, in the field of criminal investigation, object types can be divided into three categories: contraband, evidence items, and other related items. For each type, specific feature models and feature extraction schemes are developed. Please refer to Table 1, which is a table of feature extraction strategies for object categories.

[0049] Table 1 Object Category Feature Extraction Strategy Table

[0050] In one possible implementation, illegal mechanical devices are typical objects in the identification process in the field of criminal investigation, and a multi-level feature extraction strategy can be adopted to ensure a high recognition rate. Specifically, firstly, at the shape level, a deep learning model (such as a variant based on ResNet or YOLO) is used to learn the overall contour features of the illegal mechanical device, including the geometry and relative positional relationships of key components such as the grip. Secondly, at the texture level, detailed textures such as metallic luster, scratches, and wear on the surface of the illegal mechanical device are analyzed. These features help distinguish real illegal mechanical devices from toy illegal mechanical devices and simulated illegal mechanical devices. Finally, at the component assembly level, the model will learn the component assembly patterns of the illegal mechanical device as a whole object. For example, a complete illegal mechanical device usually contains multiple cooperating components such as the grip. By combining these multi-level features and introducing attention mechanisms (such as CBAM) to focus on key areas in the image, the detection accuracy of illegal mechanical devices under complex backgrounds, different viewing angles, and lighting conditions can be effectively improved.

[0051] In another possible implementation, while knives come in a wide variety of types and shapes, their core characteristics are relatively clear. The following features can be extracted: 1) Contour features: Using edge detection and contour analysis algorithms, the overall contour of the knife is accurately extracted, especially the curves of the blade and back. For folding knives, the contour changes in both open and closed states should also be considered. 2) Blade features: The blade is the most recognizable part of the knife. High-resolution image analysis will be used to extract details such as the blade's thinness, sharpness (judged by reflection and shadow), and any possible serrations. 3) Reflective properties: Metal blades produce unique, bright reflective areas under illumination. Analyzing the shape, intensity, and distribution of these reflective areas can serve as an important basis for determining whether a knife is metal. For model selection, YOLOv5 or YOLOv7 models optimized for small target detection can be used, combined with data augmentation techniques (such as random rotation, cropping, and color dithering) to improve the model's robustness in recognizing knives in different postures and environments.

[0052] In another possible implementation, given the diverse forms and packaging of illicit drugs, the focus shifts from directly identifying the drugs themselves to recognizing their common packaging and concealment features. 1) Packaging Form: Illegal drugs are often packaged in various forms, such as small bags, capsules, tablets, and powders. Training the model can identify these specific packaging forms; for example, identifying the presence of numerous small plastic bags or foil-wrapped lumps in an image. 2) Color and Material: Some illicit drugs or their packaging have specific colors (e.g., white powder, colored tablets) or materials (e.g., transparent plastic bags, foil). Analyzing these color and material characteristics can narrow down the search. 3) Scene Association: Judgment is made by combining scene information. For example, finding a large number of suspicious small packets of powder in an entertainment venue greatly increases the likelihood that they are illicit drugs. Therefore, a multimodal model (such as CLIP) can be used to associate the visual features of the image with the semantic features of the scene description text, thereby improving the accuracy of identifying suspicious items in specific scenes.

[0053] In another possible implementation, regarding the classification of evidentiary items, given the extremely wide variety of evidentiary items, the feature extraction of evidentiary items should not only focus on their physical properties but also on their specific context within the case.

[0054] Specifically, firstly, the strategy for identifying counterfeit goods and funds will focus on their unique value attributes. 1) Currency characteristics: Specialized recognition models will be trained for different currencies, capable of not only identifying the currency type and denomination but also tracking them through features such as serial numbers. The model needs to learn anti-counterfeiting features such as patterns, colors, watermarks, and security threads on banknotes. 2) Luster and shape of valuable items: For valuable items such as gold and silver jewelry, precious stones, and luxury watches, their unique metallic luster, the refractive properties of gemstones, and exquisite geometric shapes are important identification features. High-gloss detection and material analysis algorithms can be used to identify these items. 3) Abnormal quantity: In ordinary life scenarios, the presence of large amounts of cash or valuable items is itself an abnormal event. Therefore, during feature extraction, in addition to identifying the items themselves, it is also necessary to count their quantity. When the quantity exceeds a certain threshold, it is automatically marked as suspicious.

[0055] Secondly, the types of tools used in crimes are very diverse, ranging from common crowbars and screwdrivers to professional lock-picking tools. Their feature extraction strategies require a high degree of professionalism and flexibility. 1) Shape of specific tools: Establish a specialized model library for common tools used in cases. For example, train models to recognize different types of crowbars, pliers, hammers, etc., focusing on their unique head shape and overall structure. 2) Wear marks: Used tools usually leave unique wear marks, scratches, or stains. Although these marks are subtle, they are often important physical evidence. High-resolution image and microscopic image analysis techniques can be used to extract the texture and shape features of these marks for tool identification. 3) Association with illegal activities: Combine behavioral analysis models to determine whether the appearance of the tool is associated with specific illegal activities (such as lock picking or vandalism). For example, when someone is detected holding a crowbar near a door lock, an alarm can be automatically triggered.

[0056] Thirdly, regarding the examination of documentary evidence, for documents such as contracts, bills, and certificates, the information mainly exists in the form of text and seals. 1) Optical Character Recognition (OCR): High-precision optical character recognition (OCR) technology is used to extract text information from images and convert it into an editable and searchable text format. This requires the model to be able to handle various fonts, mixed printed and handwritten text, and complex situations such as seal obscuration. 2) Seal Detection and Recognition: The model is trained to detect red seals in images and extract their outlines, text content, and other features to determine the authenticity and validity of documents. 3) Signature Handwriting Analysis: For handwritten signatures, deep learning models can be used to extract biometric features such as pen pressure, stroke order, and connection habits for identity verification and document authentication.

[0057] In another possible implementation, the processing of other related item categories can also be done in conjunction with contextual clues.

[0058] Firstly, regarding vehicle characteristics, the feature extraction strategy includes: 1) License Plate Recognition (LPR): Utilizing mature LPR technology to accurately identify license plate numbers under complex lighting, angle, and weather conditions. 2) Vehicle Model Recognition: Training the model to recognize vehicle brand, model, year, and other information, which helps to quickly locate specific vehicles. 3) Color Recognition: Accurately identifying the vehicle's main color. 4) Modification Traces: Detecting whether the vehicle has been illegally modified (such as changing the body color, adding a spoiler, etc.), these traces can serve as important clues for tracking suspect vehicles. These features can be combined to form a comprehensive vehicle "profile" for rapid retrieval and trajectory tracking in surveillance videos.

[0059] Secondly, regarding personnel characteristics, in addition to facial recognition, richer personnel features will be extracted: 1) Clothing features: identifying the color, style, and brand logo of a person's shirt, pants, and shoes. 2) Accessory features: detecting whether a person is wearing accessories such as hats, glasses, masks, backpacks, and watches, and analyzing their characteristics. 3) Carried items: analyzing items carried by a person, such as mobile phones, umbrellas, and packages. These fine-grained features combine to form a unique "gait" or "appearance" feature, enabling effective tracking and identification in videos even when the face is obscured.

[0060] Thirdly, regarding scene features, feature extraction strategies include: 1) Geographic location: determining the shooting location through GPS information or landmarks in the image (such as shop signs and street signs). 2) Environmental layout: analyzing the layout of indoor or outdoor scenes, such as room furnishings and street layout, which helps to reconstruct the crime scene. 3) Special markings: detecting special markings that may exist in the scene, such as graffiti, symbols, and advertisements, which may sometimes be related to specific activities. By extracting and analyzing scene features, isolated items and individuals involved in the case can be placed in a specific spatiotemporal context, thereby better understanding the overall picture of the case and improving the accuracy of feature extraction.

[0061] In one possible implementation, in order to ensure the accuracy of feature extraction for target data, when extracting features from target data, in addition to determining the feature extraction model based on the object type of the target object contained in the target data and the feature extraction requirements, the feature model can also be further screened based on at least one preset evaluation index for the model to obtain at least one target feature extraction model that best suits the processing of the current target data, so as to ensure the accuracy of subsequent target data processing.

[0062] Specifically, the step "based on the object type and feature extraction requirements, determine at least one target feature extraction model for extracting features from the target data" includes: a1: Based on the object type and feature extraction requirements, determine at least one candidate feature extraction model; wherein, different object types correspond to different candidate feature extraction models, and the feature vectors extracted after processing by the feature extraction model are also different for different object types.

[0063] a2: For each candidate feature extraction model, determine the feature extraction evaluation score of the candidate feature extraction model based on at least one preset evaluation index.

[0064] a3: Determine at least one target feature extraction model from the candidate feature extraction models according to the feature extraction evaluation score of each candidate feature extraction model.

[0065] In one possible implementation, at least one candidate feature extraction model is determined based on the above-mentioned selection criteria, object type, and feature extraction requirements. Then, the candidate feature extraction model can be further screened based on at least one preset evaluation index.

[0066] Please refer to Table 2 here. Table 2 is a table of preset evaluation indicators. As shown in Table 2, the preset evaluation indicators may include precision, recall, mAP, inference speed, model size, resource consumption, occlusion resistance, fuzziness resistance, and resistance to illumination changes. Different preset evaluation indicators belong to different evaluation dimensions. Specifically, precision, recall, and mAP belong to the accuracy evaluation dimension; inference speed, model size, and resource consumption belong to the efficiency evaluation dimension; and occlusion resistance, fuzziness resistance, and resistance to illumination changes belong to the robustness evaluation dimension.

[0067] Table 2 Pre-set Evaluation Indicators

[0068] In one possible implementation, regarding the accuracy evaluation dimension, precision refers to the proportion of samples predicted as positive by the model that are actually positive. The formula is: Precision = TP / (TP + FP). High precision means a low false positive rate, avoiding the mislabeling of innocent people or irrelevant items. Recall refers to the proportion of all true positive samples that are successfully predicted as positive by the model. The formula is: Recall = TP / (TP + FN). High recall means a low false negative rate, maximizing the detection of all relevant items or people involved in the case. Mean Average Precision (mAP): This is the most commonly used comprehensive performance metric in object detection tasks. It calculates the average precision at different recall levels, averaging across all categories. mAP reflects the model's performance in both precision and recall, and is considered the gold standard for evaluating the overall model performance. The evaluation will focus on mAP@0.5 (mAP with an IoU threshold of 0.5) and the more stringent mAP@0.5:0.95 (mAP average with an IoU threshold from 0.5 to 0.95, with a step size of 0.05), the latter being a better reflection of the model’s positioning accuracy.

[0069] In another possible implementation, for efficiency evaluation, inference speed is typically measured in frames per second (FPS) or the time required to process a single image (ms). High FPS is essential for real-time demanding scenarios such as video analytics. Model size refers to the storage space occupied by the model file, usually measured in MB or GB. Smaller models are easier to deploy on edge computing devices or resource-constrained servers and load faster. Resource consumption primarily refers to the CPU, GPU, and memory resources used by the model during runtime. Models with low resource consumption can reduce system hardware costs and power consumption, improving system stability and scalability.

[0070] In another possible implementation, regarding robustness, the following metrics are considered: Occlusion resistance: the model's recognition performance when the target is partially or mostly occluded; Blur resistance: the model's performance when handling low-quality images such as motion blur and out-of-focus images; Illumination resistance: the model's stability under different lighting conditions (e.g., daytime, nighttime, backlighting, strong light); Viewpoint resistance: the model's consistency when the target appears from different angles (e.g., overhead, under-eye, side view). By calculating these accuracy metrics on the test set, the performance degradation of the model under various complex conditions can be quantitatively evaluated, thus allowing the selection of the most robust model.

[0071] Furthermore, for each candidate feature model, the index score of the candidate feature model on each preset evaluation index can be evaluated separately. Then, according to the importance of each preset evaluation index in the field of criminal investigation, the weight coefficient corresponding to each preset evaluation index is determined, and the index score on each preset evaluation index is weighted and calculated to determine the feature extraction evaluation score of the candidate feature model.

[0072] In one possible implementation, after determining the feature extraction evaluation score of each candidate feature extraction model, the model with a feature extraction evaluation score greater than the preset feature extraction evaluation score can be identified as the target feature extraction model.

[0073] In another possible implementation, after determining the feature extraction evaluation score of each candidate feature extraction model, the candidate feature extraction models can be sorted in descending order of their corresponding feature extraction evaluation scores, and the model whose sorting position is before the preset position can be determined as the target feature extraction model.

[0074] In this embodiment of the application, in order to ensure the accuracy of model selection and thus the accuracy of subsequent feature extraction, different models can be fine-tuned according to their performance on different preset evaluation indicators.

[0075] In one possible implementation, for common, relatively fixed-shape target objects (such as ordinary knives, mobile phones, vehicles, etc.), a general object detection model that performs well on publicly available datasets, such as YOLOv5, YOLOv7, or SSD, will be selected first. These models have the advantages of high speed and high accuracy. To adapt them to the specific data in the criminal investigation field, these pre-trained models will be fine-tuned using collected crime-related image datasets. The fine-tuning process can adjust the parameters of the last few layers of the network, enabling it to learn the feature distribution specific to the crime-related images, thereby significantly improving the recognition accuracy on specific tasks while maintaining high speed. The advantages of this method are short development cycle, relatively low cost, and suitability for rapid deployment and iteration.

[0076] In another possible implementation, for specific items that are complex in form, difficult to identify, and crucial in the case (such as specific types of illegal mechanical devices, disguised illegal drugs, special tools used in crimes, etc.), general-purpose models often fail to meet accuracy requirements. Therefore, dedicated deep learning models can be trained from scratch or based on more powerful backbone networks (such as ResNet-50, ResNet-101) for these key items. In model design, more complex network structures will be employed, and advanced techniques such as attention mechanisms (such as CBAM), deformable convolution, and multi-scale feature fusion (such as FPN, ASPP) will be introduced to enhance the model's ability to perceive fine-grained features and complex scenes.

[0077] In another possible implementation, to achieve deep semantic understanding of image content and support natural language queries, a multimodal model, such as the CLIP (Contrastive Language-Image Pre-Training) model developed by OpenAI, can be introduced. The CLIP model learns through contrastive comparison on massive amounts of image-text pairs, mapping images and text to the same semantic space. This gives the system powerful "zero-shot" classification capabilities, meaning that new objects in an image can be identified without requiring specialized training for new categories; only the corresponding text description is needed. In criminal investigation applications, this means that investigators can search for images using natural language (such as "a black backpack") without needing to pre-build specialized recognition models for all items. The introduction of the CLIP model will greatly enhance the system's flexibility and ease of use, serving as a core technological support for achieving multimodal query functionality.

[0078] In one possible implementation, each candidate feature model can be evaluated according to a preset evaluation index and a preset evaluation frequency, and the evaluation results can be used to determine whether the candidate feature extraction model needs to be updated and optimized.

[0079] Specifically, the extraction method further includes: b1: Evaluate the candidate feature extraction model based on at least one preset evaluation index, and determine at least one index score for the candidate feature extraction model.

[0080] b2: For each candidate feature extraction model, the score of at least one indicator corresponding to the candidate feature extraction model is weighted according to the preset indicator weight to determine the model indicator score corresponding to the candidate feature extraction model.

[0081] b3: Update and optimize the candidate feature extraction models whose model index scores are lower than the preset index scores to obtain the updated and optimized feature extraction models.

[0082] In one possible implementation, for each candidate feature model, the indicator score of the candidate feature model on each preset evaluation index can be evaluated separately. Then, according to the importance of each preset evaluation index in the field of criminal investigation, the weight coefficient corresponding to each preset evaluation index is determined, and the indicator score on each preset evaluation index is weighted and calculated to determine the feature model indicator score of the candidate feature model.

[0083] In one possible implementation, after determining the model index score of each candidate feature extraction model, the models with model index scores lower than the preset model index score can be updated and optimized.

[0084] In another possible implementation, after determining the model index score of each candidate feature extraction model, the candidate feature extraction models can be sorted in order of their corresponding model index scores from low to high, and the models whose sorted positions are before the preset positions can be updated and optimized.

[0085] Furthermore, to further improve the accuracy and robustness of object recognition and image feature extraction, different models can be fused. Specifically, for the same recognition task, multiple different models can be deployed simultaneously (e.g., a general YOLO model and a specialized ResNet model). When a new image is input, all relevant models will make predictions. Then, the system uses a voting mechanism (such as majority voting or weighted voting) to synthesize the predictions of all models and arrive at the final recognition conclusion. This method can effectively reduce the random errors of a single model and improve the reliability of the overall prediction.

[0086] Furthermore, after determining at least one target feature extraction model for processing the target data, the target data can be sequentially input into each target feature extraction model, and each target feature extraction model processes the target data to obtain the feature vector corresponding to the target data.

[0087] S103. The target data is sequentially input into each of the target feature extraction models so that each of the target feature extraction models outputs a feature vector after feature extraction of the target data.

[0088] Specifically, the target data can be input into each target feature extraction model, and each target feature extraction model converts the image and / or video to be processed into a high-dimensional feature vector, outputting multiple feature vectors.

[0089] Furthermore, after determining the feature vectors output by each target feature extraction model after extracting features from the target data, at least one feature vector corresponding to the target data can be stored in the vector database.

[0090] S104. Store the target data and at least one corresponding feature vector in a vector database so that the target query data can be determined through the multiple feature vectors stored in the vector database during subsequent data querying.

[0091] In this embodiment of the application, the number of feature vectors obtained after feature extraction of a target data may be multiple feature vectors. When storing multiple feature vectors corresponding to the target data into a vector database, the multiple feature vectors can be fused and then stored into the vector database.

[0092] Specifically, the step "store the target data and at least one corresponding feature vector in the vector database" includes: c1: Concatenate at least one feature vector corresponding to the target data to obtain a target concatenated vector, and store the target data and the target concatenated vector in a vector database; or... c2: Weight the target data by averaging at least one feature vector according to the weight coefficients corresponding to each feature vector to obtain a target weighted vector; then store the target data and the target weighted vector in a vector database; or... c3: Input at least one feature vector corresponding to the target data into the vector fusion model to obtain the target fusion vector, and store the target data and the target fusion vector in the vector database.

[0093] In one possible implementation, at least one feature vector corresponding to the target data can be directly concatenated end to end to form a target concatenated vector, and the target data and the target concatenated vector can be stored in a vector database.

[0094] In another possible implementation, different weight coefficients can be set for the feature vectors output by different target feature extraction models. After weighting at least one feature vector corresponding to the target data according to the weight coefficients corresponding to each feature vector, the target weighted vector is obtained. The target data and the target weighted vector are then stored in a vector database.

[0095] Here, the weighting coefficients can be set based on the importance of different features or automatically learned through machine learning models.

[0096] In another possible implementation, a vector fusion model can be trained based on an attention mechanism, at least one feature vector corresponding to the target data can be input into the vector fusion model to obtain a target fusion vector, and the target data and the target fusion vector can be stored in a vector database.

[0097] In this embodiment of the application, after the above processing, when the target data is an image to be processed, it can be represented as a single, high-dimensional, information-rich feature vector and stored in the vector database; when the target data is a video to be processed, a video may be split into multiple images. Therefore, when a video is stored in the vector database, it may correspond to the storage of multiple feature vectors.

[0098] Here, the vector database can be responsible for persistently storing vector data and its associated target data, recognition results, and other information on the disk, and provide data backup, recovery, and update management functions to ensure data security and reliability.

[0099] Furthermore, after determining the vector database, the target image / video that matches the user's input query information can be retrieved by indexing the vector database.

[0100] Specifically, the extraction method further includes: d1: Retrieves the target query information.

[0101] d2: Extract features from the target query information to determine the target query vector.

[0102] d3: Determine at least one feature vector from the vector database that has a similarity greater than a preset similarity threshold with the target query vector.

[0103] d4: Determine the image and / or video corresponding to the feature vector as the target image and / or target video corresponding to the target query information.

[0104] In one possible implementation, the target query information input by the user can correspond to different forms. For different forms of query information, the query information can be processed differently to determine the target query vector, which is then compared with the feature vectors stored in the vector database to determine the target image / or target video.

[0105] Specifically, the step "extracting features from the target query information to determine the target query vector" includes: e1: When the target query information contains query text, the query text is input into a pre-trained natural language processing model, so that the natural language processing model performs vector transformation on the query text and outputs the target query vector; or... e2: When the target query information includes a query image, the query image is input into a pre-trained image feature extraction model, so that the image feature extraction model extracts features from the query image and outputs the target query vector; or... e3: When the target query information includes query text and query image, the query text and query image are input into the multimodal feature extraction model, so that the multimodal feature extraction model performs vector transformation on the query text to obtain a text feature vector, performs feature extraction on the query image to obtain an image feature vector, and after fusing the text feature vector and the image feature vector, the target query vector is output.

[0106] In one possible implementation, users can directly enter query text to perform the query. For example, a user can enter: "Find all white vans that appeared at the XX intersection between 10 pm and 12 am last night" or "Find all pictures of someone using a USB flash drive to copy files in an office setting".

[0107] Furthermore, after receiving the query text, the query text can be input into a pre-trained natural language processing model, so that the natural language processing model can perform vector transformation on the query text and output the target query vector.

[0108] In another possible implementation, the user can simply enter a query image; for example, the user uploads an image containing the target person.

[0109] Furthermore, the query image is input into a pre-trained image feature extraction model, so that the image feature extraction model can extract features from the query image and output the target query vector.

[0110] In another possible implementation, when a user enters both the query text and the query image, for example, the user uploads an image containing the target person and enters the query text description "find a photo of this person wearing a black baseball cap".

[0111] Furthermore, the query text and query image are input into the multimodal feature extraction model, so that the multimodal feature extraction model performs vector transformation on the query text to obtain text feature vectors, extracts features from the query image to obtain image feature vectors, and then fuses the text feature vectors and image feature vectors to output the target query vector.

[0112] In one possible implementation, after determining the target query vector, the similarity between the target query vector and the feature vectors stored in the vector database can be calculated, and then the feature vectors that match the target query vector can be selected, thereby determining the target image / or target video corresponding to the target query information.

[0113] Here, the similarity between the target query vector and the feature vectors stored in the vector database can be calculated using methods such as Euclidean distance, cosine similarity, and inner product.

[0114] Specifically, Euclidean distance measures the straight-line distance between two vectors in a multidimensional space. The smaller the distance, the more similar they are. It is suitable when the dimensions of feature vectors have the same scale. Cosine similarity measures the cosine value of the angle between two vectors. It focuses on the direction of the vectors rather than their magnitude, and is suitable for scenarios such as text or image semantic features where directional consistency is more important. The closer the cosine value is to 1, the more consistent the directions, i.e., the more similar they are. In some cases, the inner product can also be used as a measure of similarity. After calculating the similarity between the query vector and all candidate vectors in the database, the system sorts them according to the similarity score and returns the top N results (Top-N) to the user. Users can use the similarity score to judge the confidence level of the results, and then conduct subsequent analysis and decision-making.

[0115] In one possible implementation, after determining the similarity between the target query vector and each feature vector stored in the vector database, the image / or video corresponding to the feature vector with a similarity greater than a preset similarity threshold can be identified as the target image / or target video corresponding to the target query information.

[0116] In another possible implementation, after determining the similarity between the target query vector and each feature vector stored in the vector database, the feature vectors can be sorted in descending order of their corresponding feature similarity. The image / or video corresponding to the feature vector whose sorting position is before the preset position is then determined as the target image / or target video corresponding to the target query information.

[0117] For example, when the target query information contains query text, the query process for images or videos can be as follows: First, use natural language processing (NLP) technology to parse the user's query statement and extract key entities (such as time, location, and objects) and relationships; then, convert this semantic information into a high-dimensional query vector; finally, perform a similarity search in the vector database to find the image or video that best matches the query vector.

[0118] When the target query information includes a query image, the query process for images or videos can be as follows: First, run the same feature extraction process as when building the database on the uploaded query image to generate a query feature vector; then, in the vector database, calculate the similarity between the query vector and all stored vectors; finally, return a list of the most similar results.

[0119] When the target query information includes both query text and query image, the query process for images or videos can be as follows: extract the feature vectors of the image and text respectively, then fuse them (e.g., by weighted summation) into a comprehensive query vector, and then perform a similarity search.

[0120] Furthermore, after identifying the target image / or target video that matches the target query information, the target image / or target video can be displayed to the user, who can then make further selections based on the query results.

[0121] Specifically, after the step of "determining the image and / or video corresponding to the feature vector as the target image / or target video corresponding to the target query information", the extraction method further includes: f1: Determine the similarity score and generated descriptive information for each target image / or target video.

[0122] f2: Display the target image / or target video in a preset display format, combined with the similarity score and the generated descriptive information.

[0123] In one possible implementation, the target image / or target video can be displayed as a thumbnail. At the same time, the calculated similarity score, as well as the tags and descriptions generated by the AI ​​model, can be displayed. Users can further filter, sort, and perform correlation analysis on the results. For example, a face in the search results can be compared with a population database, or an item can be associated with physical evidence mentioned in a case record.

[0124] In another possible implementation, while keeping the overall technical concept of the present application unchanged, an equivalent alternative can also implement the image feature extraction method in the embodiments of the present application.

[0125] Specifically, without employing deep learning, traditional machine learning models can be used as an alternative. For example, low-level features of images can be manually extracted using algorithms such as SIFT, SURF, and HOG, and then trained and recognized using classifiers such as Support Vector Machine (SVM), Random Forest, or K-Nearest Neighbors (KNN).

[0126] Another possible implementation is to directly use a powerful, general-purpose multimodal large model (such as GPT-4V, Gemini, etc.) to handle all tasks uniformly. The advantage of this approach is that it is relatively simple to implement, does not require the design and maintenance of complex model libraries, and large models usually have strong zero-shot and few-shot learning capabilities. However, it may be costly.

[0127] In another possible implementation, for the vector retrieval part, in addition to mainstream vector databases such as Milvus and Pinecone, other open-source solutions such as Faiss and Weaviate can be considered, or a custom retrieval engine based on different indexing technologies can be implemented. For example, only quantized IVF indexes can be used, or only graph-based HNSW indexes can be used. These alternatives each have their own focus in terms of performance, functionality, and ease of use. For example, Faiss performs well in single-machine performance, while Milvus has advantages in distributed deployment and cloud-native features. The choice of which solution to use depends on the specific application scenario, data scale, performance requirements, and budget.

[0128] The image feature extraction method provided in this application involves acquiring data to be processed and preprocessing the data to obtain processed target data and the object type of the target object contained in the target data. The data to be processed includes images and / or videos to be processed. Based on the object type of the target object contained in the target data and the feature extraction requirements, at least one target feature extraction model is determined to extract features from the target data. The target data is sequentially input into each target feature extraction model, so that each target feature extraction model outputs a feature vector after feature extraction from the target data. The target data and the corresponding at least one feature vector are stored in a vector database so that the target query data can be determined through multiple feature vectors stored in the vector database during subsequent data querying. In this way, after preprocessing the acquired data to be processed to obtain target data, at least one target feature extraction model corresponding to the object type of the target object contained in the target data is selected. Targeted feature extraction of the target data is performed based on the selected at least one target feature extraction model, which can improve the accuracy of feature extraction for specific target data, thereby improving the accuracy of subsequent query and comparison processes based on the extracted feature vectors.

[0129] Based on the same inventive concept, this application also provides an image feature extraction device corresponding to the image feature extraction method. Since the principle of the device in this application is similar to the image feature extraction method described above in this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0130] Please see Figure 2 , Figure 3 , Figure 2 This is one of the structural schematic diagrams of an image feature extraction device provided in an embodiment of this application. Figure 3 This is a second schematic diagram of an image feature extraction device provided in an embodiment of this application. Figure 2 As shown, the extraction device 200 includes: The data preprocessing module 210 is used to acquire data to be processed and preprocess the data to be processed to obtain processed target data and the object type of the target object contained in the target data; wherein, the data to be processed includes images and / or videos to be processed. The feature extraction model selection module 220 is used to determine at least one target feature extraction model for extracting features from the target data based on the object type of the target object contained in the target data and the feature extraction requirements. Feature extraction module 230 is used to sequentially input the target data into each of the target feature extraction models, so that each of the target feature extraction models outputs a feature vector after feature extraction of the target data; The vector database storage module 240 is used to store the target data and at least one corresponding feature vector in a vector database so as to determine the target query data through multiple feature vectors stored in the vector database during subsequent data querying.

[0131] In one possible implementation, when the vector database storage module 240 is used to store the target data and at least one corresponding feature vector into the vector database, the vector database storage module 240 is configured to: At least one feature vector corresponding to the target data is concatenated to obtain a target concatenated vector; the target data and the target concatenated vector are then stored in a vector database; or... The target data is weighted by summing at least one feature vector corresponding to the target data according to the weight coefficients corresponding to each feature vector to obtain the target weighted vector. The target data and the target weighted vector are then stored in a vector database; or... At least one feature vector corresponding to the target data is input into the vector fusion model to obtain the target fusion vector, and the target data and the target fusion vector are stored in the vector database.

[0132] In one possible implementation, when the feature extraction model selection module 220 determines at least one target feature extraction model for feature extraction of the target data based on the object type and feature extraction requirements, the feature extraction model selection module 220 is configured to: Based on the object type and feature extraction requirements, at least one candidate feature extraction model is determined; wherein, different object types correspond to different candidate feature extraction models, and the feature vectors extracted after processing by the feature extraction model are also different for different object types; For each candidate feature extraction model, a feature extraction evaluation score is determined based on at least one preset evaluation index. Based on the feature extraction evaluation score of each candidate feature extraction model, at least one target feature extraction model is determined from the candidate feature extraction models.

[0133] In one possible implementation, such as Figure 3 As shown, the extraction device 200 further includes a model update module 250, which is used for: The candidate feature extraction model is evaluated based on at least one preset evaluation index to determine at least one index score for the candidate feature extraction model. For each candidate feature extraction model, the score of at least one indicator corresponding to the candidate feature extraction model is weighted according to the preset indicator weight to determine the model indicator score corresponding to the candidate feature extraction model. The candidate feature extraction models whose model index scores are lower than the preset index scores are updated and optimized to obtain the updated and optimized feature extraction models.

[0134] In one possible implementation, such as Figure 3 As shown, the extraction device 200 further includes a query module 260, which is used for: Retrieve the target query information input; Feature extraction is performed on the target query information to determine the target query vector; Determine at least one feature vector from the vector database that has a similarity greater than a preset similarity threshold with the target query vector; The image and / or video corresponding to the feature vector are determined as the target image and / or target video corresponding to the target query information.

[0135] In one possible implementation, when the query module 260 is used to extract features from the target query information and determine the target query vector, the query module 260 is used to: When the target query information contains query text, the query text is input into a pre-trained natural language processing model, which then performs vector transformation on the query text and outputs the target query vector; or... When the target query information includes a query image, the query image is input into a pre-trained image feature extraction model, so that the image feature extraction model extracts features from the query image and outputs the target query vector; or... When the target query information includes query text and query image, the query text and query image are input into a multimodal feature extraction model, so that the multimodal feature extraction model performs vector transformation on the query text to obtain a text feature vector, performs feature extraction on the query image to obtain an image feature vector, and then merges the text feature vector and the image feature vector to output the target query vector.

[0136] In one possible implementation, such as Figure 3 As shown, the extraction device 200 further includes a query result display module 270, which is used for: Determine the similarity score and generated descriptive information for each target image / or target video; The target image / or target video is displayed in a preset display format, combined with the similarity score and the generated descriptive information.

[0137] The image feature extraction apparatus provided in this application acquires data to be processed and preprocesses the data to obtain processed target data and the object type of the target object contained in the target data. The data to be processed includes images and / or videos to be processed. Based on the object type of the target object contained in the target data and feature extraction requirements, at least one target feature extraction model for feature extraction of the target data is determined. The target data is sequentially input into each target feature extraction model so that each target feature extraction model outputs a feature vector after feature extraction of the target data. The target data and the corresponding at least one feature vector are stored in a vector database so that the target query data can be determined through multiple feature vectors stored in the vector database during subsequent data querying. Thus, after preprocessing the acquired data to be processed to obtain target data, at least one target feature extraction model corresponding to the object type of the target object contained in the target data is selected. Target feature extraction is then performed on the target data in a targeted manner based on the selected at least one target feature extraction model, which improves the accuracy of feature extraction for specific target data, thereby improving the accuracy of subsequent query and comparison processes based on the extracted feature vectors. Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 400 includes a processor 410, a memory 420, and a bus 430.

[0138] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, they can perform the operations described above. Figure 1 The steps of the image feature extraction method in the illustrated method embodiment can be found in the method embodiment for specific implementation, and will not be repeated here.

[0139] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps of the image feature extraction method in the illustrated method embodiment can be found in the method embodiment for specific implementation, and will not be repeated here.

[0140] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0141] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0142] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0143] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0144] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0145] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for extracting image features, characterized in that, The extraction method includes: Acquire the data to be processed and preprocess the data to obtain the processed target data and the object type of the target object contained in the target data; wherein, the data to be processed includes the image to be processed and / or the video to be processed; Based on the object type of the target object contained in the target data and the feature extraction requirements, at least one target feature extraction model is determined for feature extraction of the target data; The target data is sequentially input into each of the target feature extraction models, so that each of the target feature extraction models outputs a feature vector after feature extraction of the target data; The target data and at least one corresponding feature vector are stored in a vector database so that the target query data can be determined through multiple feature vectors stored in the vector database during subsequent data querying.

2. The extraction method according to claim 1, characterized in that, The step of storing the target data and at least one corresponding feature vector into a vector database includes: At least one feature vector corresponding to the target data is concatenated to obtain a target concatenated vector; the target data and the target concatenated vector are then stored in a vector database; or... The target data is weighted by summing at least one feature vector corresponding to the target data according to the weight coefficients corresponding to each feature vector to obtain the target weighted vector. The target data and the target weighted vector are then stored in a vector database; or... At least one feature vector corresponding to the target data is input into the vector fusion model to obtain the target fusion vector, and the target data and the target fusion vector are stored in the vector database.

3. The extraction method according to claim 1, characterized in that, The step of determining at least one target feature extraction model for extracting features from the target data based on the object type and feature extraction requirements includes: Based on the object type and feature extraction requirements, at least one candidate feature extraction model is determined; wherein, different object types correspond to different candidate feature extraction models, and the feature vectors extracted after processing by the feature extraction model are also different for different object types; For each candidate feature extraction model, a feature extraction evaluation score is determined based on at least one preset evaluation index. Based on the feature extraction evaluation score of each candidate feature extraction model, at least one target feature extraction model is determined from the candidate feature extraction models.

4. The extraction method according to claim 3, characterized in that, The extraction method further includes: The candidate feature extraction model is evaluated based on at least one preset evaluation index to determine at least one index score for the candidate feature extraction model. For each candidate feature extraction model, the score of at least one indicator corresponding to the candidate feature extraction model is weighted according to the preset indicator weight to determine the model indicator score corresponding to the candidate feature extraction model. The candidate feature extraction models whose model index scores are lower than the preset index scores are updated and optimized to obtain the updated and optimized feature extraction models.

5. The extraction method according to claim 1, characterized in that, The extraction method further includes: Retrieve the target query information input; Feature extraction is performed on the target query information to determine the target query vector; Determine at least one feature vector from the vector database that has a similarity greater than a preset similarity threshold with the target query vector; The image and / or video corresponding to the feature vector are determined as the target image and / or target video corresponding to the target query information.

6. The extraction method according to claim 5, characterized in that, The step of extracting features from the target query information to determine the target query vector includes: When the target query information contains query text, the query text is input into a pre-trained natural language processing model, so that the natural language processing model performs vector transformation on the query text and outputs the target query vector; or... When the target query information includes a query image, the query image is input into a pre-trained image feature extraction model, so that the image feature extraction model extracts features from the query image and outputs the target query vector; or... When the target query information includes query text and query image, the query text and query image are input into a multimodal feature extraction model, so that the multimodal feature extraction model performs vector transformation on the query text to obtain a text feature vector, performs feature extraction on the query image to obtain an image feature vector, and then merges the text feature vector and the image feature vector to output the target query vector.

7. The extraction method according to claim 5, characterized in that, After determining the image and / or video corresponding to the feature vector as the target image and / or target video corresponding to the target query information, the extraction method further includes: Determine the similarity score and generated descriptive information for each target image / or target video; The target image / or target video is displayed in a preset display format, combined with the similarity score and the generated descriptive information.

8. An image feature extraction device, characterized in that, The extraction device includes: A data preprocessing module is used to acquire data to be processed and preprocess the data to obtain processed target data and the object type of the target object contained in the target data; wherein, the data to be processed includes images and / or videos to be processed; The feature extraction model selection module is used to determine at least one target feature extraction model for extracting features from the target data based on the object type of the target object contained in the target data and the feature extraction requirements. The feature extraction module is used to sequentially input the target data into each of the target feature extraction models, so that each of the target feature extraction models outputs a feature vector after feature extraction of the target data; The vector database storage module is used to store the target data and at least one corresponding feature vector in the vector database so as to determine the target query data through multiple feature vectors stored in the vector database during subsequent data querying.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is in operation, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of the image feature extraction method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the image feature extraction method according to any one of claims 1 to 7.

Images