Image query method, device and program product

By extracting traditional features and depth features in image materials, generating fusion features, and matching features, the problem of time-consuming, labor-intensive and poor accuracy of existing image query methods is solved, and efficient and accurate image query is achieved.

CN119088997AActive Publication Date: 2024-12-06GUANGZHOU BAOLUN ELECTRONICS CO LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202410995937.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-12-06
Estimated Expiration
2044-07-24

Smart Images

  • Figure CN119088997A_ABST
    Figure CN119088997A_ABST
Patent Text Reader

Abstract

The invention provides an image query method and device and a program product, and relates to the technical field of image search. The image query method comprises the steps of obtaining an image material corresponding to a to-be-queried image; extracting traditional features and depth features in a to-be-queried image and the image material according to a preset scale, and generating fusion features corresponding to the traditional features and the depth features; performing feature matching on the to-be-queried image and the image material based on the fusion feature and a preset scale; and displaying a target image matched with the to-be-queried image in the image material according to a matching result. According to the embodiment of the invention, the target image to be matched with the to-be-queried image can be automatically searched in the image material, the query mode is simple, the efficiency is high, omission and errors are not easy to occur, and the accuracy of image query is effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image search technology, and in particular to an image query method, device, and program product. Background Art

[0002] With the development of electronic technology and the advancement of image capture technology, people have generated a large amount of image materials through various image capture devices (such as mobile phones, cameras, webcams, etc.). These image materials often contain a lot of valuable image information. For example, when a user loses an item in a public place with surveillance equipment, the user can analyze and query the surveillance video output by the surveillance equipment based on the relevant information about the item provided by the user, and find the image information of the item in the surveillance video to quickly find the item.

[0003] At present, the query of image information in image materials still relies on manual image retrieval, which is not only time-consuming and labor-intensive, but also has low query efficiency and is prone to errors, thus reducing the accuracy of image query. Summary of the invention

[0004] The present application provides an image query method, device and program product, which can solve the problems of the existing image query method being time-consuming and labor-intensive, low query efficiency, prone to errors and poor accuracy. To achieve this purpose, the present application provides the following solutions.

[0005] According to one aspect of an embodiment of the present application, there is provided an image query method, comprising:

[0006] Acquire image material corresponding to the image to be queried, where the image material includes at least one of a video and a picture;

[0007] Extracting traditional features and deep features from the query image and the image material according to a preset scale, and generating fusion features corresponding to the traditional features and the deep features;

[0008] Performing feature matching between the image to be queried and the image material based on the fusion feature and a preset scale;

[0009] The target image in the image material that matches the image to be queried is displayed according to the matching result.

[0010] In a possible implementation, obtaining the image material corresponding to the image to be queried includes:

[0011] Acquire the image to be queried according to the acquired image interception operation;

[0012] If it is determined that the preset conditions are currently met, the image material corresponding to the image to be queried is obtained according to the preset conditions, and the preset conditions include at least one of image material import, receiving a search instruction, and currently loading image material.

[0013] In a possible implementation, the step of acquiring the image to be queried according to the acquired image capture operation includes:

[0014] Loading a video file and playing the video screen corresponding to the video file;

[0015] A cropping area in the video picture is determined according to an image capture operation of the captured object, and the video picture is cropped based on the cropping area to obtain the image to be queried.

[0016] In a possible implementation, extracting the traditional features and the deep features from the query image and the image material according to a preset scale includes:

[0017] Constructing an image pyramid corresponding to the image to be queried and the image material based on the preset scale, wherein the preset scale includes at least one of an object size and an image resolution;

[0018] The traditional features and the deep features are extracted at each scale of the image pyramid.

[0019] In a possible implementation, generating fusion features corresponding to the traditional features and the deep features includes:

[0020] The current weights corresponding to the traditional features and the deep features are obtained, and the traditional features and the deep features are fused based on the current weights to generate the fused features, wherein the current weights are generated based on the initial weights, the current scene, the current scale, and error feedback.

[0021] In a possible implementation, performing feature matching on the query image and the image material based on the fusion feature and the preset scale includes:

[0022] Matching the fusion features of the image to be queried with the fusion features of the image material at each scale corresponding to the preset scale;

[0023] The matching result between the image to be queried and the image material is obtained according to the matching result corresponding to each scale.

[0024] In a possible implementation, obtaining the matching result between the image to be queried and the image material according to the matching result corresponding to each scale includes:

[0025] Acquire matching data within a preset time period according to the matching results, wherein the matching data includes the quantity and quality of the matching results;

[0026] Extraction parameters corresponding to the traditional features and the deep features are adjusted based on the matching data, where the extraction parameters include feature extraction density.

[0027] In a possible implementation, the image material is a video, and after obtaining the matching result between the image to be queried and the image material according to the matching result corresponding to each scale, the method includes:

[0028] Acquire the matched feature points in each frame according to the matching results;

[0029] The motion trajectory of the feature point is tracked according to the time sequence information of the video, and the feature point is selected based on the motion trajectory.

[0030] According to one aspect of an embodiment of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method as described above.

[0031] According to one aspect of an embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0032] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0033] The image query method provided by the present application obtains the image material corresponding to the image to be queried; extracts the traditional features and depth features in the image to be queried and the image material according to a preset scale, and generates fusion features corresponding to the traditional features and the depth features; performs feature matching between the image to be queried and the image material based on the fusion features and the preset scale; and displays the target image in the image material that matches the image to be queried according to the matching result. The embodiment of the present application extracts the traditional features and depth features in the image to be queried and the image material, generates fusion features by combining the traditional features and the depth features, performs feature matching between the query image and the image material according to the fusion features and the preset scale, and obtains the target image in the image material that corresponds to the image to be queried. The embodiment of the present application can automatically search for the target image to be matched with the image to be queried in the image material, and the query method is simple and efficient, and is not prone to omissions and errors, thereby effectively improving the accuracy of image query. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following is a brief introduction to the drawings required for use in describing the embodiments of the present application.

[0035] Figure 1A flowchart of an image query method provided in an embodiment of the present application;

[0036] Figure 2 A flowchart of image matching in an image query method provided in an embodiment of the present application;

[0037] Figure 3 A flowchart of the image query method provided in the embodiment of the present application;

[0038] Figure 4 A flowchart of image capture provided in an embodiment of the present application;

[0039] Figure 5 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0041] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation of other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the present technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the one element may be directly connected or coupled to the other element, or it may refer to that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" indicates that it is implemented as "A", or is implemented as "A", or is implemented as "A and B".

[0042] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0043] The following describes several exemplary embodiments to illustrate the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application. It should be noted that the following embodiments can refer to, draw on or combine with each other, and the same terms, similar features and similar implementation steps in different embodiments will not be described repeatedly.

[0044] The image query method, device, and program product provided in this application are intended to solve at least one technical problem existing in the prior art.

[0045] An image query method is provided in an embodiment of the present application. The device executing the image query method may be a mobile phone, a server, a laptop computer, a monitoring device management platform, or other device capable of acquiring images to be queried and image materials to be matched.

[0046] Alternatively, if Figure 1-Figure 4 As shown, the image query method includes:

[0047] S101: Obtain image material corresponding to the image to be queried.

[0048] Optionally, the image material includes at least one of a video and a picture. Acquiring the image material corresponding to the image to be queried includes: acquiring the image to be queried according to the acquired image capture operation; if it is determined that the preset condition is currently met, acquiring the image material corresponding to the image to be queried according to the preset condition, the preset condition including at least one of image material import, receiving a search instruction, and currently loading the image material.

[0049] Optionally, the image to be queried may not be obtained through an image capture operation, but may be obtained through an imported image to be queried according to an import operation of a user or may be determined according to an image to be queried indication instruction input by a user.

[0050] Optionally, after obtaining the image to be queried, the image to be queried can be saved, and in order to facilitate the user to view the image to be queried, after obtaining the image to be queried, a new window can be displayed to display the image to be queried in the new window. In order to facilitate the user to view and find the image to be queried, the image to be queried can also be highlighted.

[0051] Optionally, when the preset condition is the import of image material, the imported image material can be determined as the image material corresponding to the image to be queried; when the preset condition is the reception of a search instruction, the currently loaded image material can be determined as the image material corresponding to the image to be queried, or the image material corresponding to the instruction to be queried can be obtained based on the information in the search instruction or the image material selection instruction related to the search instruction; when the preset condition is that the image material is currently loaded, the currently loaded image material can be directly determined as the image material corresponding to the image to be queried.

[0052] Optionally, obtaining the image to be queried according to the acquired image capture operation includes: loading a video file and playing a video screen corresponding to the video file; determining a cropping area in the video screen according to the image capture operation of the captured object, and cropping the video screen based on the cropping area to obtain the image to be queried. The captured object may be a user, and the cropping area in the video screen is determined by the image capture operation performed by the user.

[0053] Optionally, the image capture operation may include an image capture tool selection operation and an image capture tool movement operation, and the image capture tool used to capture the image is determined according to the image capture tool selection operation, and the image capture tool is moved to the image screen based on the image capture tool movement operation, and the area occupied by the image capture tool in the video screen is determined as the cropping area. Human-computer interaction information (such as the movement track of the mouse on the video screen, the finger sliding track, etc.) can also be obtained through the image capture operation, and the cropping area is determined according to the human-computer interaction information.

[0054] In one embodiment, a Qt-based multimedia module or a third-party library (such as FFmpeg) is used to read a video file, and the video frame in the video file is converted to the image format of Qt. The video frame is displayed on the interface using Qt's image display control (such as QLabel) so that the user can perform the operation of cropping the screen. The user can perform image interception operations by other means such as mouse interaction, draw a rectangle, polygon or custom area of ​​any shape on the video screen, and use the drawn area as the cropping area. Based on the cropping area, the displayed video frame is subjected to corresponding image processing operations, such as cropping, masking, etc., to obtain the cropping screen required by the user. The cropping screen is displayed on the interface, or saved as a file for subsequent processing. A graphical toolbar can also be displayed on the interface, and the graphical toolbar includes graphical tools of multiple shapes, by which the cropping area is demarcated in the video screen.

[0055] S102: extracting traditional features and deep features from the image to be queried and the image material according to a preset scale, and generating fusion features corresponding to the traditional features and the deep features.

[0056] Optionally, when the image material is a video, before extracting the traditional features and the deep features, video frames are first extracted from the video, and each video frame is preprocessed, wherein the preprocessing includes at least one of graying, denoising, and smoothing.

[0057] Optionally, traditional features and deep features in the image to be queried and the image material are extracted according to a preset scale, including: constructing an image pyramid corresponding to the image to be queried and the image material based on the preset scale, the preset scale including at least one of object size and image resolution; extracting traditional features and deep features at each scale of the image pyramid.

[0058] Optionally, the traditional features include feature points extracted by methods such as SIFT and SURF. The descriptor of the traditional features is a vector describing a local area of ​​the feature points. The deep features can be extracted by a pre-trained convolutional neural network (CNN), and the extracted feature points are represented in the form of feature vectors.

[0059] In one embodiment, the image material is a vehicle image, and SIFT is used to extract feature points such as headlights, license plates, and window corners and their descriptors. The feature points at the front headlight position, the descriptor contains the gradient information of the area around the headlight. The feature points at the wheel edge, the descriptor contains information about the wheel texture change. The feature points at the window corners, the descriptor contains the gradient direction and amplitude of the window edge. The feature points at the door handle position, the descriptor contains the image features around the door handle, and the high-level semantic features of the body contour, headlight shape, and license plate area extracted by the pre-trained CNN.

[0060] Optionally, generating fused features corresponding to traditional features and deep features includes: obtaining current weights corresponding to the traditional features and the deep features, fusing the traditional features and the deep features based on the current weights to generate fused features, wherein the current weights are generated based on initial weights, current scene, current scale, and error feedback.

[0061] Optionally, the initial weights of traditional features and deep features can both be 0.5. The importance and similarity of deep features and traditional features can be determined according to the current scene, current scale and error feedback, and the current weights can be determined based on the importance and similarity. Then, the traditional features and deep features can be weightedly fused according to the current weights.

[0062] Optionally, the weights can be adjusted dynamically in real time through algorithms such as Adaptive Boosting and Meta-Learning. The error feedback information includes the accuracy and stability of the matching results. The error feedback information can be obtained based on the matching results of other objects in the same image material or the matching results of other image materials in the same scene. The above algorithm is trained based on historical data of scenes, scales and error feedback, and the trained algorithm is used for feature fusion. When performing feature fusion, the algorithm for adjusting the weights determines the current weights of traditional features and deep features based on error feedback, current scene, and current scale, and performs feature fusion based on the current weights.

[0063] Optionally, the performance of traditional features and deep features during image matching can be obtained, and error feedback can be generated based on the performance results. Specifically, if traditional features (such as SIFT) show higher accuracy in image query (it is easier to query the matching target image through traditional features) within several consecutive frames, error feedback is generated to increase the weight of traditional features and reduce the weight of deep features. Conversely, if the performance of deep features (such as the body contour identified by CNN) is better, error feedback is generated to increase the weight of deep features, thereby improving the adaptability of the algorithm.

[0064] In one embodiment, the image material is a video, and image pyramids of different scales are constructed based on the video frames of the image and video to be queried, and each pyramid level represents a different scale, such as the original image, the image reduced to half the original image, the image reduced to one third the original image, etc. In the feature extraction process, traditional features and deep features are extracted accordingly for each pyramid level. For example, feature points and descriptors of traditional features are extracted for the original image, feature points and descriptors of traditional features are extracted for the image reduced to half the original image, and feature points and descriptors of traditional features are extracted for the image reduced to one third the original image. The above method ensures that features can be detected and described at different scales. And the traditional features and deep features are fused at each scale to obtain fused features.

[0065] S103: Perform feature matching between the image to be queried and the image material based on the fusion features and the preset scale.

[0066] Optionally, feature matching is performed between the image to be queried and the image material based on the fusion features and the preset scale, including: matching the fusion features of the image to be queried with the fusion features of the image material at each scale corresponding to the preset scale; and obtaining the matching result between the image to be queried and the image material according to the matching result corresponding to each scale, wherein the fusion feature can be a vector.

[0067] Optionally, feature matching can be performed by means of distance metrics, including Euclidean distance, cosine similarity, etc. Determine the correspondence between the fused features in the image to be queried and the fused features in each image material (such as each video frame), generate feature pairs based on the correspondence, and calculate the distance between the two fused features in the feature pairs. The matching score at the current scale is obtained based on the average distance or weighted distance of all feature pairs at the current scale. The multi-scale matching method of the present application can process images or video streams with large scale changes, improve the accuracy and stability of matching, and can adapt to scenes with inconsistent image resolution or changes in target object size.

[0068] Optionally, the matching results of different scales may be preset with corresponding weights, and the matching results of the query image and the image material may include a matching score, which is obtained by weighting based on the weight. And the weights corresponding to the matching results of different scales can be adjusted according to the matching feedback (such as the stability and accuracy of the matching score, and at least one of the received parameter adjustment instructions). When the image material is a video, the matching result may also include the timestamp and position information of the matching frame determined based on the matching score (such as the video frame with a matching score greater than a preset preset), and the obtained matching result is saved for subsequent analysis and processing.

[0069] Optionally, when the image material is a video, in order to eliminate accidental matches, after obtaining the matching results of the query image and the image material according to the matching results corresponding to each scale, the following steps are performed: obtaining the matched feature points in each frame according to the matching results; wherein the traditional features and depth features in the video frame are represented in the form of feature points; tracking the motion trajectory of the feature points according to the timing information of the video, and filtering the feature points based on the motion trajectory. By using the timing information to perform a timing consistency analysis on the feature points, it is possible to better identify targets in dynamic scenes and obtain the motion trajectory of the targets, and eliminate feature points that affect the matching results.

[0070] In one embodiment, the timing information includes a timestamp or sequence number of each video frame, which is used to determine the temporal sequence and interval of each frame. The feature points in each video frame that match the image to be queried are obtained, and the motion trajectory of the feature points in the video is determined based on the timing information. The displacement or speed of the feature points is calculated based on the motion trajectory, and inconsistent or abnormal feature points (such as the displacement distance of adjacent frames is greater than a preset distance or the speed is greater than a preset speed) are filtered based on the displacement or speed, thereby improving the accuracy and stability of the matching results. Among them, the operation of eliminating inconsistent or abnormal feature points through timing information can also be performed after feature point extraction. After extracting the feature points of each video frame in the video, inconsistent or abnormal feature points in the video frame are extracted according to the timing information.

[0071] Optionally, the matching results of the image to be queried and the image material are obtained according to the matching results corresponding to each scale, including: obtaining matching data within a preset time period according to the matching results, the matching data including the quantity and quality of the matching results; adjusting the extraction parameters corresponding to the traditional features and the deep features based on the matching data, the extraction parameters including the feature extraction density.

[0072] Optionally, the number of matching results may be the number of matched feature points in the image material, and the quality may be a comparison result between the number of matched feature points and a predetermined threshold.

[0073] In one embodiment, the extraction parameter is sift->getEdgeThreshold(), and the sensitivity of feature point detection is changed by adjusting the extraction parameter. The matching data includes a matchCount parameter, which is used to indicate the number of matching results in the current frame or a certain period of time. Based on this number, the feature point matching density in the current scene can be inferred, so as to dynamically adjust the extraction parameters. When the matchCount parameter indicates a small number (such as less than a predetermined threshold), it means that the number of feature point matches is insufficient, and the amount of matching points extracted can be increased by increasing the density of feature extraction or adjusting the relevant threshold. When the matchCount parameter indicates a large number, it means that the number of feature point matches is too large, which may introduce noise or unnecessary feature points. The number of matched feature points can be dynamically and adaptively adjusted by reducing the density of feature extraction or adjusting the relevant threshold.

[0074] Optionally, each material in the image material (such as a photo or a video frame) is matched with the object to be queried through the above measures until the object to be queried is matched with all corresponding image materials.

[0075] S104: Displaying a target image in the image material that matches the query image according to the matching result.

[0076] Optionally, after obtaining a target image matching the query image, the target image may be output. When the target image is a video frame, the timestamp and position information of the video frame may be displayed. Furthermore, a folder for storing the target images may be generated, and the target images may be stored in the folder according to the time order between the target images, the order of similarity with the query image, or other ordering information.

[0077] Optionally, after the matched target image is displayed, the target image may be provided to the user for processing, and the target image may be exported, downloaded, content edited, sent, or subjected to other operations according to the user's processing operations.

[0078] The image query method of this application has the following advantages:

[0079] 1. Customize the shape of the captured video screen: Provide a graphic toolbar, allowing users to draw areas of any shape for capture to meet complex monitoring needs.

[0080] 2. Automatic material search and strong reminder: After importing image materials, you can automatically search and give strong reminders to improve search efficiency and accuracy.

[0081] 3. Flexible interception method: Compared with the traditional rectangular interception, it provides a more practical interception method (such as providing a graphic toolbar and determining the cropping area through human-computer interaction).

[0082] 4. Improve the efficiency of information acquisition: reduce the workload of manual frame-by-frame inspection and enhance the timeliness and reliability of information acquisition.

[0083] 5. Quickly import search materials: Users select the images and image materials to be imported and queried through the buttons on the interface.

[0084] 6. Automatically search for materials: Automatically search for target images that match the imported query images in all loaded video frames.

[0085] 7. Strong reminder: If a matching target image is found in a certain video frame, the target image will be highlighted or a prompt window will pop up to give a strong reminder, and the current target image can be automatically saved without manual inspection frame by frame.

[0086] 8. Combining multiple feature extraction techniques, multi-scale analysis, temporal consistency and adaptive parameter adjustment, the image matching of this application is improved in robustness, accuracy, real-time and various aspects.

[0087] In an optional embodiment, an electronic device is provided, such as Figure 5 As shown, Figure 5 The electronic device 4000 shown includes: a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0088] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0089] The bus 4002 may include a path to transmit information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0090] Memory 4003 can be ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or it can be EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disc, optical disk, digital versatile disc, Blu-ray disc, etc.), magnetic disk storage medium, other magnetic storage devices or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation here.

[0091] The memory 4003 is used to store the computer program for executing the embodiment of the present application, and the execution is controlled by the processor 4001. The processor 4001 is used to execute the computer program stored in the memory 4003 to implement the steps shown in the above method embodiment.

[0092] Among them, the electronic device can be any electronic product that can interact with an object, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.

[0093] The electronic device may also include a network device and / or an object device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud consisting of a large number of hosts or network servers based on cloud computing.

[0094] The network where the electronic device is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (Virtual Private Network, VPN), etc.

[0095] An embodiment of the present application provides a computer storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0096] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.

[0097] The terms "first", "second", "third", "fourth", "1", "2", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that shown or described in the drawings.

[0098] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the implementation order of these steps is not limited to the order indicated by the arrows. Unless clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage in these sub-steps or stages may also be executed at different times respectively. In different scenarios of execution time, the execution order of these sub-steps or stages may be flexibly configured according to demand, and the embodiment of the present application does not limit this.

[0099] The above is only an optional implementation method for some implementation scenarios of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the solution of the present application, other similar implementation methods based on the technical ideas of the present application are also within the protection scope of the embodiments of the present application.

Claims

1. An image query method, characterized in that: include: Acquire image material corresponding to the image to be queried, where the image material includes at least one of a video and a picture; Extracting traditional features and deep features from the query image and the image material according to a preset scale, and generating fusion features corresponding to the traditional features and the deep features; Performing feature matching between the image to be queried and the image material based on the fusion feature and a preset scale; The target image in the image material that matches the image to be queried is displayed according to the matching result.

2. The image query method according to claim 1, characterized in that: The step of obtaining the image material corresponding to the image to be queried includes: Acquire the image to be queried according to the acquired image interception operation; If it is determined that the preset conditions are currently met, the image material corresponding to the image to be queried is obtained according to the preset conditions, and the preset conditions include at least one of image material import, receiving a search instruction, and currently loading image material.

3. The image query method according to claim 2, characterized in that: The step of obtaining the image to be queried according to the obtained image interception operation includes: Loading a video file and playing the video screen corresponding to the video file; A cropping area in the video picture is determined according to an image capture operation of the captured object, and the video picture is cropped based on the cropping area to obtain the image to be queried.

4. The image query method according to claim 1, characterized in that: The extracting the traditional features and the deep features in the query image and the image material according to the preset scale includes: Constructing an image pyramid corresponding to the image to be queried and the image material based on the preset scale, wherein the preset scale includes at least one of an object size and an image resolution; The traditional features and the deep features are extracted at each scale of the image pyramid.

5. The image query method according to claim 1, characterized in that: The generating of the fusion features corresponding to the traditional features and the deep features includes: The current weights corresponding to the traditional features and the deep features are obtained, and the traditional features and the deep features are fused based on the current weights to generate the fused features, wherein the current weights are generated based on the initial weights, the current scene, the current scale, and error feedback.

6. The image query method according to claim 1, characterized in that: The performing feature matching between the image to be queried and the image material based on the fusion feature and the preset scale includes: Matching the fusion features of the image to be queried with the fusion features of the image material at each scale corresponding to the preset scale; The matching result between the image to be queried and the image material is obtained according to the matching result corresponding to each scale.

7. The image query method according to claim 6, characterized in that: The obtaining the matching result between the image to be queried and the image material according to the matching result corresponding to each scale includes: Acquire matching data within a preset time period according to the matching results, wherein the matching data includes the quantity and quality of the matching results; Extraction parameters corresponding to the traditional features and the deep features are adjusted based on the matching data, where the extraction parameters include feature extraction density.

8. The image query method according to claim 6, characterized in that: The image material is a video, and after obtaining the matching result between the image to be queried and the image material according to the matching result corresponding to each scale, the method includes: Acquire the matched feature points in each frame according to the matching results; The motion trajectory of the feature point is tracked according to the time sequence information of the video, and the feature point is selected based on the motion trajectory.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the method according to any one of claims 1 to 8.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Citation Information

Patent Citations

  • Image classification and searching method based on geographic position characteristics and overall situation vision characteristics

    CN104820718A

  • Image retrieval method and device

    CN107577687A

  • Fast video object segmentation method and device based on pixel and region feature matching

    CN112784750A

  • Image retrieval method based on multi-feature fusion

    CN114140657A

  • Image retrieval method and related equipment

    CN114168768A