Dimensionality reduction model training method, retrieval method and device based on artificial intelligence

Abstract

The present invention provides a dimensionality reduction model training method, retrieval method, device, electronic device and storage medium based on artificial intelligence; the method includes: obtaining a query vector corresponding to the multimedia information to be queried; determining the similarity between the query vector and multiple candidate vectors, and determining the candidate vector corresponding to the similarity that satisfies the similarity condition as a sampling vector, wherein the candidate vector corresponds to the candidate multimedia information; performing dimensionality reduction processing on the query vector through a dimensionality reduction model to obtain a dimensionality reduction query vector, and performing dimensionality reduction processing on the sampling vector through the dimensionality reduction model to obtain a dimensionality reduction sampling vector; constructing a loss function according to the dimensionality reduction query vector and the dimensionality reduction sampling vector; performing gradient descent processing on the loss function, and updating the weight parameters in the dimensionality reduction model according to the result of the gradient descent processing. Through the present invention, the dimensionality reduction effect of the dimensionality reduction model can be improved, and the accuracy of retrieval can be improved.

Description

Technical Field

[0001] The present invention relates to artificial intelligence technology, and in particular to an artificial intelligence-based dimensionality reduction model training method, retrieval method, device, electronic equipment and storage medium. Background Art

[0002] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. Machine Learning (ML) is an important branch of artificial intelligence, which specializes in how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance.

[0003] Vector dimensionality reduction is an application direction of machine learning. It converts high-dimensional vectors into low-dimensional vectors through dimensionality reduction models, and performs subsequent processing on low-dimensional vectors, thereby reducing the consumption of related storage and computing resources. However, after the dimensionality reduction of the vector, the relative position relationship between the vector and other vectors in the low-dimensional space usually changes, resulting in the retrieval results obtained in the high-dimensional space and the low-dimensional space in the application scenario of multimedia retrieval. The retrieval results may be inconsistent. In summary, in the solutions provided by the relevant technology, the dimensionality reduction effect of applying the dimensionality reduction model to the retrieval scenario is poor, and the accuracy of the retrieval results obtained based on the vector after dimensionality reduction is low. Summary of the invention

[0004] The embodiments of the present invention provide a dimensionality reduction model training method, retrieval method, device, electronic device and storage medium based on artificial intelligence, which can improve the dimensionality reduction effect of vector dimensionality reduction in retrieval scenarios and improve the accuracy of retrieval based on the reduced dimensionality vector.

[0005] The technical solution of the embodiment of the present invention is achieved as follows:

[0006] The embodiment of the present invention provides a dimensionality reduction model training method based on artificial intelligence, comprising:

[0007] Obtaining a query vector corresponding to the multimedia information to be queried;

[0008] Determine the similarity between the query vector and a plurality of candidate vectors, and determine the candidate vectors corresponding to the similarity that meet the similarity condition as sampling vectors, wherein the candidate vectors correspond to candidate multimedia information;

[0009] Performing dimensionality reduction processing on the query vector through a dimensionality reduction model to obtain a reduced-dimensionality query vector, and performing dimensionality reduction processing on the sampling vector through the dimensionality reduction model to obtain a reduced-dimensionality sampling vector;

[0010] Constructing a loss function according to the reduced-dimensionality query vector and the reduced-dimensionality sampling vector;

[0011] The loss function is subjected to a gradient descent process, and the weight parameters in the dimensionality reduction model are updated according to the result of the gradient descent process.

[0012] The embodiment of the present invention provides a retrieval method based on a dimensionality reduction model, comprising:

[0013] Obtaining a query vector corresponding to the multimedia information to be queried;

[0014] Performing dimension reduction processing on the query vector by using a dimension reduction model to obtain a reduced dimension query vector;

[0015] Acquire a plurality of dimensionality reduction candidate vectors, wherein the dimensionality reduction candidate vectors correspond to candidate multimedia information;

[0016] Determine the similarity between the reduced-dimensionality query vector and the plurality of reduced-dimensionality candidate vectors, and determine the reduced-dimensionality candidate vectors corresponding to the similarity that satisfies the similarity condition as the search vector;

[0017] The multimedia information corresponding to the search vector is determined as a search result.

[0018] The embodiment of the present invention provides a dimensionality reduction model training device based on artificial intelligence, comprising:

[0019] A vector acquisition module, used to acquire a query vector corresponding to the multimedia information to be queried;

[0020] A sampling module, used to determine the similarity between the query vector and a plurality of candidate vectors, and determine the candidate vectors corresponding to the similarity that meet the similarity condition as sampling vectors, wherein the candidate vectors correspond to candidate multimedia information;

[0021] A dimension reduction module, configured to perform dimension reduction processing on the query vector through a dimension reduction model to obtain a reduced dimension query vector, and perform dimension reduction processing on the sampling vector through the dimension reduction model to obtain a reduced dimension sampling vector;

[0022] A function construction module, used to construct a loss function according to the reduced-dimensionality query vector and the reduced-dimensionality sampling vector;

[0023] An updating module is used to perform gradient descent processing on the loss function and update the weight parameters in the dimensionality reduction model according to the result of the gradient descent processing.

[0024] The embodiment of the present invention provides a retrieval device based on a dimension reduction model, comprising:

[0025] A query vector acquisition module, used to acquire a query vector corresponding to the multimedia information to be queried;

[0026] A query vector dimension reduction module, used to perform dimension reduction processing on the query vector through a dimension reduction model to obtain a reduced-dimensional query vector;

[0027] A candidate vector acquisition module, used to acquire a plurality of reduced-dimensionality candidate vectors, wherein the reduced-dimensionality candidate vectors correspond to candidate multimedia information;

[0028] A retrieval vector determination module, used to determine the similarity between the dimension reduction query vector and the plurality of dimension reduction candidate vectors, and determine the dimension reduction candidate vector corresponding to the similarity that satisfies the similarity condition as the retrieval vector;

[0029] The result determination module is used to determine the multimedia information corresponding to the search vector as the search result.

[0030] An embodiment of the present invention provides an electronic device, including:

[0031] A memory for storing executable instructions;

[0032] The processor is used to implement the artificial intelligence-based dimensionality reduction model training method or the dimensionality reduction model-based retrieval method provided in an embodiment of the present invention when executing the executable instructions stored in the memory.

[0033] An embodiment of the present invention provides a storage medium storing executable instructions for causing a processor to execute instructions to implement an artificial intelligence-based dimensionality reduction model training method or a retrieval method based on a dimensionality reduction model provided in an embodiment of the present invention.

[0034] The embodiments of the present invention have the following beneficial effects:

[0035] The embodiment of the present invention reduces the dimensions of a query vector and similar sampling vectors through a dimension reduction model, and constructs a loss function based on the results of the dimension reduction, thereby performing gradient descent processing and updating the weight parameters in the dimension reduction model. When the dimension reduction of the vector is performed according to the updated dimension reduction model, the change in the relative position relationship between the vector and other vectors in the low-dimensional space is reduced, thereby improving the vector dimension reduction effect of the dimension reduction model in the retrieval scenario, and also improving the accuracy of similar retrieval based on the vector after the dimension reduction. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is an optional structural diagram of the dimensionality reduction model training system based on artificial intelligence provided by an embodiment of the present invention;

[0037] Figure 2A is an optional structural diagram of a server provided in an embodiment of the present invention;

[0038] Figure 2B is another optional structural diagram of a server provided in an embodiment of the present invention;

[0039] Figure 3 It is an optional structural schematic diagram of an artificial intelligence-based dimensionality reduction model training device provided in an embodiment of the present invention;

[0040] Figure 4A It is an optional flow chart of the dimensionality reduction model training method based on artificial intelligence provided by an embodiment of the present invention;

[0041] Figure 4B It is another optional flowchart of the artificial intelligence-based dimensionality reduction model training method provided in an embodiment of the present invention;

[0042] Figure 5 It is an optional flow chart of a retrieval method based on a dimension reduction model provided in an embodiment of the present invention;

[0043] Figure 6 It is another optional flowchart of the artificial intelligence-based dimensionality reduction model training method provided in an embodiment of the present invention;

[0044] Figure 7 is another optional flow chart of the retrieval method based on the dimension reduction model provided in an embodiment of the present invention;

[0045] Figure 8 It is an optional structural diagram of a dimensionality reduction model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present invention. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

[0047] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0048] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0050] Before further describing the embodiments of the present invention in detail, the nouns and terms involved in the embodiments of the present invention are described. The nouns and terms involved in the embodiments of the present invention are subject to the following interpretations.

[0051] 1) Multimedia information: in this article, it refers to information in the form of at least one media medium. For example, multimedia information can be audio, image or video.

[0052] 2) Loss function: A function used to estimate the difference between the output value of the model and the true value. The model is usually trained by finding the optimal solution of the loss function.

[0053] 3) Gradient descent processing: A way to find the optimal solution to the loss function by determining the gradient of the loss function with respect to the model parameters, thereby solving the extreme value along the direction of gradient descent.

[0054] 4) Neural network model: A complex network system formed by a large number of processing units (neurons) connected to each other. It is also a highly complex nonlinear dynamic learning system. In this article, it is specifically used to extract features from multimedia information and construct corresponding representation vectors.

[0055] The inventor found in the process of implementing the present invention that in the scheme provided by the related art, the vector dimensionality reduction method is mainly divided into two categories: machine learning and deep learning. The machine learning method is such as the principal component analysis (PCA, Principal Component Analysis) model. The PCA model reconstructs low-dimensional features on the basis of high-dimensional features by maximizing the covariance of samples in low-dimensional space, and completes the transformation of high-dimensional vectors to low-dimensional vectors; the deep learning method is such as the auto-encoder model. The auto-encoder model designs the network structure into an encoder (Encoder)-decoder (Decoder) structure, wherein the encoder maps the high-dimensional vector to a low dimension, and the decoder maps the low-dimensional vector to a high dimension. The auto-encoder model optimizes the mean square error between the high-dimensional vector reconstructed from the low-dimensional vector and the high-dimensional vector of the input model, and extracts the output of the encoder as the representation of the high-dimensional vector in the low dimension. However, after the vector is subjected to dimensionality reduction processing in the above manner, the relative position relationship between the vector and other vectors in the low-dimensional space usually changes, resulting in the retrieval result may be different from the retrieval result of the high-dimensional space when similar retrieval is performed according to the vector after dimensionality reduction. There is an error. In summary, in the vector dimensionality reduction solution provided by the related art, the accuracy of similarity retrieval based on the reduced dimensionality vector is reduced, and it is not suitable for the application scenario of multimedia retrieval.

[0056] The embodiments of the present invention provide a dimensionality reduction model training method, a retrieval method, an electronic device and a storage medium based on artificial intelligence, which can improve the dimensionality reduction effect in the retrieval scenario and improve the accuracy of retrieval based on the vector after dimensionality reduction. The exemplary application of the electronic device provided by the embodiments of the present invention is described below.

[0057] See also Figure 1 , Figure 1 It is an optional architectural diagram of an artificial intelligence-based dimensionality reduction model training system 100 provided in an embodiment of the present invention. To support an artificial intelligence-based dimensionality reduction model application, a terminal 400 (terminal 400-1 and terminal 400-2 are shown as examples) is connected to a server 200 via a network 300, and the server 200 is connected to a database 500, wherein the network 300 may be a wide area network or a local area network, or a combination of the two.

[0058] The server 200 is used to obtain a query vector corresponding to the multimedia information to be queried from the database 500, and obtain multiple candidate vectors; determine the similarity between the query vector and the multiple candidate vectors, and determine the candidate vector corresponding to the similarity that satisfies the similarity condition as a sampling vector, wherein the candidate vector corresponds to the candidate multimedia information; perform dimensionality reduction processing on the query vector through a dimensionality reduction model to obtain a reduced dimensionality query vector, and perform dimensionality reduction processing on the sampling vector through the dimensionality reduction model to obtain a reduced dimensionality sampling vector; construct a loss function based on the reduced dimensionality query vector and the reduced dimensionality sampling vector; perform gradient descent processing on the loss function, and update the weight parameters in the dimensionality reduction model according to the result of the gradient descent processing.

[0059] Terminal 400 is used to send the multimedia information to be queried to server 200 in response to the user's interactive operation; server 200 is also used to perform feature extraction processing on the multimedia information through a neural network model to obtain a query vector; perform dimensionality reduction processing on the query vector through an updated dimensionality reduction model to obtain a reduced dimensionality query vector; obtain multiple reduced dimensionality candidate vectors, wherein the reduced dimensionality candidate vectors correspond to candidate multimedia information; determine the similarity between the reduced dimensionality query vector and multiple reduced dimensionality candidate vectors, and determine the reduced dimensionality candidate vector corresponding to the similarity that meets the similarity condition as a retrieval vector; determine the multimedia information corresponding to the retrieval vector as a retrieval result, and send the retrieval result to terminal 400; terminal 400 is also used to display the retrieval results on a graphical interface 410 (graphical interface 410-1 and graphical interface 410-2 are shown as examples). Figure 1 , search result 1, search result 2, and search result 3 are exemplarily shown.

[0060] It is worth noting that in Figure 1 In the figure, it is taken that the server 200 obtains multimedia information from the terminal 400 and then performs a search to obtain the search results. In actual application scenarios, it is not limited to this. For example, the server 200 can send the neural network model and the updated dimensionality reduction model to the terminal 400. After the terminal 400 determines the multimedia information to be queried, it performs feature extraction and retrieval operations locally at the terminal 400 to obtain the search results.

[0061] The following is a description of exemplary applications of the electronic device provided by the embodiment of the present invention. The electronic device can be implemented as various types of terminal devices such as a laptop computer, a tablet computer, a desktop computer, a set-top box, a mobile device (e.g., a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable gaming device), and can also be implemented as a server. The following description is made by taking the electronic device as a server as an example.

[0062] See also Figure 2A , Figure 2A The server 200 provided in the embodiment of the present invention (for example, it may be Figure 1 A schematic diagram of the structure of the server 200 shown in FIG. Figure 2A The server 200 shown includes: at least one processor 210, a memory 250, at least one network interface 220 and a user interface 230. The various components in the server 200 are coupled together via a bus system 240. It is understood that the bus system 240 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 240 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 240 is not described in detail. Figure 2A Various buses are labeled as bus system 240 .

[0063] The processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0064] The user interface 230 includes one or more output devices 231 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 230 also includes one or more input devices 232, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0065] The memory 250 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 250 may optionally include one or more storage devices that are physically remote from the processor 210.

[0066] The memory 250 includes a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. The nonvolatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 250 described in the embodiment of the present invention is intended to include any suitable type of memory.

[0067] In some embodiments, memory 250 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0068] Operating system 251, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0069] A network communication module 252, used to reach other computing devices via one or more (wired or wireless) network interfaces 220, exemplary network interfaces 220 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB);

[0070] a presentation module 253 for enabling presentation of information via one or more output devices 231 (e.g., display screen, speaker, etc.) associated with the user interface 230 (e.g., a user interface for operating peripherals and displaying content and information);

[0071] The input processing module 254 is used to detect one or more user inputs or interactions from one of the one or more input devices 232 and translate the detected inputs or interactions.

[0072] In some embodiments, the artificial intelligence-based dimensionality reduction model training device provided in the embodiments of the present invention can be implemented in a software manner. Figure 2A An artificial intelligence-based dimensionality reduction model training device 2550 stored in a memory 250 is shown, which can be software in the form of programs and plug-ins, including the following software modules: a vector acquisition module 25501, a sampling module 25502, a dimensionality reduction module 25503, a function construction module 25504 and an update module 25505. These modules are logical and can therefore be arbitrarily combined or further split according to the functions implemented.

[0073] In some embodiments, the retrieval device based on the dimension reduction model provided in the embodiment of the present invention can also be implemented in software, see Figure 2B , Figure 2B The server 200 provided in the embodiment of the present invention (for example, it may be Figure 1 Another optional structural diagram of the server 200 shown in FIG. 1 is shown. Except for the retrieval device 2551 based on the dimension reduction model shown in FIG. 1 , the rest of the parts can refer to the above description of the retrieval device 2551. Figure 2AThe relevant description of the retrieval device 2551 based on the dimension reduction model stored in the memory 250 can be software in the form of programs and plug-ins, including the following software modules: a query vector acquisition module 25511, a query vector dimension reduction module 25512, a candidate vector acquisition module 25513, a retrieval vector determination module 25514 and a result determination module 25515. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented.

[0074] The functions of each module will be described below.

[0075] In other embodiments, the dimensionality reduction model training device based on artificial intelligence and the retrieval device based on the dimensionality reduction model provided in the embodiment of the present invention can be implemented in hardware. As an example, the dimensionality reduction model training device based on artificial intelligence provided in the embodiment of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the dimensionality reduction model training method based on artificial intelligence provided in the embodiment of the present invention; the retrieval device based on the dimensionality reduction model provided in the embodiment of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the retrieval method based on the dimensionality reduction model provided in the embodiment of the present invention. For example, the processor in the form of a hardware decoding processor can adopt one or more application specific integrated circuits (ASIC, Application Specific Integrated Circuit), DSP, programmable logic device (PLD, Programmable Logic Device), complex programmable logic device (CPLD, Complex Programmable Logic Device), field programmable gate array (FPGA, Field-Programmable Gate Array) or other electronic components.

[0076] The dimensionality reduction model training method based on artificial intelligence and the retrieval method based on dimensionality reduction model provided in the embodiments of the present invention can be executed by the above-mentioned server, or by a terminal device (for example, it can be Figure 1 The process may be executed by the terminal 400-1 and the terminal 400-2 shown in the figure, or may be executed jointly by the server and the terminal device.

[0077] The following will combine the exemplary applications and structures of the electronic devices recorded above to illustrate the process of implementing the artificial intelligence-based dimensionality reduction model training method in the electronic device by embedding the artificial intelligence-based dimensionality reduction model training device.

[0078] See also Figure 3 and Figure 4A , Figure 32 is a schematic diagram of the structure of an artificial intelligence-based dimensionality reduction model training device 2550 provided in an embodiment of the present invention, showing a processing flow of implementing model training through a series of modules. Figure 4A is a flow chart of a dimensionality reduction model training method based on artificial intelligence provided by an embodiment of the present invention, which will be combined with Figure 3 right Figure 4A The steps shown are explained.

[0079] In step 101, a query vector corresponding to multimedia information to be queried is obtained.

[0080] The query vector is extracted from the multimedia information to be queried, and is a representation vector of the multimedia information, wherein the multimedia information may be an audio segment, an image, or a video segment, etc., which is not limited in the embodiment of the present invention.

[0081] In some embodiments, the above-mentioned acquisition of the query vector corresponding to the multimedia information to be queried may be implemented in the following manner: performing feature extraction processing on the multimedia information to be queried through a neural network model to obtain the query vector;

[0082] Between any steps, the dimension reduction model training method further includes: performing feature extraction processing on candidate multimedia information through the neural network model to obtain candidate vectors.

[0083] As an example, see Figure 3 , the multimedia information to be queried and the candidate multimedia information can be obtained from the database, online business flow or other data sources. For the multimedia information to be queried, feature extraction processing is performed through the neural network model trained in the vector acquisition module 25501 to obtain a query vector, wherein the neural network model is such as the computer vision group (VGG, Visual Geometry Group) model or the inception_resnet_v2 model, etc., which is not limited. Similarly, for the candidate multimedia information, feature extraction processing is performed according to the same neural network model to obtain a candidate vector. The above method processes different multimedia information through a unified neural network model to obtain a query vector and a candidate vector, thereby avoiding potential differences caused by different models for feature extraction.

[0084] In step 102, the similarity between the query vector and a plurality of candidate vectors is determined, and the candidate vectors corresponding to the similarity that meet the similarity condition are determined as sampling vectors, wherein the candidate vectors correspond to candidate multimedia information.

[0085] As an example, see Figure 3In the sampling module 25502, the similarity between the query vector and each candidate vector is determined. The embodiment of the present invention does not limit the type of similarity. For example, the similarity can be cosine similarity, Euclidean distance, or Manhattan distance. For ease of understanding, the following text uses cosine similarity for unified description. The candidate vector corresponding to the cosine similarity that meets the similarity condition is determined as the sampling vector. The similarity condition is such as the cosine similarities that are ranked in the front after sorting multiple cosine similarities in descending order.

[0086] In step 103, the query vector is subjected to dimensionality reduction processing by a dimensionality reduction model to obtain a reduced-dimensionality query vector, and the sampling vector is subjected to dimensionality reduction processing by the dimensionality reduction model to obtain a reduced-dimensionality sampling vector.

[0087] As an example, see Figure 3 In the dimension reduction module 25503, the dimension reduction model is initialized, and the query vector is subjected to dimension reduction processing by the dimension reduction model to obtain a reduced dimension query vector, and the sampling vector is subjected to dimension reduction processing by the dimension reduction model to obtain a reduced dimension sampling vector. For example, the query vector before dimension reduction is 1024-dimensional, and after the query vector is subjected to dimension reduction processing by the dimension reduction model, the obtained reduced dimension query vector is 128-dimensional. It is worth noting that the embodiment of the present invention does not limit the dimension reduction model applied, for example, it can be the dimension reduction model of applying machine learning or the dimension reduction model of applying deep learning mentioned above.

[0088] In some embodiments, between any steps, the dimension reduction model training method further includes: performing dimension reduction processing on the candidate vector through the dimension reduction model to obtain a dimension reduction candidate vector, and storing a plurality of the dimension reduction candidate vectors.

[0089] The candidate vectors used for model training can also be used for subsequent model-based retrieval. Therefore, in the embodiment of the present invention, the dimension reduction model can be used to reduce the dimension of each candidate vector to obtain a reduced dimension candidate vector, and multiple reduced dimension candidate vectors can be stored. Since the reduced dimension candidate vector is smaller in size than the vector before the dimension reduction, the storage cost can be reduced by storing the reduced dimension candidate vector.

[0090] In step 104, a loss function is constructed according to the reduced-dimensionality query vector and the reduced-dimensionality sampling vector.

[0091] As an example, see Figure 3 In the function construction module 25504, a loss function is constructed according to the reduced dimension query vector and the reduced dimension sampling vector, and the value of the loss function reflects the degree of difference between the reduced dimension query vector and the reduced dimension sampling vector.

[0092] In step 105, the loss function is subjected to a gradient descent process, and the weight parameters in the dimensionality reduction model are updated according to the result of the gradient descent process.

[0093] As an example, see Figure 3 In the update module 25505, the loss function is subjected to gradient descent processing, specifically, the value of the loss function is minimized by continuously adjusting the weight parameters, that is, the optimal solution is obtained. When the set stop condition is reached, it is determined that the gradient descent processing is completed, and the original weight parameters in the dimensionality reduction model are updated according to the result of the gradient descent processing, that is, the weight parameters corresponding to the optimal solution of the loss function.

[0094] In some embodiments, between any steps, the dimensionality reduction model training method also includes: splitting the sample vector set into a training vector set and a verification vector set, wherein each vector in the sample vector set corresponds to a multimedia information; updating the weight parameters of the dimensionality reduction model according to the training vector set; verifying the updated dimensionality reduction model according to the verification vector set to obtain a verification result; when the verification result meets the verification condition, determining that the dimensionality reduction model update is completed.

[0095] When training a dimensionality reduction model, a sample vector set can be obtained, and each vector in the sample vector set corresponds to a multimedia information. The sample vector set is split into a training vector set and a verification vector set. The split ratio can be set according to the actual application scenario, such as randomly selecting 80% of the vectors in the sample vector set to form a training vector set, and the verification vector set is composed of the other 20% of the vectors. The training vector set is used to update the weight parameters in the dimensionality reduction model, and the verification vector set is used to verify the updated dimensionality reduction model to obtain a verification result. When the verification result meets the verification condition (such as when the verification result is the output value of the loss function, the verification condition is that the verification result is less than the preset threshold), it is determined that the update of the dimensionality reduction model is complete; when the verification result does not meet the verification condition, the weight parameters in the dimensionality reduction model are updated again according to the training vector set. By constructing vector sets with different functions as described above, the training effect of the dimensionality reduction model is guaranteed.

[0096] In some embodiments, the above-mentioned updating of the weight parameters of the dimensionality reduction model according to the training vector set can be achieved in the following manner: traversing the training vector set; determining the traversed vector as the query vector, and determining the vectors in the training vector set except the query vector as candidate vectors; updating the weight parameters of the dimensionality reduction model according to the query vector and the corresponding candidate vector until the traversal of the training vector set is completed.

[0097] When updating the weight parameters of the dimensionality reduction model according to the training vector set, the training vector set is traversed, and the traversed vector is determined as the query vector, and the vectors in the training vector set other than the query vector are determined as candidate vectors. For the determined query vector and multiple candidate vectors, the weight parameters of the dimensionality reduction model are updated in accordance with steps 102 to 105. Repeat the above process until the training vector set is traversed. Of course, in actual application scenarios, the number of traversals can also be set, such as setting the training vector set to be traversed 5 times. Through the above traversal method, the simplicity of model training is improved.

[0098] In some embodiments, the above-mentioned updating of the weight parameters of the dimensionality reduction model according to the query vector and the corresponding candidate vector can be achieved in the following manner: determining a plurality of query vectors whose number satisfies a batch processing parameter; and updating the weight parameters of the dimensionality reduction model according to the plurality of query vectors and the candidate vector corresponding to each of the query vectors.

[0099] The embodiments of the present invention do not limit the specific method of gradient descent processing. For example, a stochastic gradient descent algorithm, a batch gradient descent algorithm, or a small batch gradient descent algorithm may be used. One method is to obtain a preset batch processing parameter (batchsize), such as 256, and determine the 256 query vectors traversed. For each query vector, a sampling vector is determined from the corresponding candidate vector. Then, a loss function is constructed based on the 256 query vectors after dimensionality reduction and the corresponding sampling vectors, and a gradient descent process is performed as a whole, thereby updating the weight parameters of the dimensionality reduction model according to the results of the gradient descent process. By setting the batch processing parameters in the above manner, a balance between the efficiency and accuracy of the gradient descent process is achieved.

[0100] In some embodiments, the above-mentioned verification of the updated dimensionality reduction model according to the verification vector set to obtain a verification result can be achieved in the following manner: traversing the verification vector set; determining the traversed vector as the query vector, and determining the vectors in the verification vector set except the query vector as candidate vectors; performing dimensionality reduction processing on the query vector through the updated dimensionality reduction model to obtain a reduced dimensionality query vector, and performing dimensionality reduction processing on the candidate vector through the updated dimensionality reduction model to obtain a reduced dimensionality candidate vector; determining the similarity between the reduced dimensionality query vector and multiple reduced dimensionality candidate vectors, and determining the reduced dimensionality candidate vector corresponding to the similarity that meets the similarity condition as the output vector; determining the loss value of the loss function according to the reduced dimensionality query vector and the output vector; and determining the verification result of the dimensionality reduction model according to the loss values ​​corresponding to each of the traversed query vectors.

[0101] When verifying the updated dimensionality reduction model according to the verification vector set, the verification vector set is traversed, the traversed vector is determined as the query vector, and the vectors in the verification vector set other than the query vector are determined as candidate vectors. Then, the query vector and each candidate vector are subjected to dimensionality reduction processing through the updated dimensionality reduction model, and the cosine similarity between the reduced dimensionality query vector and each reduced dimensionality candidate vector is determined, and the reduced dimensionality candidate vector corresponding to the cosine similarity that meets the similarity condition is determined as the output vector.

[0102] According to the loss function constructed above, the dimension reduction query vector and the corresponding output vector are processed, and the value output by the loss function is determined as the loss value. For each query vector traversed, there is a corresponding loss value, and the loss values ​​are fused, for example, all the loss values ​​are averaged to obtain the verification result of the dimension reduction model. On this basis, multiple verification vector sets can be traversed, that is, multiple rounds of verification are performed, and the multiple verification results obtained from multiple rounds of verification are averaged to obtain the final verification result. In the above manner, the dimension reduction effect of the updated dimension reduction model can be effectively determined, which is convenient for further judgment on whether to continue training.

[0103] Through the embodiments of the invention Figure 4A It can be seen from the above exemplary implementation that the embodiment of the present invention trains the dimensionality reduction model through query vectors and similar sampling vectors, thereby improving the consistency of the retrieval results obtained in the high-dimensional space and the low-dimensional space, and improving the dimensionality reduction effect of the dimensionality reduction model in the multimedia retrieval scenario.

[0104] In some embodiments, see Figure 4B , Figure 4B is another optional flow chart of the artificial intelligence-based dimensionality reduction model training method provided in an embodiment of the present invention, Figure 4A The illustrated step 102 can be implemented through steps 201 to 202, which will be described in conjunction with each step.

[0105] In step 201, the similarity between the query vector and a plurality of candidate vectors is determined, and the plurality of candidate vectors are sorted according to the similarity to obtain a sorted vector set.

[0106] Here, after determining the similarity between the query vector and each candidate vector, the plurality of candidate vectors are sorted in descending order of similarity to obtain a sorted vector set.

[0107] In step 202, T first sampling vectors, R second sampling vectors and S third sampling vectors are determined in sequence in the sorted vector set; wherein T is an integer greater than 1, and R and S are both integers greater than 0.

[0108] Here, the similarity of any first sampling vector is greater than the similarity of any second sampling vector, and the similarity of any second sampling vector is greater than the similarity of any third sampling vector. The values ​​of T, R and S can be determined according to the actual application scenario, for example, T=5, R=20, S=38.

[0109] In some embodiments, the above-mentioned sequentially determining T first sampling vectors, R second sampling vectors and S third sampling vectors in the sorted vector set can be achieved in the following manner: the first T candidate vectors in the sorted vector set are all determined as first sampling vectors; the T first sampling vectors are removed from the sorted vector set, and R candidate vectors are determined as second sampling vectors from the first Z candidate vectors in the sorted vector set after the removal; the first Z candidate vectors are removed from the sorted vector set, and S candidate vectors are determined as third sampling vectors from the sorted vector set after the removal; wherein Z is an integer greater than or equal to R.

[0110] After sorting the candidate vectors in descending order of similarity to obtain a sorted vector set, the first T candidate vectors in the sorted vector set are determined as the first sampling vectors. Then, all the first sampling vectors are removed from the sorted vector set, and R candidate vectors are randomly selected from the first Z candidate vectors in the sorted vector set after removal as the second sampling vector, where Z ≥ R. For example, when R = 20, Z can be 45. Then, the first Z candidate vectors are continued to be removed from the sorted vector set, and S candidate vectors are randomly selected from the sorted vector set after removal as the third sampling vector. In the above manner, it is ensured that the similarity decreases from the first sampling vector to the second sampling vector, and then from the second sampling vector to the third sampling vector, which facilitates the subsequent accurate construction of the loss function.

[0111] Figure 4A The illustrated step 104 can be implemented through steps 203 to 206 , which will be described in conjunction with each step.

[0112] In step 203, a first loss term is constructed according to the reduced-dimensionality query vector, the t-1th first sampling vector after the reduced dimension, and the tth first sampling vector after the reduced dimension, until T-1 first loss terms are obtained.

[0113] Set an integer t greater than 1, and traverse t in the range of [2, T] starting from 2. For each t traversed, construct the first loss term according to the reduced-dimensionality query vector, the t-1th first sampling vector after dimensionality reduction, and the tth first sampling vector after dimensionality reduction, until the traversal is completed and T-1 first loss terms are obtained.

[0114] In some embodiments, the above-mentioned construction of the first loss term based on the reduced-dimensionality query vector, the t-1th first sampling vector after dimensionality reduction, and the tth first sampling vector after dimensionality reduction can be achieved in the following manner: determining a first similarity between the reduced-dimensionality query vector and the tth first sampling vector after dimensionality reduction, determining a second similarity between the reduced-dimensionality query vector and the t-1th first sampling vector after dimensionality reduction; constructing a first loss term based on the first similarity and the second similarity, the first loss term being used to constrain the difference between the first similarity and the second similarity to be minimized.

[0115] When constructing the first loss term, the first similarity between the reduced-dimensionality query vector and the t-th first sampling vector after dimensionality reduction is determined, the second similarity between the reduced-dimensionality query vector and the t-1th first sampling vector after dimensionality reduction is determined, and the first loss term is constructed based on the first similarity and the second similarity. Since the similarity between the query vector before dimensionality reduction and the t-1th first sampling vector is greater than the similarity between the query vector before dimensionality reduction and the t-th first sampling vector, the purpose of constructing the first loss term is to constrain the difference between the first similarity and the second similarity to be minimized, so that the similarity relationship between the low-dimensional space and the high-dimensional space is as consistent as possible, wherein the difference between the first similarity and the second similarity refers to the result of subtracting the second similarity from the first similarity. Similarly, the second loss term and the third loss term can also be constructed in the same way. By constructing the loss term, an effective training method for the dimensionality reduction model is provided.

[0116] In step 204, a second loss term is constructed according to the reduced-dimensionality query vector, the Tth first sampling vector after the reduced dimension, and the rth second sampling vector after the reduced dimension, until R second loss terms are obtained.

[0117] Set an integer r greater than 0, and start traversing r in the range of [1, R] with 1 as the starting point. For each traversed r, construct the second loss term according to the reduced-dimensionality query vector, the Tth first sampling vector after dimensionality reduction, and the rth second sampling vector after dimensionality reduction, until the traversal is completed and R second loss terms are obtained.

[0118] In step 205, a third loss term is constructed according to the reduced-dimensionality query vector, the r-th second sampling vector after the reduced-dimensionality, and the s-th third sampling vector after the reduced-dimensionality, until R×S third loss terms are obtained.

[0119] Set an integer s greater than 0, and traverse r in the range of [1, R] starting from 1, and traverse s in the range of [1, R] starting from 1. For each r and s traversed, construct the third loss term according to the reduced-dimensionality query vector, the r-th second sampling vector after dimensionality reduction, and the s-th third sampling vector after dimensionality reduction, until all r and all s are traversed to obtain R×S third loss terms.

[0120] In step 206, a loss function is constructed based on the T-1 first loss terms, the R second loss terms and the R×S third loss terms; wherein t is an integer greater than 1 and not exceeding T, r is an integer greater than 0 and not exceeding R, and s is an integer greater than 0 and not exceeding S.

[0121] Here, the T-1 first loss terms, R second loss terms, and R×S third loss terms are summed to obtain the loss function. Of course, the construction method of the loss function is not limited to summation, and can be adjusted according to the actual application scenario. After the loss function is constructed, the loss function is subjected to gradient descent processing to update the weight parameters in the dimensionality reduction model, so that the candidate vectors with high similarity to the query vector still have high similarity to the reduced dimensionality query vector after the dimensionality reduction processing by the dimensionality reduction model, that is, the change in the relative position relationship between the vector and other vectors in the low-dimensional space is minimized.

[0122] Through the embodiments of the invention Figure 4B It can be seen from the above exemplary implementation that the embodiment of the present invention determines the first sampling vector, the second sampling vector and the third sampling vector with decreasing similarity and constructs a loss function, thereby updating the weight parameters of the dimensionality reduction model, thereby improving the effectiveness and accuracy of the update.

[0123] The following will describe the process of implementing a retrieval method based on a dimensionality reduction model by embedding a retrieval device based on a dimensionality reduction model in an electronic device in combination with the exemplary applications and structures of the electronic device described above.

[0124] Figure 5 is a flowchart of a retrieval method based on a dimensionality reduction model provided by an embodiment of the present invention, which is combined with Figure 5 The steps shown are explained.

[0125] In step 301, a query vector corresponding to the multimedia information to be queried is obtained.

[0126] The multimedia information to be queried is obtained. For example, in response to a user selecting the multimedia information, the selected multimedia information is determined as the multimedia information to be queried. The multimedia information is subjected to feature extraction processing according to the same neural network model as above to obtain a query vector. Of course, the query vector can also be directly obtained.

[0127] In step 302, the query vector is subjected to dimensionality reduction processing by using a dimensionality reduction model to obtain a reduced-dimensionality query vector.

[0128] Here, the query vector is reduced in dimension by using the updated dimensionality reduction model to obtain a reduced-dimensional query vector. After the dimensionality reduction, the volume of the vector is reduced, and less computing resources are consumed in subsequent retrieval.

[0129] In step 303, a plurality of reduced-dimensionality candidate vectors are obtained, where the reduced-dimensionality candidate vectors correspond to candidate multimedia information.

[0130] In some embodiments, the acquisition of multiple reduced-dimensionality candidate vectors can be achieved in the following manner: feature extraction processing is performed on candidate multimedia information through a neural network model to obtain candidate vectors, and dimensionality reduction processing is performed on the candidate vectors through the dimensionality reduction model to obtain reduced-dimensionality candidate vectors; or, multiple stored reduced-dimensionality candidate vectors are acquired.

[0131] In the case of only candidate multimedia information, feature extraction is performed on the candidate multimedia information through the same neural network model as above to obtain a candidate vector, and the candidate vector is reduced in dimension through the updated dimensionality reduction model to obtain a reduced-dimensional candidate vector. In another case, the reduced-dimensional candidate vector has been predetermined and stored, so multiple stored reduced-dimensional candidate vectors are directly obtained here. The flexibility of obtaining reduced-dimensional candidate vectors is improved in the above manner.

[0132] In step 304, the similarity between the reduced-dimensionality query vector and the plurality of reduced-dimensionality candidate vectors is determined, and the reduced-dimensionality candidate vector corresponding to the similarity that satisfies the similarity condition is determined as a search vector.

[0133] Taking cosine similarity as an example, the similarity between the reduced dimension query vector and each reduced dimension candidate vector is determined, and the reduced dimension candidate vector corresponding to the cosine similarity that meets the similarity condition is determined as the retrieval vector. The similarity condition is such as several cosine similarities that are ranked in the front after sorting multiple cosine similarities in order from large to small.

[0134] In step 305, the multimedia information corresponding to the search vector is determined as a search result.

[0135] After the search vector is obtained, the multimedia information represented by the search vector is determined as the search result, and the search result is output, for example, the multimedia information as the search result is presented on the front end in a list form.

[0136] Through the embodiments of the invention Figure 5 It can be seen from the above exemplary implementation that the embodiment of the present invention performs dimensionality reduction processing on the query vector through the updated dimensionality reduction model, and queries the retrieval vector whose similarity meets the similarity condition, so as to determine the retrieval result. On the one hand, by reducing the dimensionality of the vector, the computing resources consumed by the vector processing are reduced, which can greatly improve the retrieval efficiency; on the other hand, the updated dimensionality reduction model has a better dimensionality reduction effect for the retrieval scenario. While reducing the dimensionality of the vector, it can ensure that the relative position relationship between the vector and other vectors in the low-dimensional space changes little, thereby improving the accuracy of retrieval based on the vector after dimensionality reduction.

[0137] The following describes an exemplary application of an embodiment of the present invention in a practical application scenario.

[0138] See also Figure 6 , Figure 6 It is another optional flowchart of the artificial intelligence-based dimensionality reduction model training method provided in an embodiment of the present invention.

[0139] exist Figure 6 In the example, the query vector is named v q , multiple candidate vectors constitute the candidate vector set V k First, for the candidate vector set V k Sampling is performed. Specifically, parameters T, R and S are set. T, R and S are all integers greater than 0. Determine the candidate vector set V k Each candidate vector v in k With the query vector v q The cosine similarity between them is calculated, and the candidate vector set V is sorted in descending order of cosine similarity. k Multiple candidate vectors v within k Sort.

[0140] In the sorted candidate vector set V k In the above example, the first T candidate vectors are used as the first sampling vector, that is, {v k1 ,v k2 ,...,v kT}, then, from the candidate vector set V k Determine R candidate vectors whose cosine similarity is lower than any first sampling vector as the second sampling vector, that is, {v k(T+1) ,v k(T+2) ,…,v k(T+R)}, then, from the candidate vector set Vk S candidate vectors whose cosine similarity is lower than any second sampling vector are determined as the third sampling vector, that is, {v k(T+R+1) ,v k(T+R+2) ,...,v k(T+R+S)}, the sampling is completed.

[0141] For ease of understanding, the sampling process is described by taking T = 5, R = 20, and S = 38 as an example. k In the above example, the first five candidate vectors with the largest cosine similarity are taken as the first sampling vector, that is, {v k1 ,v k2 ,...,v k5}, then, in the candidate vector set V k Remove all the first sampling vectors from the candidate vector set V after removal. k From the first 45 candidate vectors, 20 candidate vectors are randomly selected as the second sampling vectors, that is, {v k6 ,v k7 ,...,v k25}, then, remove the candidate vector set V k The first 45 candidate vectors in the candidate vector set V k Among the remaining candidate vectors, 38 candidate vectors are randomly selected as the third sampling vector, that is, {v k26 ,v k27 ,...,v k63}.

[0142] After sampling, the query vector is reduced in dimension through the dimensionality reduction model f(·) to obtain the reduced-dimensional query vector f(v q ), and each sampling vector is reduced in dimension by using the dimensionality reduction model f(·) to obtain the reduced dimensionality sampling vector, including f(v k1 )、f(v k2 ),... Then, the loss function is constructed based on the reduced-dimensionality query vector and the reduced-dimensionality sampling vector as follows:

[0143]

[0144] In the above formula, Loss(v q ) is the query vector v q The loss function, L(f(v q ),f(v k(t-1) ),f(v kt )) is the first loss term above, L(f(v q ),f(v kT ),f(v kr )) is the second loss term above, L(f(v q ),f(vkr ),f(v ks )) is the third loss term above. It is worth noting that in the formula of the loss function, v kt refers to the tth first sampling vector, v kr refers to the rth second sampling vector, v ks Refers to the sth third sampling vector.

[0145] Among them, L(α,ρ,n)=max(β(α,n)-β(α,ρ)+margin,0), and

[0146]

[0147] It is worth noting that max() in the above formula is the maximum value function, and the parameters α, ρ, n, x, and y have no actual meaning. Margin is a hyperparameter that can be set according to the actual application scenario. For example, when T = 5, R = 20, and S = 38, margin can be set to 0.2.

[0148] The training goal of the dimensionality reduction model is to solve the following optimization problem:

[0149]

[0150] Among them, i is the serial number of the query vector, and the loss function Loss(v q ) performs gradient descent processing to obtain the loss function Loss(v q ) and according to the loss function Loss(v q ) obtains the corresponding weight parameters when the optimal solution is obtained, and updates the original weight parameters in the dimensionality reduction model f(·).

[0151] In the actual training process, the optimizer can use the Adam optimizer, and the initial learning rate can be set to 0.001. Take the sample vector set as a 1024-dimensional vector set with 2429705 vectors as an example, where each vector is a representation vector obtained after feature extraction of multimedia information (such as video). 80% of the vectors in the sample vector set are randomly selected to form the training vector set, and the remaining 20% ​​of the vectors in the sample vector set constitute the verification vector set. Each vector in the training vector set can be used as a query vector. When a vector in the training vector set is determined as a query vector, the remaining vectors in the training vector set are determined as candidate vectors corresponding to the query vector. The training vector set is traversed, and in each training, 256 query vectors and the sampling vectors corresponding to each query vector are input into the dimensionality reduction model to calculate the loss, thereby updating the parameters of the model once, where the sampling vector corresponding to the query vector is sampled from multiple candidate vectors corresponding to the query vector, and the 256 above is the set batch size.

[0152] After the dimensionality reduction model is trained according to the training vector set, the model is verified. In the process of model verification, only the verification vector set is used. Similar to the training vector set, when a vector in the verification vector set is determined as a query vector, the remaining vectors in the verification vector set are determined as candidate vectors corresponding to the query vector. The verification vector set is traversed to obtain the verification result of the dimensionality reduction model, which is the output value of the loss function. When the verification result meets the verification condition, such as being less than the preset threshold, it is determined that the dimensionality reduction model update is completed; when the verification result does not meet the verification condition, the dimensionality reduction model is trained again according to the training vector set.

[0153] See also Figure 7 , Figure 7 FIG. 1 is another optional flow chart of the retrieval method based on the dimension reduction model provided in the embodiment of the present invention. After the dimension reduction model f(·) has been updated, the query vector v is obtained. q , and update the dimension reduction model f(·) to the query vector v q Perform dimensionality reduction processing to obtain the reduced-dimensional query vector f(v q ), and the candidate vector set V is updated by the dimension reduction model f(·) k The candidate vectors in the dimensionality reduction process are processed, and the obtained dimensionality reduction candidate vectors constitute a new dimensionality reduction candidate vector set f(V k ). Determine the reduced-dimensional query vector f(v q ) and the cosine similarity between each dimension reduction candidate vector, and the dimension reduction candidate vector corresponding to the cosine similarity that meets the similarity condition is determined as the search result. In the actual application scenario, the multimedia information corresponding to the dimension reduction candidate vector can also be determined as the search result. Figure 7In , an example of a similarity condition is the T cosine similarities with the highest values.

[0154] See also Figure 8 , Figure 8 It is an optional structural diagram of a dimensionality reduction model provided in an embodiment of the present invention, which is expressed by a formula:

[0155] f 0 =tanh(W·v)

[0156] v u-1 =tanh(B u-1 ·f u-1 )

[0157] d u-1 =tanh(R u-1 ·(vv u-1 ))

[0158] f u =f u-1 +d u-1

[0159] In the above formula, tanh() is the activation function, That is, v is the m-dimensional vector of the input dimensionality reduction model, That is, f u The n-dimensional vector output by the dimensionality reduction model after dimensionality reduction processing on v, where m>n, and m and n are both integers greater than zero. u is the time step of the dimensionality reduction model, which is a hyperparameter and can be set according to the actual application scenario, such as setting u=1. W, B u-1 and R u-1 are all weight parameters of the dimensionality reduction model, that is, the update object of model training, among which, It should be noted that in Figure 8 In the dimensionality reduction model shown in the figure, the bias term is omitted. Through the above dimensionality reduction model, The high-dimensional vector is mapped to the low-dimensional space, thereby reducing the related processing resource consumption.

[0160] Combination Figure 6 , Figure 7 and Figure 8, when the retrieval index does not change much, the query speed can be accelerated and the storage cost of the candidate vector can be reduced. After experimental verification, in the video retrieval scenario, the number of input videos to be queried is 78,781, and the video candidate set includes 250,000 videos. When the 1024-dimensional vector is used for retrieval, the retrieval accuracy is 99.8%, and the total number of retrieval recalls is 9016; when the vector after dimensionality reduction, that is, the 128-dimensional vector is used for retrieval, the accuracy is 98.23%, and the total number of retrieval recalls is 9444. Among them, the total number of retrieval recalls is the total number of correct candidate videos retrieved based on the input video. For example, videos A, B, C and D are four short videos split from the same long video. The video candidate set includes videos B, C, D and E. When the input video to be queried is A, and videos B, C and E are retrieved, since videos B and C are correct retrieval results, videos B and C are included in the total number of retrieval recalls. The experimental data is as follows:

[0161] method Accuracy Retrieve total recall 1024 dimensions (before dimensionality reduction) 99.8％ 9016 128 dimensions (after dimensionality reduction) 98.23％ 9444

[0162] With a small loss of accuracy (specifically 1.57%), the total number of retrieval recalls increased by 428 through the dimensionality reduction model, and the retrieval efficiency was greatly improved. At the same time, the videos in the video candidate set can also be stored in the form of reduced dimensionality vectors, greatly reducing the storage cost.

[0163] The following is a description of an exemplary structure of a dimensionality reduction model training device 2550 based on artificial intelligence provided by an embodiment of the present invention implemented as a software module. In some embodiments, Figure 2A As shown, the software modules in the artificial intelligence-based dimensionality reduction model training device 2550 stored in the memory 250 may include: a vector acquisition module 25501, used to obtain a query vector corresponding to the multimedia information to be queried; a sampling module 25502, used to determine the similarity between the query vector and multiple candidate vectors, and determine the candidate vector corresponding to the similarity that meets the similarity condition as the sampling vector, wherein the candidate vector corresponds to the candidate multimedia information; a dimensionality reduction module 25503, used to perform dimensionality reduction processing on the query vector through a dimensionality reduction model to obtain a reduced dimensionality query vector, and perform dimensionality reduction processing on the sampling vector through the dimensionality reduction model to obtain a reduced dimensionality sampling vector; a function construction module 25504, used to construct a loss function based on the reduced dimensionality query vector and the reduced dimensionality sampling vector; an update module 25505, used to perform gradient descent processing on the loss function, and update the weight parameters in the dimensionality reduction model according to the result of the gradient descent processing.

[0164] In some embodiments, the sampling module 25502 is also used to: sort the multiple candidate vectors according to the similarity to obtain a sorted vector set; determine T first sampling vectors, R second sampling vectors and S third sampling vectors in the sorted vector set in sequence; wherein T is an integer greater than 1, and R and S are both integers greater than 0.

[0165] In some embodiments, the function construction module 25504 is also used to: construct a first loss term according to the reduced dimension query vector, the t-1th first sampling vector after dimension reduction, and the tth first sampling vector after dimension reduction, until T-1 first loss terms are obtained; construct a second loss term according to the reduced dimension query vector, the Tth first sampling vector after dimension reduction, and the rth second sampling vector after dimension reduction, until R second loss terms are obtained; construct a third loss term according to the reduced dimension query vector, the rth second sampling vector after dimension reduction, and the sth third sampling vector after dimension reduction, until R×S third loss terms are obtained; construct a loss function according to the T-1 first loss terms, the R second loss terms, and the R×S third loss terms; wherein, t is an integer greater than 1 and not exceeding T, r is an integer greater than 0 and not exceeding R, and s is an integer greater than 0 and not exceeding S.

[0166] In some embodiments, the function construction module 25504 is also used to: determine a first similarity between the reduced-dimensionality query vector and the t-th first sampling vector after dimensionality reduction, and determine a second similarity between the reduced-dimensionality query vector and the t-1th first sampling vector after dimensionality reduction; construct a first loss term based on the first similarity and the second similarity, and the first loss term is used to constrain the difference between the first similarity and the second similarity to be minimized.

[0167] In some embodiments, the sampling module 25502 is also used to: determine the first T candidate vectors in the sorted vector set as the first sampling vectors; remove the T first sampling vectors in the sorted vector set, and determine R candidate vectors as the second sampling vectors from the first Z candidate vectors in the sorted vector set after the removal; remove the first Z candidate vectors in the sorted vector set, and determine S candidate vectors as the third sampling vector in the sorted vector set after the removal; wherein Z is an integer greater than or equal to R.

[0168] In some embodiments, the artificial intelligence-based dimensionality reduction model training device 2550 also includes: a splitting module, used to split the sample vector set into a training vector set and a verification vector set, wherein each vector in the sample vector set corresponds to a multimedia information; an updating submodule, used to update the weight parameters of the dimensionality reduction model according to the training vector set; a verification module, used to verify the updated dimensionality reduction model according to the verification vector set to obtain a verification result; and a confirmation module, used to determine that the dimensionality reduction model update is completed when the verification result meets the verification condition.

[0169] In some embodiments, the update submodule is also used to: traverse the training vector set; determine the traversed vector as the query vector, and determine the vectors in the training vector set except the query vector as candidate vectors; update the weight parameters of the dimensionality reduction model according to the query vector and the corresponding candidate vector until the training vector set is traversed.

[0170] In some embodiments, the updating submodule is further used to: determine a plurality of query vectors whose number satisfies a batch processing parameter; and update a weight parameter of the dimensionality reduction model according to the plurality of query vectors and the candidate vector corresponding to each of the query vectors.

[0171] In some embodiments, the verification module is also used to: traverse the verification vector set; determine the traversed vector as the query vector, and determine the vectors in the verification vector set except the query vector as candidate vectors; perform dimensionality reduction processing on the query vector through the updated dimensionality reduction model to obtain a reduced dimensionality query vector, and perform dimensionality reduction processing on the candidate vector through the updated dimensionality reduction model to obtain a reduced dimensionality candidate vector; determine the similarity between the reduced dimensionality query vector and multiple reduced dimensionality candidate vectors, and determine the reduced dimensionality candidate vector corresponding to the similarity that meets the similarity condition as the output vector; determine the loss value of the loss function based on the reduced dimensionality query vector and the output vector; determine the verification result of the dimensionality reduction model based on the loss values ​​corresponding to each of the traversed query vectors.

[0172] In some embodiments, the vector acquisition module 25501 is further used to: perform feature extraction processing on the multimedia information to be queried through a neural network model to obtain a query vector;

[0173] The artificial intelligence-based dimensionality reduction model training device 2550 also includes: a candidate vector extraction module, which is used to perform feature extraction processing on candidate multimedia information through the neural network model to obtain candidate vectors.

[0174] The following is a description of an exemplary structure of a retrieval device 2551 based on a dimension reduction model provided in an embodiment of the present invention implemented as a software module. In some embodiments, Figure 2BAs shown, the software modules stored in the retrieval device 2551 based on the dimension reduction model in the memory 250 may include: a query vector acquisition module 25511, used to obtain a query vector corresponding to the multimedia information to be queried; a query vector dimension reduction module 25512, used to perform dimension reduction processing on the query vector through the dimension reduction model to obtain a reduced dimension query vector; a candidate vector acquisition module 25513, used to obtain multiple reduced dimension candidate vectors, and the reduced dimension candidate vectors correspond to candidate multimedia information; a retrieval vector determination module 25514, used to determine the similarity between the reduced dimension query vector and the multiple reduced dimension candidate vectors, and determine the reduced dimension candidate vector corresponding to the similarity that meets the similarity condition as the retrieval vector; a result determination module 25515, used to determine the multimedia information corresponding to the retrieval vector as the retrieval result.

[0175] In some embodiments, the candidate vector acquisition module 25513 is also used to: perform feature extraction processing on the candidate multimedia information through a neural network model to obtain a candidate vector, and perform dimensionality reduction processing on the candidate vector through the dimensionality reduction model to obtain a reduced dimensionality candidate vector; or, obtain multiple stored dimensionality reduction candidate vectors.

[0176] An embodiment of the present invention provides a storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will be caused to execute the method provided by the embodiment of the present invention, for example, Figure 4A or Figure 4B The artificial intelligence-based dimensionality reduction model training method shown, or Figure 5 The retrieval method based on the dimension reduction model is shown.

[0177] In some embodiments, the storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories.

[0178] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0179] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0180] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0181] In summary, through the embodiments of the present invention, on the one hand, the dimensionality reduction effect of the dimensionality reduction model in the retrieval scenario is improved. After the vector dimensionality reduction is performed according to the dimensionality reduction model, the relative position relationship between the vector and other vectors in the low-dimensional space changes little, which ensures the consistency of the retrieval results of the high-dimensional space and the low-dimensional space, and improves the accuracy of retrieval based on the reduced dimensionality vector; and, by performing vector dimensionality reduction, subsequent processing of the lower dimensional vector is performed, which speeds up the retrieval efficiency and greatly reduces the storage cost.

[0182] The above description is only an embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement and improvement made within the spirit and scope of the present invention are included in the protection scope of the present invention.

Claims

1. A dimensionality reduction model training method based on artificial intelligence, characterized in that: include: Obtaining a query vector corresponding to the multimedia information to be queried; Determine the similarity between the query vector and a plurality of candidate vectors, sort the plurality of candidate vectors according to the similarity to obtain a sorted vector set, and sequentially determine T first sampling vectors, R second sampling vectors, and S third sampling vectors included in the sorted vector set, wherein the candidate vectors correspond to candidate multimedia information, T is an integer greater than 1, and R and S are both integers greater than 0; Performing dimension reduction processing on the query vector through a dimension reduction model to obtain a reduced dimension query vector, and performing dimension reduction processing on the sampling vector through the dimension reduction model to obtain a reduced dimension sampling vector; A first loss term is constructed according to the reduced-dimensionality query vector, the t-1th first sampling vector after dimensionality reduction, and the tth first sampling vector after dimensionality reduction, until T-1 first loss terms are obtained; a second loss term is constructed according to the reduced-dimensionality query vector, the Tth first sampling vector after dimensionality reduction, and the rth second sampling vector after dimensionality reduction, until R second loss terms are obtained; a third loss term is constructed according to the reduced-dimensionality query vector, the rth second sampling vector after dimensionality reduction, and the sth third sampling vector after dimensionality reduction, until R×S third loss terms are obtained; a loss function is constructed according to the T-1 first loss terms, the R second loss terms, and the R×S third loss terms, wherein t is an integer greater than 1 and not exceeding T, r is an integer greater than 0 and not exceeding R, and s is an integer greater than 0 and not exceeding S; The loss function is subjected to a gradient descent process, and the weight parameters in the dimensionality reduction model are updated according to the result of the gradient descent process.

2. The dimensionality reduction model training method according to claim 1, characterized in that: The constructing a first loss term according to the reduced-dimensionality query vector, the t-1th first sampling vector after the reduced-dimensionality, and the tth first sampling vector after the reduced-dimensionality includes: Determine a first similarity between the reduced-dimensionality query vector and the t-th first sampling vector after the dimensionality reduction, and determine a second similarity between the reduced-dimensionality query vector and the t-1-th first sampling vector after the dimensionality reduction; A first loss term is constructed according to the first similarity and the second similarity, and the first loss term is used to constrain the difference between the first similarity and the second similarity to be minimized.

3. The dimensionality reduction model training method according to claim 1, characterized in that: The step of sequentially determining T first sampling vectors, R second sampling vectors, and S third sampling vectors in the sorted vector set includes: Determine the first T candidate vectors in the sorted vector set as the first sampling vector; Removing the T first sampling vectors from the sorted vector set, and determining R candidate vectors from the first Z candidate vectors of the sorted vector set after the removal as second sampling vectors; Removing the first Z candidate vectors from the sorted vector set, and determining S candidate vectors from the sorted vector set after the removal as third sampling vectors; Wherein, Z is an integer greater than or equal to R.

4. The dimensionality reduction model training method according to claim 1, characterized in that: Also includes: Splitting the sample vector set into a training vector set and a verification vector set, wherein each vector in the sample vector set corresponds to a multimedia information; Updating weight parameters of the dimensionality reduction model according to the training vector set; Verifying the updated dimension reduction model according to the verification vector set to obtain a verification result; When the verification result satisfies the verification condition, it is determined that the updating of the dimensionality reduction model is completed.

5. The dimensionality reduction model training method according to claim 4, characterized in that: The step of updating the weight parameters of the dimensionality reduction model according to the training vector set includes: Traversing the training vector set; Determine the traversed vector as the query vector, and determine the vectors in the training vector set except the query vector as candidate vectors; The weight parameters of the dimensionality reduction model are updated according to the query vector and the corresponding candidate vectors until the training vector set is traversed completely.

6. The dimensionality reduction model training method according to claim 5, characterized in that: The updating of the weight parameters of the dimensionality reduction model according to the query vector and the corresponding candidate vector includes: Determine a number of query vectors satisfying a batch processing parameter; The weight parameters of the dimensionality reduction model are updated according to the multiple query vectors and the candidate vector corresponding to each query vector.

7. The dimensionality reduction model training method according to claim 4, characterized in that: The step of verifying the updated dimension reduction model according to the verification vector set to obtain a verification result includes: Traversing the verification vector set; Determine the traversed vector as the query vector, and determine the vectors in the verification vector set except the query vector as candidate vectors; Performing dimension reduction processing on the query vector through the updated dimension reduction model to obtain a reduced dimension query vector, and performing dimension reduction processing on the candidate vector through the updated dimension reduction model to obtain a reduced dimension candidate vector; Determine the similarity between the reduced-dimensionality query vector and the plurality of reduced-dimensionality candidate vectors, and determine the reduced-dimensionality candidate vectors corresponding to the similarity that satisfies the similarity condition as output vectors; Determining a loss value of the loss function according to the reduced-dimensionality query vector and the output vector; The verification result of the dimensionality reduction model is determined according to the loss value corresponding to each of the traversed query vectors.

8. The dimensionality reduction model training method according to any one of claims 1 to 7, characterized in that: The obtaining of a query vector corresponding to the multimedia information to be queried includes: The neural network model is used to extract features from multimedia information to be queried, and a query vector is obtained; The dimensionality reduction model training method further includes: The neural network model is used to perform feature extraction processing on candidate multimedia information to obtain candidate vectors.

9. A retrieval method based on the dimension reduction model according to any one of claims 1 to 8, characterized in that: include: Obtaining a query vector corresponding to the multimedia information to be queried; Performing dimension reduction processing on the query vector by using a dimension reduction model to obtain a reduced dimension query vector; Acquire a plurality of dimensionality reduction candidate vectors, wherein the dimensionality reduction candidate vectors correspond to candidate multimedia information; Determine the similarity between the reduced-dimensionality query vector and the plurality of reduced-dimensionality candidate vectors, and determine the reduced-dimensionality candidate vectors corresponding to the similarity that satisfies the similarity condition as the search vector; The multimedia information corresponding to the search vector is determined as a search result.

10. A dimensionality reduction model training device based on artificial intelligence, characterized in that: include: A vector acquisition module, used to acquire a query vector corresponding to the multimedia information to be queried; a sampling module, configured to determine a similarity between the query vector and a plurality of candidate vectors, sort the plurality of candidate vectors according to the similarity to obtain a sorted vector set, and sequentially determine T first sampling vectors, R second sampling vectors, and S third sampling vectors included in the sorted vector set, wherein the candidate vectors correspond to candidate multimedia information, T is an integer greater than 1, and R and S are both integers greater than 0; A dimension reduction module, configured to perform dimension reduction processing on the query vector through a dimension reduction model to obtain a reduced dimension query vector, and perform dimension reduction processing on the sampling vector through the dimension reduction model to obtain a reduced dimension sampling vector; A function construction module, used to construct a first loss term according to the reduced dimension query vector, the t-1th first sampling vector after dimension reduction, and the tth first sampling vector after dimension reduction, until T-1 first loss terms are obtained; construct a second loss term according to the reduced dimension query vector, the Tth first sampling vector after dimension reduction, and the rth second sampling vector after dimension reduction, until R second loss terms are obtained; construct a third loss term according to the reduced dimension query vector, the rth second sampling vector after dimension reduction, and the sth third sampling vector after dimension reduction, until R×S third loss terms are obtained; construct a loss function according to the T-1 first loss terms, the R second loss terms, and the R×S third loss terms, wherein t is an integer greater than 1 and not exceeding T, r is an integer greater than 0 and not exceeding R, and s is an integer greater than 0 and not exceeding S; An updating module is used to perform gradient descent processing on the loss function and update the weight parameters in the dimensionality reduction model according to the result of the gradient descent processing.

11. A retrieval device based on a dimensionality reduction model, characterized in that: include: A query vector acquisition module, used to acquire a query vector corresponding to the multimedia information to be queried; A query vector dimension reduction module, used to perform dimension reduction processing on the query vector through a dimension reduction model to obtain a reduced-dimensional query vector; A candidate vector acquisition module, used to acquire a plurality of reduced-dimensionality candidate vectors, wherein the reduced-dimensionality candidate vectors correspond to candidate multimedia information; A retrieval vector determination module, used to determine the similarity between the dimension reduction query vector and the plurality of dimension reduction candidate vectors, and determine the dimension reduction candidate vector corresponding to the similarity that satisfies the similarity condition as the retrieval vector; The result determination module is used to determine the multimedia information corresponding to the search vector as the search result.

12. An electronic device, characterized in that: include: A memory for storing executable instructions; The processor is used to implement the artificial intelligence-based dimensionality reduction model training method described in any one of claims 1 to 8, or the dimensionality reduction model-based retrieval method described in claim 9 when executing the executable instructions stored in the memory.

13. A storage medium, characterized in that: Executable instructions are stored, which are used to cause the processor to execute and implement the artificial intelligence-based dimensionality reduction model training method described in any one of claims 1 to 8, or the dimensionality reduction model-based retrieval method described in claim 9.

Images