Data retrieval method and device based on artificial intelligence, computer equipment and medium

Abstract

The invention belongs to the technical field of artificial intelligence, and relates to an artificial intelligence-based data retrieval method and device, computer equipment and a storage medium, and the method comprises the following steps: obtaining a retrieval text corresponding to a target video; performing text preprocessing on the retrieval text to obtain a text embedding representation; performing image feature extraction on the target video to obtain image features; calling a cross attention layer in a stacked pointer network to perform cross attention processing on the text embedded representation and the image features to obtain feature representations of multiple frames; performing prediction processing on the feature representations of the plurality of frames to obtain a fragment initial position, and determining a fragment initial interval based on the fragment initial position; performing offset prediction processing on the fragment starting interval based on the text embedding representation to obtain a prediction result; and generating a target fragment position corresponding to the retrieval text based on the prediction result. In addition, the target fragment location may be stored in a blockchain. According to the invention, the processing efficiency and accuracy of video data retrieval are improved based on the use of the stacked pointer network.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence development technology, financial technology, and medical health, and in particular to artificial intelligence-based data retrieval methods, devices, computer equipment, and storage media. Background Art

[0002] In today's world of rapid development of information technology, video, as an important multimedia information carrier, has been applied to all aspects of social life, especially in the financial and medical fields. Video materials play an irreplaceable role in investigation and evidence collection, education and training, customer service, public health publicity, academic exchanges and research promotion, commercial marketing and brand building. In order to effectively manage and utilize these massive video resources, text-video moment retrieval technology has emerged. Text-video moment retrieval technology is an advanced retrieval method that combines the advantages of text retrieval and video retrieval. It allows users to quickly and accurately locate video clips or specific moments containing relevant content from a huge video library by entering keywords, phrases or questions. This technology greatly improves the retrieval efficiency of video information, allowing users to obtain the required key information in a short time, which is of great significance to improving work efficiency and decision-making accuracy in the financial field.

[0003] However, despite the many advantages of text-video moment retrieval technology, it still faces many challenges in practical applications. At present, most mainstream text-video moment retrieval methods are based on sliding window or anchor-base strategies. These methods perform well when dealing with simple or single video clip retrieval tasks, but their performance is often greatly reduced when facing complex segment moment retrieval such as nested segments. Specifically, the sliding window method sets a fixed-length window to slide on the video stream and performs text matching on the video content in each window, but this method cannot accurately capture the dynamic changes in the video and has limited recognition ability for nested segments. The anchor-base method relies on pre-set anchor points to locate key segments in the video. Although it improves the accuracy of retrieval to a certain extent, it is still not adaptable enough for complex and changeable video content, and it is easy to cause omissions or misjudgments of retrieval results. Therefore, the existing text-video moment retrieval technology generally has the problems of low retrieval efficiency and difficulty in ensuring accuracy when dealing with complex segment moment retrieval tasks. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a data retrieval method, device, computer equipment and storage medium based on artificial intelligence to solve the technical problems that the existing text-video moment retrieval technology generally has low retrieval efficiency and difficulty in ensuring accuracy when processing complex segment moment retrieval tasks.

[0005] In order to solve the above technical problems, the present application embodiment provides a data retrieval method based on artificial intelligence, which adopts the following technical solutions:

[0006] Obtaining the search text corresponding to the target video input by the user;

[0007] Performing text preprocessing on the search text to obtain a corresponding text embedding representation;

[0008] Extracting image features from the target video to obtain corresponding image features;

[0009] Calling a preset stacked pointer network, and performing cross-attention processing on the text embedding representation and the image features based on a cross-attention layer in the stacked pointer network to obtain feature representations of corresponding multiple frames;

[0010] Performing prediction processing on the feature representations of the plurality of frames to obtain corresponding segment start positions, and determining corresponding segment start intervals based on the segment start positions;

[0011] Based on the text embedding representation, bias prediction processing is performed on the starting interval of the segment to obtain a corresponding prediction result;

[0012] A target segment position corresponding to the search text is generated based on the prediction result.

[0013] Furthermore, the step of performing text preprocessing on the search text to obtain a corresponding text embedding representation specifically includes:

[0014] Segmenting the search text based on a preset word segmenter to obtain corresponding word segmentation results;

[0015] Call the preset word embedding model;

[0016] Based on the word embedding model, the word segmentation result is converted to obtain a corresponding embedding representation;

[0017] The embedding representation is used as the text embedding representation.

[0018] Furthermore, the step of extracting image features from the target video to obtain corresponding image features specifically includes:

[0019] Performing frame sampling on the target video to obtain corresponding image frames;

[0020] Call the preset image encoder;

[0021] Extracting features from the image frame based on the image encoder to obtain a corresponding feature vector;

[0022] The feature vector is used as the image feature.

[0023] Furthermore, the stacked pointer network further includes a linear layer; the step of performing prediction processing on the feature representations of the plurality of frames to obtain corresponding fragment start positions, and determining corresponding fragment start intervals based on the fragment start positions specifically includes:

[0024] Based on the linear layer, a prediction process is performed on the starting position of the feature representation of each of the frames to obtain a probability value of each of the frames;

[0025] Obtaining a preset probability threshold;

[0026] Filtering out a designated frame whose probability value is greater than the probability threshold from each of the frames, and using the designated frame as the starting position of the segment;

[0027] A corresponding segment start interval is determined based on the segment start position.

[0028] Furthermore, the stacked pointer network further includes a fully connected layer; the step of performing bias prediction processing on the starting interval of the segment based on the text embedding representation to obtain a corresponding prediction result specifically includes:

[0029] Extracting first feature representations of all target frames contained in the starting interval of the segment;

[0030] Performing cross-attention processing on the first feature representation and the text embedding representation to obtain a corresponding second feature representation;

[0031] Aggregating the second feature representation based on a preset pooling strategy to obtain a corresponding representation vector;

[0032] Mapping the representation vector to the bias prediction task based on the fully connected layer to obtain a corresponding prediction confidence and a prediction bias value;

[0033] The prediction confidence and the prediction bias value are used as the prediction result corresponding to the starting interval of the segment.

[0034] Furthermore, the step of generating a target segment position corresponding to the search text based on the prediction result specifically includes:

[0035] Filtering out the first segment starting interval whose prediction confidence is greater than a preset confidence threshold from all the segment starting intervals;

[0036] Obtaining a specified prediction bias value corresponding to a starting interval of the first segment;

[0037] Based on the specified prediction offset value, a corresponding position fine-tuning process is performed on the first segment start interval to obtain a fine-tuned second segment start interval;

[0038] Performing conversion processing on the second segment start interval based on a preset target conversion form to obtain a processed third segment start interval;

[0039] A target segment position corresponding to the search text is generated based on the third segment start interval.

[0040] Furthermore, after the step of generating the target segment position corresponding to the search text based on the prediction result, the method further includes:

[0041] Generate corresponding search results based on the target fragment position;

[0042] Obtaining a data push method corresponding to the user;

[0043] Based on the data pushing method, the search result is pushed to the user.

[0044] In order to solve the above technical problems, the embodiment of the present application further provides a data retrieval device based on artificial intelligence, which adopts the following technical solution:

[0045] A first acquisition module is used to acquire a search text input by a user corresponding to a target video;

[0046] A preprocessing module, used for performing text preprocessing on the search text to obtain a corresponding text embedding representation;

[0047] An extraction module, used to extract image features from the target video to obtain corresponding image features;

[0048] A second processing module is used to call a preset stacked pointer network, and perform cross-attention processing on the text embedding representation and the image feature based on a cross-attention layer in the stacked pointer network to obtain feature representations of corresponding multiple frames;

[0049] A first prediction module, configured to perform prediction processing on the feature representations of the plurality of frames to obtain corresponding segment start positions, and determine corresponding segment start intervals based on the segment start positions;

[0050] A second prediction module, configured to perform bias prediction processing on the segment start interval based on the text embedding representation to obtain a corresponding prediction result;

[0051] The first generating module is used to generate a target segment position corresponding to the search text based on the prediction result.

[0052] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0053] Obtaining the search text corresponding to the target video input by the user;

[0054] Performing text preprocessing on the search text to obtain a corresponding text embedding representation;

[0055] Extracting image features from the target video to obtain corresponding image features;

[0056] Calling a preset stacked pointer network, and performing cross-attention processing on the text embedding representation and the image features based on a cross-attention layer in the stacked pointer network to obtain feature representations of corresponding multiple frames;

[0057] Performing prediction processing on the feature representations of the plurality of frames to obtain corresponding segment start positions, and determining corresponding segment start intervals based on the segment start positions;

[0058] Based on the text embedding representation, bias prediction processing is performed on the starting interval of the segment to obtain a corresponding prediction result;

[0059] A target segment position corresponding to the search text is generated based on the prediction result.

[0060] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0061] Obtaining the search text corresponding to the target video input by the user;

[0062] Performing text preprocessing on the search text to obtain a corresponding text embedding representation;

[0063] Extracting image features from the target video to obtain corresponding image features;

[0064] Calling a preset stacked pointer network, and performing cross-attention processing on the text embedding representation and the image features based on a cross-attention layer in the stacked pointer network to obtain feature representations of corresponding multiple frames;

[0065] Performing prediction processing on the feature representations of the plurality of frames to obtain corresponding segment start positions, and determining corresponding segment start intervals based on the segment start positions;

[0066] Based on the text embedding representation, bias prediction processing is performed on the starting interval of the segment to obtain a corresponding prediction result;

[0067] A target segment position corresponding to the search text is generated based on the prediction result.

[0068] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0069] The present application first obtains a search text input by a user that corresponds to a target video; then performs text preprocessing on the search text to obtain a corresponding text embedding representation; and performs image feature extraction on the target video to obtain corresponding image features; then calls a preset cascade pointer network, and performs cross-attention processing on the text embedding representation and the image features based on the cross-attention layer in the cascade pointer network to obtain feature representations of the corresponding multiple frames; subsequently, performs prediction processing on the feature representations of the multiple frames to obtain a corresponding fragment starting position, and determines a corresponding fragment starting interval based on the fragment starting position; further performs biased prediction processing on the fragment starting interval based on the text embedding representation to obtain a corresponding prediction result; finally, generates a target fragment position corresponding to the search text based on the prediction result. After obtaining the search text input by the user corresponding to the target video, the present application will perform text preprocessing on the search text to obtain a text embedding representation, and perform image feature extraction on the target video to obtain image features, and then perform cross-attention processing on the text embedding representation and the image features based on the use of a cascaded pointer network to obtain feature representations of multiple frames, and then by processing the feature representations of multiple frames, first preliminarily predict the starting position of the fragment, and then accurately perform bias prediction, and then based on the obtained prediction results, it is possible to quickly and accurately generate the target fragment position corresponding to the input search text, thereby improving the processing efficiency of video data retrieval and ensuring the data accuracy of the retrieved target fragment position. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0071] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0072] Figure 2 A flowchart of an embodiment of an artificial intelligence-based data retrieval method according to the present application;

[0073] Figure 3 is a schematic structural diagram of an embodiment of an artificial intelligence-based data retrieval device according to the present application;

[0074] Figure 4It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0075] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0076] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0077] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0078] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0079] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0080] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to the laptop computer 1011, tablet computer 1012 or mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0081] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0082] It should be noted that the artificial intelligence-based data retrieval method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the artificial intelligence-based data retrieval device is generally set in the server / terminal device.

[0083] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0084] Continue to refer Figure 2 , showing a flow chart of an embodiment of an artificial intelligence-based data retrieval method according to the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted. The artificial intelligence-based data retrieval method provided in the embodiment of the present application can be applied to any scenario where data retrieval is required, and the artificial intelligence-based data retrieval method can be applied to products in these scenarios, for example, data retrieval in the financial insurance field. The artificial intelligence-based data retrieval method comprises the following steps:

[0085] Step S201, obtaining a search text input by a user corresponding to a target video.

[0086] In this embodiment, the electronic device (e.g. Figure 1The server / terminal device shown in the figure) can obtain the search text through a wired connection or a wireless connection. It should be pointed out that the above-mentioned wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or developed in the future. The executor of the present application may be a data retrieval system, or simply referred to as a system. There is a certain semantic match between the description in the above-mentioned search text input by the user and the image content in the above-mentioned target video. For example, if the search text is "Introducing the precautions for purchasing auto insurance", then a certain clip in the video should contain a picture of an insurance personnel explaining the precautions for purchasing auto insurance.

[0087] Step S202: perform text preprocessing on the search text to obtain a corresponding text embedding representation.

[0088] In this embodiment, the above-mentioned specific implementation process of performing text preprocessing on the search text to obtain the corresponding text embedding representation will be further described in detail in the subsequent specific embodiments of this application, and will not be elaborated in detail here.

[0089] Step S203: extract image features from the target video to obtain corresponding image features.

[0090] In the present embodiment, the above-mentioned specific implementation process of extracting image features from the target video to obtain corresponding image features will be described in further detail in subsequent specific embodiments of the present application, and will not be elaborated on here.

[0091] Step S204, calling a preset cascade pointer network, and performing cross-attention processing on the text embedding representation and the image features based on the cross-attention layer in the cascade pointer network to obtain feature representations of corresponding multiple frames.

[0092] In this embodiment, the stacked pointer network includes at least a cross attention layer, a linear layer, and a fully connected layer. The cross attention layer can be specifically a Transformer model with multiple encoder layers. By taking the above image features as the input sequence of the Transformer model, each feature vector corresponds to a sequence element. Then convert the text embedding representation into a format that can be processed by the Transformer model (such as adding position encoding, packaging into a sequence, etc.). After that, in each layer of the Transformer, a cross attention mechanism is used, where Q (Query) is the image feature, K (Key) and V (Value) are text embedding representations. This is usually achieved through a multi-head attention mechanism. After multiple layers of cross attention processing, an interactive feature representation is obtained. These representations still maintain the dimension of (total_frame, dim). Among them, in the process of image feature extraction for the target video, the feature vectors of all frames can be combined into a feature matrix with a dimension of (total_frame, dim), where total_frame is the total number of frames and dim is the dimension of the feature vector.

[0093] Step S205 , predicting the feature representations of the multiple frames to obtain corresponding segment start positions, and determining corresponding segment start intervals based on the segment start positions.

[0094] In the present embodiment, the above-mentioned specific implementation process of predicting the feature representations of the multiple frames to obtain the corresponding fragment starting positions, and determining the corresponding fragment starting intervals based on the fragment starting positions, will be described in further detail in subsequent specific embodiments of the present application and will not be elaborated on here.

[0095] Step S206: performing bias prediction processing on the segment start interval based on the text embedding representation to obtain a corresponding prediction result.

[0096] In this embodiment, the above-mentioned specific implementation process of performing biased prediction processing on the starting interval of the fragment based on the text embedding representation to obtain the corresponding prediction result will be further described in detail in the subsequent specific embodiments of this application, and will not be elaborated in detail here.

[0097] Step S207: generating a target segment position corresponding to the search text based on the prediction result.

[0098] In this embodiment, the specific implementation process of generating the target segment position corresponding to the search text based on the prediction result will be described in further detail in subsequent specific embodiments of the present application, and will not be elaborated on here.

[0099] The present application first obtains a search text input by a user that corresponds to a target video; then performs text preprocessing on the search text to obtain a corresponding text embedding representation; and performs image feature extraction on the target video to obtain corresponding image features; then calls a preset cascade pointer network, and performs cross-attention processing on the text embedding representation and the image features based on the cross-attention layer in the cascade pointer network to obtain feature representations of the corresponding multiple frames; subsequently, performs prediction processing on the feature representations of the multiple frames to obtain a corresponding fragment starting position, and determines a corresponding fragment starting interval based on the fragment starting position; further performs biased prediction processing on the fragment starting interval based on the text embedding representation to obtain a corresponding prediction result; finally, generates a target fragment position corresponding to the search text based on the prediction result. After obtaining the search text input by the user corresponding to the target video, the present application will perform text preprocessing on the search text to obtain a text embedding representation, and perform image feature extraction on the target video to obtain image features, and then perform cross-attention processing on the text embedding representation and the image features based on the use of a cascaded pointer network to obtain feature representations of multiple frames, and then by processing the feature representations of multiple frames, first preliminarily predict the starting position of the fragment, and then accurately perform bias prediction, and then based on the obtained prediction results, it is possible to quickly and accurately generate the target fragment position corresponding to the input search text, thereby improving the processing efficiency of video data retrieval and ensuring the data accuracy of the retrieved target fragment position.

[0100] In some optional implementations, step S202 includes the following steps:

[0101] The search text is segmented based on a preset word segmenter to obtain corresponding word segmentation results.

[0102] In this embodiment, the search text may be segmented by using a word segmenter with a word segmentation function, and the search text may be split into tokens, ie, word segmentation results.

[0103] Call the preset word embedding model.

[0104] In this embodiment, there is no specific limitation on the selection of the above-mentioned word embedding model. For example, models such as Word2Vec and GloVe can be used.

[0105] The word segmentation result is converted based on the word embedding model to obtain a corresponding embedding representation.

[0106] In this embodiment, the word segmentation results (tokens after word segmentation) are converted into embedded representations by using a word embedding model. These embedded representations are usually vectors of fixed dimensions.

[0107] The embedding representation is used as the text embedding representation.

[0108] The present application performs word segmentation on the search text based on a preset word segmenter to obtain a corresponding word segmentation result; then calls a preset word embedding model; then converts the word segmentation result based on the word embedding model to obtain a corresponding embedding representation; and subsequently uses the embedding representation as the text embedding representation. The present application performs word segmentation on the search text based on the use of a word segmenter to obtain a word segmentation result, and then converts the word segmentation result based on the use of a word embedding model, thereby achieving efficient and accurate text preprocessing of the search text, and improving the accuracy and standardization of the generated text embedding representation.

[0109] In some optional implementations of this embodiment, step S203 includes the following steps:

[0110] Perform frame sampling on the target video to obtain corresponding image frames.

[0111] In this embodiment, the target video may be read by using a video processing library, and frame sampling may be performed on the target video at a specified sampling rate (eg, two frames per second) to obtain corresponding image frames.

[0112] Invoke the preset image encoder.

[0113] In this embodiment, there is no specific limitation on the selection of the above-mentioned image encoder, for example, models such as CLIP and BLIP2 may be used.

[0114] Based on the image encoder, feature extraction is performed on the image frame to obtain a corresponding feature vector.

[0115] In this embodiment, an image encoder is used to perform feature extraction on the sampled image frame to convert the image frame into a feature vector of a fixed dimension.

[0116] The feature vector is used as the image feature.

[0117] The present application obtains the corresponding image frame by performing frame sampling on the target video; then calls a preset image encoder; then extracts features from the image frame based on the image encoder to obtain a corresponding feature vector; and subsequently uses the feature vector as the image feature. The present application obtains the corresponding image frame by performing frame sampling on the target video, and then extracts features from the image frame based on the use of an image encoder, thereby achieving efficient and accurate image feature extraction processing for the target video, thereby improving the accuracy and standardization of the generated image features.

[0118] In some optional implementations, the stacked pointer network further includes a linear layer; step S205 includes the following steps:

[0119] Based on the linear layer, a prediction process is performed on the starting position of the feature representation of each of the frames to obtain a probability value of each of the frames.

[0120] In this embodiment, the feature representation of each frame output by the cross attention layer is mapped to a binary classification task using a linear layer, and an activation function (such as sigmoid) is applied to convert the output into a probability value, indicating the probability of whether the frame is the starting position of the fragment.

[0121] Get the preset probability threshold.

[0122] In this embodiment, there is no specific limitation on the value of the above probability threshold, which can be set according to actual business needs, for example, it can be set to 0.5.

[0123] A designated frame having a probability value greater than the probability threshold is selected from each of the frames, and the designated frame is used as the starting position of the segment.

[0124] In this embodiment, by traversing the probability values ​​of all frames, the frames whose probability values ​​exceed the above probability threshold (such as 0.5) are marked as possible fragment starting positions. For each feature of a frame, it is predicted whether it is the starting position of the fragment to be retrieved to ensure that all fragments are predicted.

[0125] A corresponding segment start interval is determined based on the segment start position.

[0126] In this embodiment, all possible segment start intervals are determined based on the obtained start position. These segment start intervals may overlap or be non-adjacent.

[0127] The present application predicts the starting position of the feature representation of each frame based on the linear layer to obtain the probability value of each frame; then obtains a preset probability threshold; then filters out the specified frame with a probability value greater than the probability threshold from each frame, and uses the specified frame as the starting position of the fragment; and then determines the corresponding fragment starting interval based on the fragment starting position. The present application predicts the starting position of the feature representation of each frame based on the use of a linear layer in a stacked pointer network to obtain the probability value of each frame; then filters out the specified frame with a probability value greater than the preset probability threshold from each frame, and uses the specified frame as the starting position of the fragment, and then determines the corresponding fragment starting interval based on the fragment starting position, so that the feature representation of multiple frames can be quickly and accurately predicted and processed and the corresponding fragment starting position and fragment starting interval can be obtained, which is beneficial for the subsequent accurate determination of the target fragment position corresponding to the retrieved text based on the obtained fragment starting interval.

[0128] In some optional implementations, the stacked pointer network further includes a fully connected layer; step S206 includes the following steps:

[0129] Extract the first feature representation of all target frames contained in the starting interval of the segment.

[0130] In this embodiment, since the predicted segment start interval is only preliminary, a more accurate offset for the predicted start position prediction is required. For each segment start interval that is initially determined, the first feature representation of all frames within the segment start interval is extracted.

[0131] The first feature representation and the text embedding representation are cross-attention processed to obtain a corresponding second feature representation.

[0132] In this embodiment, the feature representations are cross-attended again with the text embedding representation to further fuse the text and image information, and obtain the feature representation after cross-attention, i.e., the second feature representation mentioned above. The cross-attention processing can be achieved by adding another encoder layer to the cross-attention layer or using another independent cross-attention layer.

[0133] The second feature representation is aggregated based on a preset pooling strategy to obtain a corresponding representation vector.

[0134] In this embodiment, a representation vector of a fixed dimension can be obtained by aggregating the second feature representation after cross attention using a pooling method (such as average pooling).

[0135] Based on the fully connected layer, the representation vector is mapped to the bias prediction task to obtain a corresponding prediction confidence and a prediction bias value.

[0136] In this embodiment, the above representation vector is mapped to the bias prediction task by using a fully connected layer. This usually includes an output for predicting confidence and an output for predicting left and right biases. Among them, the confidence output can be normalized using a sigmoid activation function, while the bias output can be directly output as a value between 0 and 1 (through an appropriate activation function or normalization process).

[0137] The prediction confidence and the prediction bias value are used as the prediction result corresponding to the starting interval of the segment.

[0138] The present application extracts the first feature representation of all target frames contained in the starting interval of the fragment; then the first feature representation and the text embedding representation are cross-attended to obtain the corresponding second feature representation; then the second feature representation is aggregated based on a preset pooling strategy to obtain a corresponding representation vector; subsequently, the representation vector is mapped to the bias prediction task based on the fully connected layer to obtain the corresponding prediction confidence and prediction bias value; finally, the prediction confidence and the prediction bias value are used as the prediction result corresponding to the starting interval of the fragment. The present application extracts the first feature representation of all target frames contained in the starting interval of the fragment, and the first feature representation and the text embedding representation are cross-attended to obtain the second feature representation, then the second feature representation is aggregated based on the use of the pooling strategy to obtain the representation vector, and then the representation vector is mapped to the bias prediction task based on the use of the fully connected layer in the stacked pointer network to obtain the corresponding prediction confidence and prediction bias value, thereby realizing the rapid and accurate completion of the bias prediction processing of the starting interval of the fragment using the text embedding representation, improving the accuracy of the generated prediction result, and facilitating the subsequent accurate determination of the target fragment position corresponding to the retrieved text based on the obtained prediction result.

[0139] In some optional implementations of this embodiment, step S207 includes the following steps:

[0140] A first segment starting interval whose prediction confidence is greater than a preset confidence threshold is selected from all the segment starting intervals.

[0141] In this embodiment, there is no specific limitation on the value of the above confidence threshold, which can be set according to actual business needs. Specifically, each segment start interval can be screened according to the obtained prediction confidence. High confidence intervals (segment start intervals with prediction confidence greater than the confidence threshold) are more likely to be retained as the final segment position, while low confidence intervals (segment start intervals with prediction confidence less than the confidence threshold) may be discarded or further processed.

[0142] Obtain a specified prediction bias value corresponding to the starting interval of the first segment.

[0143] In this embodiment, the specified prediction offset value refers to a prediction offset value that has a matching relationship with the start interval of the first segment.

[0144] Based on the specified prediction offset value, a corresponding position fine-tuning process is performed on the first segment start interval to obtain a fine-tuned second segment start interval.

[0145] In this embodiment, the start position and end position of the first segment start interval after screening can be fine-tuned by using the obtained specified prediction offset value. The prediction offset value represents the offset of the start position of the first segment start interval relative to the actual start position, so the adjusted start position can be obtained by adding the prediction offset value to the frame number of the first segment start interval. The end position can be obtained by adding a fixed interval length (or dynamically determined according to other information) to the start position, thereby obtaining the fine-tuned second segment start interval.

[0146] The second segment start interval is converted based on a preset target conversion form to obtain a processed third segment start interval.

[0147] In this embodiment, the start position and end position of the fine-tuned second segment start interval can be expressed in the form of frame number or timestamp as the final segment position, that is, the processed third segment start interval. These positions can be used in subsequent video processing or analysis tasks.

[0148] A target segment position corresponding to the search text is generated based on the third segment start interval.

[0149] In this embodiment, the obtained third segment starting interval can be directly used as the target segment position corresponding to the search text.

[0150] The present application selects the first segment start interval whose prediction confidence is greater than a preset confidence threshold from all the segment start intervals; then obtains the specified prediction bias value corresponding to the first segment start interval; then performs corresponding position fine-tuning processing on the first segment start interval based on the specified prediction bias value to obtain the fine-tuned second segment start interval; subsequently converts the second segment start interval based on a preset target conversion form to obtain the processed third segment start interval; finally, generates the target segment position corresponding to the search text based on the third segment start interval. The present application selects the first segment start interval whose prediction confidence is greater than a preset confidence threshold from all the segment start intervals; then performs position fine-tuning processing on the first segment start interval based on the specified prediction bias value corresponding to the first segment start interval to obtain the fine-tuned second segment start interval; subsequently converts the second segment start interval based on the target conversion form, thereby realizing the rapid and accurate generation of the final target segment position corresponding to the search text, ensuring the accuracy of the obtained target segment position, and realizing efficient and accurate video content retrieval.

[0151] In some optional implementations of this embodiment, after step S207, the electronic device may further perform the following steps:

[0152] Generate corresponding retrieval results based on the target segment position.

[0153] In this embodiment, the target segment position can be input into a preset search result template to generate a corresponding search result. The search template is a template file pre-built according to the actual search result output requirements.

[0154] Obtain a data push method corresponding to the user.

[0155] In this embodiment, the above-mentioned data push method may refer to the push method preferred by the user, such as email push, interface display, short-line push, etc.

[0156] Based on the data pushing method, the search result is pushed to the user.

[0157] In this embodiment, the acquired data push method may be used to push the generated search results to the user.

[0158] The present application generates corresponding search results based on the target segment position; then obtains the data push method corresponding to the user; and subsequently pushes the search results to the user based on the data push method. After generating the target segment position corresponding to the search text based on the prediction result, the present application will also automatically and intelligently generate corresponding search results based on the target segment position, and then push the search results to the user according to the data push method corresponding to the user, so that the user can receive the feedback search results in a timely manner, thereby improving the push intelligence of the search results and improving the user's experience.

[0159] In some optional implementations, the user information obtained is subject to the user's consent and complies with relevant laws and policies.

[0160] In addition, the non-Company software tools or components that appear in the embodiments of the present application are merely examples and do not represent actual use.

[0161] In addition, the video moment retrieval based on the cascade pointer network proposed in this application unifies the moment retrieval task with the relationship extraction task. The only difference is that the input of the relationship extraction is text, and the input of the moment retrieval is a frame. Decomposing the positioning problem into preliminary positioning and secondary prediction greatly reduces the learning difficulty. That is, the retrieval task is divided into two steps. The first step is to preliminarily predict the starting position of the clip, and then accurately predict the offset of the position, making the task learning easier.

[0162] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0163] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned target fragment location, the above-mentioned target fragment location can also be stored in a node of a blockchain.

[0164] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0165] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0166] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0167] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0168] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0169] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a data retrieval device based on artificial intelligence, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0170] like Figure 3As shown, the artificial intelligence-based data retrieval device 300 described in this embodiment includes: a first acquisition module 301, a preprocessing module 302, an extraction module 303, a second processing module 304, a first prediction module 305, a second prediction module 306 and a first generation module 307. Among them:

[0171] The first acquisition module 301 is used to acquire a search text corresponding to a target video input by a user;

[0172] A preprocessing module 302 is used to perform text preprocessing on the search text to obtain a corresponding text embedding representation;

[0173] An extraction module 303 is used to extract image features from the target video to obtain corresponding image features;

[0174] A second processing module 304 is used to call a preset stacked pointer network, and perform cross-attention processing on the text embedding representation and the image feature based on a cross-attention layer in the stacked pointer network to obtain feature representations of corresponding multiple frames;

[0175] A first prediction module 305 is used to perform prediction processing on the feature representations of the multiple frames to obtain corresponding segment start positions, and determine corresponding segment start intervals based on the segment start positions;

[0176] A second prediction module 306, configured to perform bias prediction processing on the segment start interval based on the text embedding representation to obtain a corresponding prediction result;

[0177] The first generating module 307 is used to generate a target segment position corresponding to the search text based on the prediction result.

[0178] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based data retrieval method in the aforementioned implementation mode, and will not be repeated here.

[0179] In some optional implementations of this embodiment, the preprocessing module 302 includes:

[0180] A word segmentation submodule, used to segment the search text based on a preset word segmenter to obtain corresponding word segmentation results;

[0181] The first calling submodule is used to call a preset word embedding model;

[0182] A first conversion submodule, configured to convert the word segmentation result based on the word embedding model to obtain a corresponding embedding representation;

[0183] The first determination submodule is used to use the embedding representation as the text embedding representation.

[0184] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based data retrieval method in the aforementioned implementation mode, and will not be repeated here.

[0185] In some optional implementations of this embodiment, the extraction module 303 includes:

[0186] A sampling submodule, used for performing frame sampling on the target video to obtain corresponding image frames;

[0187] The second calling submodule is used to call a preset image encoder;

[0188] An extraction submodule, used for performing feature extraction on the image frame based on the image encoder to obtain a corresponding feature vector;

[0189] The second determining submodule is used to use the feature vector as the image feature.

[0190] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based data retrieval method in the aforementioned implementation mode, and will not be repeated here.

[0191] In some optional implementations of this embodiment, the stacked pointer network further includes a linear layer; the first prediction module 305 includes:

[0192] A prediction submodule, used for performing prediction processing on the starting position of the feature representation of each of the frames based on the linear layer to obtain a probability value of each of the frames;

[0193] A first acquisition submodule is used to acquire a preset probability threshold;

[0194] A first screening submodule, configured to screen out a designated frame having a probability value greater than the probability threshold from each of the frames, and use the designated frame as the starting position of the segment;

[0195] The third determining submodule is used to determine a corresponding segment starting interval based on the segment starting position.

[0196] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based data retrieval method in the aforementioned implementation mode, and will not be repeated here.

[0197] In some optional implementations of this embodiment, the stacked pointer network further includes a fully connected layer; the second prediction module 306 includes:

[0198] A second acquisition submodule is used to extract first feature representations of all target frames contained in the starting interval of the segment;

[0199] a processing submodule, configured to perform cross-attention processing on the first feature representation and the text embedding representation to obtain a corresponding second feature representation;

[0200] An aggregation submodule, used for performing aggregation processing on the second feature representation based on a preset pooling strategy to obtain a corresponding representation vector;

[0201] A mapping submodule, used to map the representation vector to a bias prediction task based on the fully connected layer to obtain a corresponding prediction confidence and a prediction bias value;

[0202] The fourth determination submodule is used to use the prediction confidence and the prediction offset value as the prediction result corresponding to the segment start interval.

[0203] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based data retrieval method in the aforementioned implementation mode, and will not be repeated here.

[0204] In some optional implementations of this embodiment, the first generating module 307 includes:

[0205] A second screening submodule, configured to screen out, from all the segment start intervals, a first segment start interval having a prediction confidence greater than a preset confidence threshold;

[0206] A third acquisition submodule, used to acquire a specified prediction bias value corresponding to the starting interval of the first segment;

[0207] A fine-tuning submodule, configured to perform corresponding position fine-tuning processing on the first segment start interval based on the specified prediction offset value to obtain a fine-tuned second segment start interval;

[0208] A second conversion submodule, configured to convert the second segment start interval based on a preset target conversion form to obtain a processed third segment start interval;

[0209] A generating submodule is used to generate a target segment position corresponding to the search text based on the third segment starting interval.

[0210] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based data retrieval method in the aforementioned implementation mode, and will not be repeated here.

[0211] In some optional implementations of this embodiment, the artificial intelligence-based data retrieval device further includes:

[0212] A second generating module, used to generate corresponding retrieval results based on the target segment position;

[0213] A second acquisition module is used to acquire a data push mode corresponding to the user;

[0214] A push module is used to push the search results to the user based on the data push method.

[0215] In this embodiment, the operations performed by the above modules or units correspond one-to-one to the steps of the artificial intelligence-based data retrieval method in the aforementioned implementation mode, and will not be repeated here.

[0216] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0217] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components 41-43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0218] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0219] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of a data retrieval method based on artificial intelligence, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0220] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run computer-readable instructions stored in the memory 41 or process data, such as computer-readable instructions for running the artificial intelligence-based data retrieval method.

[0221] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0222] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0223] In an embodiment of the present application, after obtaining the search text corresponding to the target video input by the user, the search text will be preprocessed to obtain a text embedding representation, and the target video will be image-featured extracted to obtain image features. Then, based on the use of a cascaded pointer network, the text embedding representation and the image features will be cross-attention processed to obtain feature representations of multiple frames. Then, by processing the feature representations of multiple frames, a preliminary prediction of the starting position of the fragment is first made, and then an accurate bias prediction is made. Based on the obtained prediction results, the target fragment position corresponding to the input search text can be quickly and accurately generated, thereby improving the processing efficiency of video data retrieval and ensuring the data accuracy of the retrieved target fragment position.

[0224] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the artificial intelligence-based data retrieval method as described above.

[0225] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0226] In an embodiment of the present application, after obtaining the search text corresponding to the target video input by the user, the search text will be preprocessed to obtain a text embedding representation, and the target video will be image-featured extracted to obtain image features. Then, based on the use of a cascaded pointer network, the text embedding representation and the image features will be cross-attention processed to obtain feature representations of multiple frames. Then, by processing the feature representations of multiple frames, a preliminary prediction of the starting position of the fragment is first made, and then an accurate bias prediction is made. Based on the obtained prediction results, the target fragment position corresponding to the input search text can be quickly and accurately generated, thereby improving the processing efficiency of video data retrieval and ensuring the data accuracy of the retrieved target fragment position.

[0227] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0228] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A data retrieval method based on artificial intelligence, characterized in that: The steps include: Obtaining the search text corresponding to the target video input by the user; Performing text preprocessing on the search text to obtain a corresponding text embedding representation; Extracting image features from the target video to obtain corresponding image features; Calling a preset stacked pointer network, and performing cross-attention processing on the text embedding representation and the image features based on a cross-attention layer in the stacked pointer network to obtain feature representations of corresponding multiple frames; Performing prediction processing on the feature representations of the plurality of frames to obtain corresponding segment start positions, and determining corresponding segment start intervals based on the segment start positions; Based on the text embedding representation, bias prediction processing is performed on the starting interval of the segment to obtain a corresponding prediction result; A target segment position corresponding to the search text is generated based on the prediction result.

2. The artificial intelligence-based data retrieval method according to claim 1, characterized in that: The step of performing text preprocessing on the search text to obtain a corresponding text embedding representation specifically includes: Segmenting the search text based on a preset word segmenter to obtain corresponding word segmentation results; Call the preset word embedding model; Based on the word embedding model, the word segmentation result is converted to obtain a corresponding embedding representation; The embedding representation is used as the text embedding representation.

3. The artificial intelligence-based data retrieval method according to claim 1, characterized in that: The step of extracting image features from the target video to obtain corresponding image features specifically includes: Performing frame sampling on the target video to obtain corresponding image frames; Call the preset image encoder; Extracting features from the image frame based on the image encoder to obtain a corresponding feature vector; The feature vector is used as the image feature.

4. The artificial intelligence-based data retrieval method according to claim 1, characterized in that: The stacked pointer network further includes a linear layer; the step of performing prediction processing on the feature representations of the multiple frames to obtain corresponding fragment start positions, and determining corresponding fragment start intervals based on the fragment start positions specifically includes: Based on the linear layer, a prediction process is performed on the starting position of the feature representation of each of the frames to obtain a probability value of each of the frames; Obtaining a preset probability threshold; Filtering out a designated frame whose probability value is greater than the probability threshold from each of the frames, and using the designated frame as the starting position of the segment; A corresponding segment start interval is determined based on the segment start position.

5. The artificial intelligence-based data retrieval method according to claim 1, characterized in that: The stacked pointer network further includes a fully connected layer; the step of performing bias prediction processing on the segment start interval based on the text embedding representation to obtain a corresponding prediction result specifically includes: Extracting first feature representations of all target frames contained in the starting interval of the segment; Performing cross-attention processing on the first feature representation and the text embedding representation to obtain a corresponding second feature representation; Aggregating the second feature representation based on a preset pooling strategy to obtain a corresponding representation vector; Mapping the representation vector to the bias prediction task based on the fully connected layer to obtain a corresponding prediction confidence and a prediction bias value; The prediction confidence and the prediction bias value are used as the prediction result corresponding to the starting interval of the segment.

6. The artificial intelligence-based data retrieval method according to claim 5, characterized in that: The step of generating a target segment position corresponding to the search text based on the prediction result specifically includes: Filtering out the first segment starting interval whose prediction confidence is greater than a preset confidence threshold from all the segment starting intervals; Obtaining a specified prediction bias value corresponding to a starting interval of the first segment; Based on the specified prediction offset value, a corresponding position fine-tuning process is performed on the first segment start interval to obtain a fine-tuned second segment start interval; Performing conversion processing on the second segment start interval based on a preset target conversion form to obtain a processed third segment start interval; A target segment position corresponding to the search text is generated based on the third segment start interval.

7. The artificial intelligence-based data retrieval method according to claim 1, characterized in that: After the step of generating a target segment position corresponding to the search text based on the prediction result, the method further includes: Generate corresponding search results based on the target fragment position; Obtaining a data push method corresponding to the user; Based on the data pushing method, the search result is pushed to the user.

8. A data retrieval device based on artificial intelligence, characterized in that: include: A first acquisition module is used to acquire a search text input by a user corresponding to a target video; A preprocessing module, used for performing text preprocessing on the search text to obtain a corresponding text embedding representation; An extraction module, used to extract image features from the target video to obtain corresponding image features; A second processing module is used to call a preset stacked pointer network, and perform cross-attention processing on the text embedding representation and the image feature based on a cross-attention layer in the stacked pointer network to obtain feature representations of corresponding multiple frames; A first prediction module, configured to perform prediction processing on the feature representations of the plurality of frames to obtain corresponding segment start positions, and determine corresponding segment start intervals based on the segment start positions; A second prediction module, configured to perform bias prediction processing on the segment start interval based on the text embedding representation to obtain a corresponding prediction result; The first generating module is used to generate a target segment position corresponding to the search text based on the prediction result.

9. A computer device, characterized in that: It comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the artificial intelligence-based data retrieval method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the artificial intelligence-based data retrieval method according to any one of claims 1 to 7.

Images