Digital asset retrieval method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a digital asset retrieval method and device, computer equipment, a computer readable storage medium and a computer program product, relates to the technical field of data asset management, and can improve the retrieval efficiency of digital assets. The method comprises the following steps: in response to a digital asset retrieval request input by a user, obtaining an end-cloud collaboration strategy from a cloud end; under the condition that the end-cloud cooperation strategy indicates end-side processing, a vector data synchronization request is sent to the cloud, and an end-side vector database is updated according to a digital asset vector returned by the cloud in response to the vector data synchronization request; the digital asset vector is obtained by vectorizing a plurality of digital assets of the user by the cloud; obtaining a retrieval request vector obtained by vectorizing the digital asset retrieval request by the cloud; and matching the retrieval request vector with the plurality of digital asset vectors in the updated end-side vector database, and obtaining a target digital asset corresponding to the digital asset retrieval request according to a matching result.

Description

Technical Field

[0001] The present application relates to the technical field of data asset management, and in particular to a digital asset retrieval method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the rapid development of information technology, advanced indexing technologies have been introduced into the field of digital asset management, which are efficient and intelligent. Existing digital asset management methods mainly rely on cloud computing platforms and traditional keyword matching and metadata tagging technologies.

[0003] In traditional technology, digital assets are classified and indexed through centralized cloud systems for storage and retrieval with the help of predefined tags or keywords. This method is relatively effective when dealing with single, static digital data, but it is inefficient when faced with complex, multimodal and ever-changing large-scale digital assets. Summary of the invention

[0004] Based on this, it is necessary to provide a digital asset retrieval method, apparatus, computer equipment, computer-readable storage medium and computer program product that can improve the efficiency of digital asset retrieval in response to the above technical problems.

[0005] In a first aspect, the present application provides a digital asset retrieval method, comprising:

[0006] Responding to a digital asset retrieval request input by a user, obtaining a terminal-cloud collaboration strategy from the cloud;

[0007] When the end-cloud collaboration strategy indicates end-side processing, a vector data synchronization request is sent to the cloud, and an end-side vector database is updated according to a digital asset vector returned by the cloud in response to the vector data synchronization request; the digital asset vector is obtained by the cloud performing vectorization processing on multiple digital assets of the user;

[0008] Obtaining a retrieval request vector obtained by vectorizing the digital asset retrieval request in the cloud;

[0009] The search request vector is matched with a plurality of the digital asset vectors in the updated terminal-side vector database, and a target digital asset corresponding to the digital asset search request is obtained according to the matching result.

[0010] In one embodiment, the step of obtaining the end-cloud collaboration strategy from the cloud in response to a digital asset retrieval request input by a user includes:

[0011] Determine a current search scenario according to the digital asset search request;

[0012] If the current search scenario is an image search scenario, an image recognition model component is obtained from the cloud, so as to perform image content recognition on the image carried in the digital asset search request through the image recognition model component, and determine the search target information according to the recognized image elements;

[0013] If the current search scenario is a voice search scenario, a voice recognition model component is loaded from the cloud, so as to convert the request voice in the digital asset search request into a request text through the voice recognition model component, and determine the search target information according to the request text;

[0014] If the current search scenario is a video search scenario, a video analysis model component is obtained from the cloud, so as to perform video content recognition on the video in the digital asset search request through the video analysis model component, and determine the search target information according to the recognized video elements;

[0015] Acquire from the cloud a terminal-cloud collaboration strategy that matches the retrieval target information.

[0016] In one embodiment, each model component in the image recognition model component, the speech recognition model component, and the video analysis model component is obtained from the cloud through the following steps:

[0017] Sending a version query request of the terminal-side model component to the cloud to obtain a version query result;

[0018] If the version query result indicates that there is a version to be updated, sending a request to the cloud for obtaining a download address of a model component corresponding to the version to be updated;

[0019] Obtaining the download address sent by the cloud, and downloading the terminal-side model component from the cloud according to the download address;

[0020] Perform a validity check on the downloaded end-side model component. If the validity check passes, install the downloaded end-side model component.

[0021] In one embodiment, converting the request speech in the digital asset retrieval request into request text by the speech recognition model component includes:

[0022] Using the speech recognition model component, converting the request speech into text content corresponding to the digital asset retrieval request, and performing contextual adjustment on the text content to obtain a first recognized text;

[0023] Sending the first recognized text and text correction request to the cloud;

[0024] A second recognized text is obtained after correction of the first recognized text, and the second recognized text is determined as the request text.

[0025] In one embodiment, obtaining the retrieval request vector obtained by vectorizing the digital asset retrieval request on the cloud comprises:

[0026] Determine the search target information under various information modalities obtained according to the digital asset search request, and send a vectorized request carrying the search target information to the cloud; the vectorized request is used to trigger the cloud to map the search target information under various information modalities to the same feature vector space through a multimodal model to obtain the search request vector under the feature vector space;

[0027] The search request vector returned by the cloud is received.

[0028] In a second aspect, the present application provides a digital asset retrieval method, comprising:

[0029] In response to a terminal-cloud collaborative strategy acquisition request sent by the terminal, the terminal-cloud collaborative strategy is sent to the terminal; the terminal-cloud collaborative strategy acquisition request is sent by the terminal after receiving the digital asset retrieval request,

[0030] In the case where the end-cloud collaboration strategy indicates end-side processing, in response to the terminal's vector data synchronization request, the digital asset vector corresponding to the user's digital asset is obtained and sent to the terminal, so that the terminal updates the end-side vector database according to the digital asset vector, and vectorizes the digital asset retrieval request to obtain a retrieval request vector, which is matched with the multiple digital asset vectors in the updated end-side vector data, and the target digital asset corresponding to the digital asset retrieval request is obtained according to the matching result.

[0031] In a third aspect, the present application also provides a digital asset retrieval device, comprising:

[0032] A collaborative strategy acquisition module is used to obtain a terminal-cloud collaborative strategy from the cloud in response to a digital asset retrieval request input by a user;

[0033] A vector synchronization and update module, configured to send a vector data synchronization request to the cloud when the end-cloud collaboration strategy indicates end-side processing, and update the end-side vector database according to the digital asset vector returned by the cloud in response to the vector data synchronization request; the digital asset vector is obtained by the cloud performing vectorization processing on multiple digital assets of the user;

[0034] A search request vectorization module, used to obtain a search request vector obtained after the cloud performs vectorization processing on the digital asset search request;

[0035] The vector matching and asset identification module is used to match the search request vector with the multiple digital asset vectors in the updated terminal-side vector database, and obtain the target digital asset corresponding to the digital asset search request according to the matching result.

[0036] In a fourth aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above-mentioned digital asset retrieval methods when executing the computer program.

[0037] In a fifth aspect, the present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the digital asset retrieval methods described above are implemented.

[0038] In a sixth aspect, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any of the digital asset retrieval methods described above.

[0039] The above-mentioned digital asset retrieval method, device, computer equipment, computer-readable storage medium and computer program product obtain the end-cloud collaboration strategy from the cloud by responding to the digital asset retrieval request input by the user; when the end-cloud collaboration strategy indicates end-side processing, a vector data synchronization request is sent to the cloud, and the end-side vector database is updated according to the digital asset vector returned by the cloud in response to the vector data synchronization request; the digital asset vector is obtained by the cloud performing vectorized processing on multiple digital assets of the user; the retrieval request vector obtained after the cloud performs vectorized processing on the digital asset retrieval request; the retrieval request vector is matched with multiple digital asset vectors in the updated end-side vector database, and the target digital asset corresponding to the digital asset retrieval request is obtained according to the matching result. In the present application, the end-cloud collaboration strategy reasonably distributes the computing burden of the terminal and the cloud, the terminal is responsible for local retrieval matching, and the cloud is responsible for vectorized processing, so as to achieve efficient utilization of resources, and also shortens the user's retrieval waiting time and improves the response speed by synchronizing the latest vector data in real time and performing matching at the terminal, and at the same time transfers the matching operation to the terminal device, so that the terminal has independent retrieval capabilities in a weak network environment, thereby improving the retrieval efficiency of digital assets. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0041] FIG1 is a diagram of an application environment of a digital asset retrieval method in one embodiment;

[0042] FIG2 is a schematic diagram of a flow chart of a digital asset retrieval method in one embodiment;

[0043] FIG3 is a schematic diagram of a process of obtaining a model component in one embodiment;

[0044] FIG4 is a schematic diagram of a process of initializing and loading a speech recognition model component in one embodiment;

[0045] FIG5 is a schematic diagram of a process flow of real-time speech recognition in one embodiment;

[0046] FIG6 is a schematic diagram of a flow chart of a digital asset retrieval method in another embodiment;

[0047] FIG7 is a schematic diagram of a flow chart of a digital asset retrieval method in yet another embodiment;

[0048] FIG8 is a structural block diagram of a digital asset retrieval device in one embodiment;

[0049] FIG9 is a structural block diagram of a digital asset retrieval device in another embodiment;

[0050] FIG. 10 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0052] The digital asset retrieval method provided in the embodiment of the present application can be applied to the application environment as shown in Figure 1. Among them, the terminal communicates with the cloud through the network. The data storage system can store data that needs to be processed in the cloud. The data storage system can be integrated on the cloud, or it can be placed on the cloud or other network servers. Among them, the terminal can be but is not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The cloud can be an independent physical server, or it can be a server cluster or distributed system composed of multiple physical servers.

[0053] In an exemplary embodiment, as shown in FIG2 , a digital asset retrieval method is provided, which is described by taking the method applied to the terminal in FIG1 as an example, and includes the following steps S202 to S208 . Among them:

[0054] Step S202, in response to a digital asset search request input by a user, obtaining a terminal-cloud collaboration strategy from the cloud;

[0055] Among them, the digital asset retrieval request is a retrieval of digital assets initiated by the user, which is used to locate or find digital assets on the terminal device. Digital assets refer to various types of information assets stored in a digital environment, including but not limited to electronic documents, pictures, videos, audio, databases, etc. The management, retrieval and processing of digital assets are an indispensable part of the modern information society. Its efficient storage and retrieval are of great significance to improving work efficiency and the degree of informatization. The end-cloud collaboration strategy is a configuration file or policy set issued by the cloud based on the terminal request and resource conditions. It aims to guide the terminal device to reasonably use local computing power and cloud computing power to collaborate to complete tasks.

[0056] Exemplarily, a user issues a digital asset retrieval request on a terminal device. After the operating system of the terminal device detects the digital asset retrieval request, it parses the digital asset request into structured data for subsequent communication with the cloud. The terminal device generates a policy request message, which contains specific parameters of the current request, such as the request type (such as pictures, videos, documents), the operating status of the device (battery power, network bandwidth), etc., so that the cloud can generate a suitable end-cloud collaboration strategy based on these parameters. Optionally, the digital asset request is transmitted to the cloud via an encrypted communication protocol. After receiving the policy request, the cloud first performs a trusted authentication to ensure that the request comes from a valid terminal device. Then, the cloud generates an end-cloud collaboration strategy based on the current state of the terminal device and the request content, and the generated strategy is transmitted back to the terminal device. Optionally, the end-cloud collaboration strategy can be set to call the local cache or cloud database, choose to call the end-side model first, etc. This end-cloud collaboration strategy provides specific guidance for the terminal device to perform subsequent digital asset retrieval, ensuring that the entire process achieves an optimized effect in resource allocation.

[0057] Step S204, when the end-cloud collaboration strategy indicates end-side processing, a vector data synchronization request is sent to the cloud, and the end-side vector database is updated according to the digital asset vector returned by the cloud in response to the vector data synchronization request; the digital asset vector is obtained by the cloud performing vectorized processing on multiple digital assets of the user.

[0058] The vector data synchronization request is an interactive request initiated by the terminal device to the cloud, with the purpose of synchronously updating the completed digital asset vector data in the cloud. These synchronized vector data will be written into the local database of the terminal device to ensure that the terminal device can use the complete asset vector data set for local matching and retrieval.

[0059] The terminal-side vector database is a vector database stored in the terminal device, which contains digital asset vectors synchronized from the cloud. These digital asset vectors represent various digital assets stored by users in the cloud or locally, and are used to perform fast vector matching on the terminal to reduce dependence on the cloud and network latency.

[0060] Digital asset vectors are vector data generated by vectorizing the user's digital assets in multiple information modalities in the cloud. They unify complex multimodal data into a matchable vector form, thereby improving the accuracy and efficiency of data retrieval.

[0061] Exemplarily, when the terminal-cloud collaboration strategy instructs the terminal to perform processing, the terminal device generates a vector data synchronization request. After the cloud receives the vector data synchronization request from the terminal, it first verifies the validity of the vector data synchronization request. Subsequently, the cloud packages the digital asset vector data generated by vectorizing the user's multimodal digital assets and returns it to the terminal. After receiving these digital asset vectors, the terminal device stores the data in a local vector database to ensure that it can perform fast vector matching operations locally.

[0062] Step S206, obtaining a search request vector obtained after the cloud performs vectorization processing on the digital asset search request,

[0063] The search request vector is a high-dimensional vector representation obtained by vectorizing the user's digital asset search request in the cloud. Through this vector, the terminal can quickly match the information input by the user with the digital asset vector in the local vector database.

[0064] Exemplarily, the user inputs or uploads a digital asset retrieval request at the terminal. After the terminal device parses the user input, it identifies the specific content of the request and sends the request to the cloud through the network. After the cloud receives the retrieval request sent by the terminal, it calls the corresponding vectorization model based on the received data for processing, extracts the core features of the digital asset retrieval request and generates a corresponding retrieval request vector. After the cloud completes the vectorization processing of the digital asset retrieval request, it transmits the generated retrieval request vector to the terminal device through an encrypted channel. After the terminal device receives the retrieval request vector, it stores it in the local cache and waits for matching with the digital asset vector in the local vector database to quickly respond to the user's query needs.

[0065] Step S208: Match the search request vector with the multiple digital asset vectors in the updated terminal-side vector database, and obtain the target digital asset corresponding to the digital asset search request according to the matching result.

[0066] The target digital asset may be the digital asset that is most similar to the user's search request determined through a matching operation. This matching process is based on the feature similarity of the vector data in the terminal database and the search request vector, and the qualified assets are selected through calculation and returned to the user.

[0067] Exemplarily, after receiving the retrieval request vector from the cloud, the terminal device stores it in a temporary memory or cache area, ready to be compared with the digital asset vector in the end-side vector database. The terminal device starts to call the digital asset vector in the end-side vector database and compares it with the retrieval request vector one by one. Optionally, the comparison process can be based on a similarity algorithm in a multidimensional space, such as cosine similarity or Euclidean distance, and the degree of similarity is judged by the distance or angle between the vectors. The shorter the vector distance or the closer the angle, the higher the similarity.

[0068] According to the similarity score of the comparison, the matching results are sorted by similarity. The terminal device returns the digital asset with the highest matching degree as the target digital asset for retrieval based on the preset threshold or user needs. The terminal device stores the matched target digital asset in the cache and displays the corresponding content to the user, who can view or choose further operations.

[0069] In the above digital asset retrieval method, by responding to the digital asset retrieval request input by the user, the end-cloud collaboration strategy is obtained from the cloud; when the end-cloud collaboration strategy indicates end-side processing, a vector data synchronization request is sent to the cloud, and the end-side vector database is updated according to the digital asset vector returned by the cloud in response to the vector data synchronization request; the digital asset vector is obtained by the cloud vectorizing the user's multiple digital assets; the retrieval request vector obtained after the cloud vectorizes the digital asset retrieval request; the retrieval request vector is matched with the multiple digital asset vectors in the updated end-side vector database, and the target digital asset corresponding to the digital asset retrieval request is obtained according to the matching result. In this embodiment, the end-cloud collaboration strategy reasonably distributes the computing burden of the terminal and the cloud, the terminal is responsible for local retrieval matching, and the cloud is responsible for vectorization processing, so as to achieve efficient use of resources, and also shortens the user's retrieval waiting time and improves the response speed by synchronizing the latest vector data in real time and performing matching at the terminal. At the same time, the matching operation is transferred to the terminal device, so that the terminal has independent retrieval capabilities in a weak network environment, the application scenarios are broader, the sensitive information is safer, and the user experience is significantly improved, thereby improving the retrieval efficiency of digital assets.

[0070] In an exemplary embodiment, in response to a digital asset retrieval request input by a user, obtaining a terminal-cloud collaboration strategy from the cloud includes:

[0071] Determine the current search scenario according to the digital asset retrieval request; if the current search scenario is an image retrieval scenario, obtain an image recognition model component from the cloud to perform image content recognition on the image carried in the digital asset retrieval request through the image recognition model component, and determine the retrieval target information based on the recognized image elements; if the current search scenario is a voice retrieval scenario, load a voice recognition model component from the cloud to convert the request voice in the digital asset retrieval request into request text through the voice recognition model component, and determine the retrieval target information based on the request text; if the current search scenario is a video retrieval scenario, obtain a video analysis model component from the cloud to perform video content recognition on the video in the digital asset retrieval request through the video analysis model component, and determine the retrieval target information based on the recognized video elements; obtain an end-cloud collaboration strategy that matches the retrieval target information from the cloud.

[0072] The current search scenario is the analysis scenario corresponding to the type of digital asset request content input by the user. For example, if the user uploads an image, the current search scenario is determined to be an "image search scenario"; if it is a voice input, it is determined to be a "voice search scenario". This scenario division helps the system select the corresponding model components and processing methods under different data modalities.

[0073] The model component refers to a specific artificial intelligence model obtained by the terminal device from the cloud, which is used to process specific tasks in the current retrieval scenario. In one embodiment, the model component can be obtained by including but not limited to the following technologies: The natural language processing large model refers to a large deep learning model with a huge number of parameters and complex structure in the field of natural language processing. Through the training of massive language data, these large models can handle various natural language tasks, such as text classification, named entity recognition, sentiment analysis, etc. The NLP large model can effectively cope with the complexity and ambiguity of natural language, and can be directly deployed in practical applications to support efficient processing of massive text data streams. Optical Character Recognition (OCR) is a technology that can recognize printed characters on paper documents through electronic devices (such as scanners or digital cameras) and convert them into computer-readable digitized text. OCR technology is widely used in document digitization, automated data entry and other fields. By converting the text in paper documents into electronic format, the efficiency of document processing is greatly improved. Automatic Speech Recognition (ASR) is a technology designed to convert the vocabulary content in human speech into computer-readable input, such as text or other processable data formats. ASR technology enables computers to understand and recognize human speech, thereby achieving human-computer interaction. It is widely used in scenarios such as voice assistants, automatic customer service, and voice search. Text to Speech (TTS) is a technology that converts text information in a computer into voice output. Through TTS technology, users can output electronic documents, web page content, or other text data in the form of voice, thereby improving the convenience of information acquisition, especially for people with visual impairments or scenarios that require efficient information acquisition.

[0074] The target information is the information obtained by recognizing the characteristics of the user input content, which is used to obtain the end-cloud collaboration strategy in the next step. The information is extracted based on different model components, and the specific information includes but is not limited to image content elements, voice text content, video elements, etc.

[0075] Specifically, when a user inputs a digital asset retrieval request in the terminal, the system parses the request content to determine its data modality. The system can select a model component suitable for the data type, thereby improving the targetedness of data processing.

[0076] If the current scene is image retrieval, the terminal requests the cloud to load the image recognition model component. After the cloud returns the image recognition model component, the terminal calls the image recognition model component to identify the main content of the picture and extract the image elements related to the retrieval (such as object category, color, text information, etc.) to generate the retrieval target information.

[0077] If the current scenario is voice retrieval, the terminal loads the voice recognition model component from the cloud. The cloud returns the voice recognition model to the terminal, and the terminal device uses the voice recognition model to convert the voice into text data and extract the key content in the text to generate the retrieval target information.

[0078] If the current scenario is video retrieval, the terminal device requests the video analysis model component from the cloud. After the cloud transmits the model to the terminal, the terminal uses the model component to analyze the video content and extract the main content elements in the video (such as scene changes, character or object labels, etc.) to generate retrieval target information.

[0079] The terminal device sends a request to the cloud based on the extracted retrieval target information. The cloud generates an end-cloud collaboration strategy suitable for the current retrieval scenario based on the information and sends it to the terminal device.

[0080] In this embodiment, dedicated recognition model components are loaded according to different input modes to ensure the recognition accuracy of the retrieval request under the current data mode. By deploying the artificial intelligence reasoning model component on the terminal side to infer the pre-trained model, artificial intelligence processing is directly performed, avoiding data transmission back and forth, exploring the computing power of the user terminal, and saving bandwidth costs. It is organically combined with cloud artificial intelligence, and dynamically linked to realize a unique end-cloud hybrid artificial intelligence technology solution, reducing the waste of cloud and terminal resources and optimizing the overall resource allocation. .

[0081] In an exemplary embodiment, each model component in the image recognition model component, the speech recognition model component and the video analysis model component is obtained from the cloud through the following steps: sending a version query request for the terminal side model component to the cloud to obtain the version query result; if the version query result indicates that there is a version to be updated, sending a download address acquisition request for the model component corresponding to the version to be updated to the cloud; obtaining the download address sent by the cloud, and downloading the terminal side model component from the cloud according to the download address; performing a legitimacy check on the downloaded terminal side model component, and installing the downloaded terminal side model component if the legitimacy check passes.

[0082] The version query request is a request sent by the terminal device to the cloud to confirm whether there is an updated version of the currently installed terminal model component. Through this query request, it can be identified whether the terminal model component needs to be updated to keep the model in the latest state.

[0083] Legitimacy checking is the process of verifying the integrity and security of downloaded model components. This check ensures that the downloaded model components have not been tampered with or implanted with malicious code to ensure system security. Legitimacy checking can be based on signature verification or checksum comparison to verify the source and integrity of model components.

[0084] Specifically, when the terminal device is ready to load the image recognition model component, the speech recognition model component or the video analysis model component, the terminal first sends a version query request to the cloud through the communication network. After receiving the version query request, the cloud queries the corresponding model version library according to the version information provided by the terminal and generates a version query result. If the model component installed by the terminal is the latest version, the cloud returns the result of "already up to date", and the terminal device can continue to use the current model; if there is an updated version, an update instruction is returned to indicate that the terminal device needs to update the model component. If the version query result indicates that the model component has an updated version, the terminal device will send a download information acquisition request to the cloud to obtain the specific download address of the updated version of the model component. The terminal device obtains the download address of the model component according to the download information, and sends a download request to the cloud through a secure communication channel to start downloading the latest version file of the model component. After the download is completed, the terminal device performs a legitimacy check on the downloaded model component to verify the source and integrity of the component. The legitimacy check can be performed based on signature verification or file verification code. If the legitimacy check passes, the terminal device will install or replace the model component to the local environment to complete the model update; if the check fails, the terminal device will abandon the installation to avoid security risks.

[0085] In one embodiment, the terminal includes an AI engine, and the cloud includes an intelligent application middle platform and an intelligent storage middle platform. The end-side AI engine is an integrated package of end-side applications. Its main functions include integrating and deploying the end-side AI reasoning framework on mobile devices, dynamically loading end-side AI models and reasoning components according to application scenarios, and providing security protection and performance optimization services for the entire end-side AI process. Through the end-side AI engine, terminal devices can implement localized AI reasoning, improve response speed and data privacy protection capabilities. The intelligent application middle platform and the intelligent storage middle platform are cloud service clusters that provide intelligent services and storage services. The intelligent application middle platform focuses on the provision of business logic and intelligent services to ensure that various applications can run efficiently and intelligently; the intelligent storage middle platform is responsible for the storage and management of massive data, providing efficient storage solutions, and supporting data access requirements in different application scenarios. Each model component in the image recognition model component, speech recognition model component, and video analysis model component, as shown in Figure 3, can be obtained from the cloud through the following steps:

[0086] The AI ​​engine in the terminal device initiates a model component version query request and obtains the latest AI model component version information from the intelligent application center to determine whether the current local model needs to be updated.

[0087] After receiving the request from the terminal, the intelligent application platform first authenticates the request to confirm the legitimacy of the request and the identity of the terminal.

[0088] The intelligent application platform obtains the latest model version information from the model storage platform and compares it with the current version information of the terminal to determine whether the terminal model needs to be updated.

[0089] If the intelligent application platform determines that the terminal model needs to be updated, it returns the "update required" status and version information; if the model is already the latest version, it returns the "no update required" status.

[0090] After the terminal receives the update notification, the AI ​​engine initiates a request to the intelligent application center to obtain the model component download address in order to download the latest model component file.

[0091] The intelligent application platform re-verifies the terminal's download request to confirm the terminal's legitimacy and download permissions to prevent unauthorized devices or users from accessing model data.

[0092] The intelligent application middle platform interacts with the intelligent storage middle platform through internal communication in the cloud, transmits relevant information of the model component, and requests to generate the download address of the model component.

[0093] After receiving the request, the intelligent storage middle platform locates the corresponding model file, generates a download address, and returns the download address to the intelligent application middle platform.

[0094] The intelligent storage middle platform transmits the generated download address back to the intelligent application middle platform for subsequent transmission to the terminal for use.

[0095] The intelligent application platform sends the download address to the terminal's AI engine, informing the terminal that it can start downloading the latest model components.

[0096] After receiving the download address, the terminal AI engine starts downloading the model component files from the intelligent storage center to update the local AI model of the terminal.

[0097] The terminal requests the model file from the intelligent storage center through the download address. The intelligent storage center transfers the model file to the terminal and completes the file download process.

[0098] After the file transfer is completed, the intelligent storage center returns a confirmation notification of download completion to the terminal, indicating that the model file has been successfully downloaded to the terminal.

[0099] After the terminal AI engine completes the file download, it performs a legitimacy check on the model components, such as checking file integrity and signature verification, to ensure that the source and content of the model file have not been tampered with. After the legitimacy check passes, the terminal AI engine loads the model into local memory or cache so that it can be called in subsequent AI reasoning tasks.

[0100] In this embodiment, through version query and update, the terminal device can obtain the latest model components in the cloud in a timely manner, keep the latest status of each recognition model, and improve the accuracy and efficiency of the system in processing different modal data. At the same time, standardized protection is carried out under the legality detection mechanism to ensure that the downloaded model components are not tampered with during the transmission process, prevent illegal or malicious components from being installed on the terminal device, and improve the security and stability of the system.

[0101] In an exemplary embodiment, the converting of the request voice in the digital asset retrieval request into request text by the voice recognition model component includes: utilizing the voice recognition model component to convert the request voice into text content corresponding to the digital asset retrieval request, performing contextual adjustment on the text content, and obtaining a first recognized text; sending the first recognized text and a text correction request to the cloud; obtaining a second recognized text after the correction of the first recognized text, and determining the second recognized text as the request text.

[0102] Among them, the first recognized text is the text content generated by the speech recognition model component without further correction. Context adjustment is the preliminary correction process of the first recognized text by the terminal device. Through context adjustment, the first recognized text can be preliminarily processed by context-related smoothing, rule-based error correction, statistical-based error correction, etc., so as to generate a more fluent and coherent text, laying the foundation for subsequent cloud correction. The text correction request is a further optimization request sent by the terminal device to the cloud after generating the first recognized text. The request contains the first recognized text and can carry context information for the cloud to use the natural language model for precise correction to generate more accurate text content. The second recognized text is the final text after cloud correction, which reflects the user's intention and is more accurate in semantics and grammar. The text is determined as the retrieval request text and is directly used for subsequent digital asset retrieval.

[0103] Exemplarily, a user initiates a digital asset retrieval request in the form of voice on a terminal device. The terminal device captures the user's voice input through a microphone and starts a voice recognition model component. The voice recognition model component processes the user's voice data, converts the voice signal into text form, and generates a preliminary text result, namely the "first recognition text". After generating the first recognition text, the terminal device adjusts it for context. This step is to make preliminary corrections to the first recognition text through context-related smoothing, rule-based error correction, statistical-based error correction, or a language model based on user historical input. The first recognition text after context adjustment is packaged and accompanied by a text correction request, and the terminal sends it to the cloud for further optimization. After receiving the text correction request, the cloud corrects the first recognition text through its natural language processing model to solve problems such as spelling, grammar, and semantic understanding, and generates a corrected text "second recognition text". After the cloud completes the text correction, it returns the second recognition text to the terminal device. After receiving the second recognition text, the terminal device determines it as the final "request text", that is, the user's digital asset retrieval request. The request text accurately reflects the user's query intent and ensures that the retrieval results meet expectations.

[0104] In a specific embodiment, as shown in FIG4 , the speech recognition model component initialization and model loading process includes:

[0105] The user starts the voice input function on the terminal device. The voice-to-text component of the terminal starts and performs initialization operations to ensure that the relevant modules and resources are ready to process the user's voice input. After the component is initialized, the terminal device checks whether the currently loaded voice recognition model is the latest version to ensure the accuracy of the recognition effect and the compatibility of the terminal. If the local model version is detected to be too old, the terminal will return the "need to upgrade status code" and perform the model upgrade operation. At this time, the terminal can connect to the cloud to download the latest voice recognition model. If the version detection result shows that the model is the latest version, the terminal will skip the upgrade operation and directly enter the dynamic configuration stage of the model. After confirming that the version is valid, the terminal enters the model dynamic stage. This process adjusts the configuration of the model according to the resource status of the current device to ensure smooth operation on the current terminal. After the dynamic configuration is completed, the terminal attempts to load the voice recognition model into the memory so that the model can be used for voice-to-text tasks at any time. If the model fails to load, the terminal returns the "initialization failure status code" and stops the subsequent process. The terminal can display an error message to prompt the user to try to reload or check the device status. If the model is loaded successfully, the terminal returns an "initialization success status code", indicating that the speech-to-text component is ready for speech recognition tasks.

[0106] In one embodiment, as shown in FIG5 , the real-time speech recognition process includes:

[0107] The user starts voice input through the terminal device. The device's microphone captures the user's real-time voice and generates a real-time voice audio file. The voice recognition model component receives the real-time audio stream and starts the recognition process to convert the voice data into text. The terminal judges the recognition result to determine whether the recognition process is successful. The judgment criteria can be based on recognition accuracy, signal-to-noise ratio, etc. If the recognition fails to meet the requirements, the terminal returns a "recognition failure status code" and feedbacks to the user or application to prompt the cause of the error, including but not limited to factors such as excessive background noise and low audio quality. If the recognition is successful, the terminal will add punctuation to the generated text. The terminal automatically recognizes sentence segments and pauses based on the grammatical structure of Chinese, and inserts appropriate punctuation (such as periods, commas, etc.) to make the text content more coherent and easy to understand.

[0108] During the process of converting speech to text, the terminal checks whether the user's speech input is still continuing. If the user is inputting a long speech, the terminal will continuously detect new audio streams. If the speech input has not yet ended, the terminal will return and continue speech recognition so that the subsequent content can also be converted into text to ensure the complete recording of the long speech. If it is detected that the speech input has been completed, the terminal will stop capturing audio and generate the final recognized text.

[0109] In this embodiment, by combining the voice recognition model component of the terminal with the text correction function of the cloud, the accuracy of the voice recognition results is improved, the fluency of the recognized text is ensured, the text content is more in line with the user's actual expression intention, and the accuracy of the retrieval is improved. At the same time, the terminal device is only responsible for preliminary voice recognition and context adjustment, and the complex text correction tasks are handled by the cloud. This division of labor reduces the computing load of the terminal, ensures that the terminal can achieve smooth voice recognition and retrieval experience on weak computing power devices, makes full use of the computing power of the user terminal device, and saves the computing resource cost of artificial intelligence.

[0110] In an exemplary embodiment, obtaining a retrieval request vector obtained after the cloud performs vectorized processing on a digital asset retrieval request includes: determining retrieval target information under various information modalities obtained according to the digital asset retrieval request, and sending a vectorized request carrying the retrieval target information to the cloud; the vectorized request is used to trigger the cloud to map the retrieval target information under various information modalities to the same feature vector space through a multimodal model to obtain a retrieval request vector in the feature vector space; and receiving the retrieval request vector returned by the cloud.

[0111] Among them, the multimodal big model is a machine learning model that combines multiple different information modalities such as text, images, videos, and audio for training. The model can process and understand complex data from different sources, improving the performance of machine learning and artificial intelligence systems. Through the fusion of multimodal data, the multimodal big model can capture more semantic information, thereby achieving more accurate reasoning and analysis.

[0112] The feature vector space is a high-dimensional vector space to which the cloud multimodal model maps different modal data. In this feature vector space, data features are represented in the form of vectors, ensuring the consistency of different modal data in similarity calculation, which facilitates multimodal retrieval and matching.

[0113] Specifically, the terminal device extracts the retrieval target information from the input content through the model component on the end side according to the digital asset retrieval request input by the user. The terminal device sends the vectorized request containing the retrieval target information to the cloud. The vectorized request contains not only specific modal information but also a description of the target information. After receiving the vectorized request, the cloud starts the vectorization process of the multimodal model to map the feature information of each modality to a unified feature vector space. The multimodal model on the cloud extracts features from data of different modalities according to the vectorized request and converts it into a vector representation in the feature vector space. This process ensures that whether it is an image, text or voice, a vector with a unified format can be generated for subsequent similarity matching and retrieval. After the cloud completes the vectorization, the generated retrieval request vector is fed back to the terminal device. After receiving the vector, the terminal device temporarily stores it in the local cache or memory, ready to match it with the digital asset vector of the end-side vector database in the subsequent steps.

[0114] In this embodiment, a cloud-based multimodal model is used to map each modal data to the same feature vector space. By completing the complex vectorization process in the cloud, the terminal device only needs to receive and process the results, which reduces the requirements for terminal computing resources. At the same time, it ensures the normal operation of weak network environments or low-power devices, and improves the system's adaptability and user experience.

[0115] In an exemplary embodiment, as shown in FIG6 , another digital asset retrieval method is provided, which is described by taking the application in the cloud as an example, and may include the following steps:

[0116] Step S602, in response to the end-cloud collaborative strategy acquisition request sent by the terminal, the end-cloud collaborative strategy is sent to the terminal; the end-cloud collaborative strategy acquisition request is sent by the terminal after receiving the digital asset retrieval request.

[0117] Exemplarily, the cloud receives a request to obtain an end-cloud collaboration strategy from a terminal. The cloud generates an end-cloud collaboration strategy applicable to the retrieval task based on the received terminal information and the content of the retrieval request, combined with the resource status and policy library of the current system. The end-cloud collaboration strategy may involve the distribution method of data processing, the priority setting of model calls, the frequency of data synchronization, and the use distribution of terminal and cloud resources. If there is a suitable preset strategy, the cloud can directly select the corresponding strategy and prepare to send it to the terminal. After the cloud completes the generation or selection of the end-cloud collaboration strategy, it feeds back the strategy to the terminal device through a secure communication channel. After receiving the strategy, the terminal loads the strategy locally to guide subsequent digital asset retrieval operations. This step ensures that the terminal has appropriate operational guidance, enabling it to perform calculations and resource calls locally or in the cloud according to policy guidelines, thereby improving retrieval efficiency.

[0118] Step S604, when the end-cloud collaboration strategy indicates end-side processing, in response to the terminal's vector data synchronization request, obtain the digital asset vector corresponding to the user's digital asset and send it to the terminal, so that the terminal updates the end-side vector database according to the digital asset vector, and vectorizes the digital asset retrieval request to obtain a retrieval request vector, which is matched with multiple digital asset vectors in the updated back-end side vector data, and obtains the target digital asset corresponding to the digital asset retrieval request based on the matching result.

[0119] Exemplarily, when the end-cloud collaboration strategy instructs end-side processing, the terminal device sends a vector data synchronization request to the cloud. After receiving the request, the cloud confirms the validity of the request and determines whether it meets the indication conditions for end-side processing. Based on the user identity or request parameters, the cloud locates the user's digital asset vector after the digital asset is vectorized by the multimodal model, and sends the generated digital asset vector data to the terminal device through an encrypted channel. After receiving the vector data, the terminal stores the data in the local vector database to ensure that the update of the local database is consistent with the cloud data. After the vector data synchronization is completed, the terminal device matches the retrieval request vector returned by the cloud with the updated local vector database. The terminal device quickly matches the digital assets most similar to the retrieval request in the local database through multi-dimensional space similarity calculation, obtains the matching result, that is, the target digital asset, and displays it to the user.

[0120] In this embodiment, through unified policy management in the cloud, the system has the ability to generate collaborative policies in real time according to environmental changes and terminal status, providing support for the system to be expanded to more terminal devices or application scenarios. The instructions processed on the terminal side enable the terminal to complete the matching operation locally, reducing the cloud computing burden and network transmission delay, significantly shortening the user's waiting time and improving the user experience.

[0121] In order to enable those skilled in the art to better understand the above steps, the embodiment of the present application is illustrated below by using an example, but it should be understood that the embodiment of the present application is not limited to this.

[0122] In an exemplary embodiment, the terminal includes a client application and an artificial intelligence software development kit (AI SDK), and the cloud includes a business platform and an intelligent application platform. The terminal and the cloud can collaborate with each other. End-cloud collaboration means that both the terminal device and the cloud are deployed with AI capability services, and efficient data interaction and collaborative processing are achieved through specific protocols and algorithms to optimize resource allocation and realize intelligent services. Specifically, the terminal device and the cloud leverage their respective advantages through collaboration. The terminal device performs local computing tasks, and the cloud provides powerful computing resources and storage capabilities to jointly complete complex computing tasks and data processing. AISDK is an integrated package designed for mobile applications, which aims to provide the development and application of AI-related capability services. Through AISDK, developers can easily integrate artificial intelligence functions into the terminal as an AI engine, support various AI tasks, such as image recognition, speech recognition, etc., and improve the intelligence level of mobile applications.

[0123] Specifically, as shown in FIG7 , the process of end-cloud collaboration for image retrieval includes:

[0124] The user starts the image search service in the client application, triggering the client application to send a request to the business platform. The purpose of this request is to obtain the end-cloud collaboration strategy suitable for the current operation, so as to determine whether it is necessary to enable the AI ​​model on the terminal side, or choose to process in the cloud based on the number of images and resource conditions.

[0125] After receiving the request from the client application, the business platform immediately initiates a further request to the intelligent application middle platform, specifically to obtain the latest end-cloud collaboration strategy. The intelligent application middle platform has more computing and resource management capabilities and can generate a collaboration strategy suitable for the current task.

[0126] After receiving the request from the business platform, the intelligent application platform analyzes the current resource status and client requirements, generates and returns the appropriate end-cloud collaboration strategy to the business platform. This strategy includes some important parameters, such as whether to process by end-side AI or cloud processing; the interval and batch method of image data synchronization; image paging suggestions, etc.

[0127] After obtaining the end-cloud collaboration strategy, the business platform determines the amount of user image data. If the number of user images exceeds 10,000, the system returns to the cloud processing mode by default, that is, the image indexing and vector matching work will be completed in the cloud to reduce the computing pressure of the terminal. If the number of user images is less than 10,000, the system will allow terminal-side AI processing. Subsequently, the business platform returns the final end-cloud collaboration strategy to the client application to guide subsequent operations.

[0128] After receiving the end-cloud collaboration strategy, the client application first determines whether this image search request is the first time used by the user. If it is the first time for the user to use this function, the client application will request the business platform to obtain a full list of image data in order to complete the image index and vector database construction. If the user has already used this function, the client only needs to obtain the incremental image list (that is, the images added or modified since the last use), which can improve the efficiency of data synchronization and reduce network transmission.

[0129] After receiving the image request from the client application, the business platform queries the data database stored in the server or cloud and generates a detailed list of image files, including the file ID (FileID), file name (fileName), last modification time (modifyTime) and other information of each image. This file list is used to help the client application determine the image content that needs to be synchronized. Subsequently, the business platform returns the generated image list to the client application.

[0130] Based on the image list returned by the business platform, the client application sends a request to the local AI SDK, requesting the AI ​​SDK to synchronize the image vector data in the cloud to the vector database of the terminal.

[0131] Based on the client application's request, the AI ​​SDK starts the vector database synchronization process, downloads the image vector data from the cloud, and stores it in the client-side vector database. After the synchronization is complete, the AI ​​SDK will feedback the synchronization status to the client application to ensure that the client knows that the vector data has been updated and can retrieve images at any time.

[0132] After completing data synchronization, users can initiate specific image search requests in the client application, which may include "search images by text" or "search images by image". The client application sends a request to the business platform based on the text or image information entered by the user, so that the business platform can convert the text or image data into feature vectors for retrieval in the vector database.

[0133] After the intelligent application platform completes the vectorization processing, it returns the generated feature vector data to the business platform. This data contains the key features of the user input information and is mapped to a unified vector space, which facilitates efficient similarity matching in the local database.

[0134] The business platform returns the vectorized data generated by the intelligent application middle platform to the client application. After receiving the feature vector, the client application passes it to the AI ​​SDK to start the image indexing and matching process. The AI ​​SDK uses the image vector data in the local vector database to start fast vector matching operations on the terminal. The indexing method is a technical means for quickly retrieving and locating digital assets. This method enables queries to quickly and accurately locate target digital assets by building and maintaining an efficient index data structure. The design and optimization of the index data structure directly affects the retrieval efficiency, especially in a massive data environment.

[0135] Based on the data in the local vector database, AI SDK uses the vector similarity algorithm to match the user's input vector and find the most similar image. This operation is performed on the terminal side, ensuring that the image search process can be completed quickly on the user's local computer, improving the user experience.

[0136] After the AI ​​SDK completes the image matching, it returns the matching results to the client application. The client application displays the final matching results to the user, who can view the matched images or perform further operations.

[0137] In this embodiment, the real-time synchronization and data interaction between the terminal device and the cloud server improves the indexing efficiency and accuracy of digital assets, reduces the existing cloud-side computing power cost and network bandwidth cost, and supports the intelligent indexing construction and maintenance mechanism of multi-modal large models, which greatly improves the user experience and adapts to the dynamic changes of digital assets.

[0138] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0139] Based on the same inventive concept, the embodiment of the present application also provides a digital asset retrieval device for implementing the digital asset retrieval method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more digital asset retrieval device embodiments provided below can refer to the limitations of the digital asset retrieval method above, and will not be repeated here.

[0140] In an exemplary embodiment, as shown in FIG8 , a digital asset retrieval device 800 is provided, comprising: an acquisition coordination strategy module 810, a vector synchronization and update module 820, a retrieval request vectorization module 830, and a vector matching and asset identification module 840, wherein:

[0141] The acquisition coordination strategy module 810 is used to obtain the end-cloud coordination strategy from the cloud in response to the digital asset retrieval request input by the user;

[0142] The vector synchronization and update module 820 is used to send a vector data synchronization request to the cloud when the end-cloud collaboration strategy indicates end-side processing, and update the end-side vector database according to the digital asset vector returned by the cloud in response to the vector data synchronization request; the digital asset vector is obtained by the cloud performing vectorization processing on multiple digital assets of the user;

[0143] A search request vectorization module 830 is used to obtain a search request vector obtained after the cloud performs vectorization processing on the digital asset search request;

[0144] The vector matching and asset identification module 840 is used to match the search request vector with the multiple digital asset vectors in the updated terminal-side vector database, and obtain the target digital asset corresponding to the digital asset search request according to the matching result.

[0145] In one embodiment, the acquisition coordination strategy module 810 is further used to determine a current search scenario according to the digital asset search request;

[0146] If the current search scenario is an image search scenario, an image recognition model component is obtained from the cloud, so as to perform image content recognition on the image carried in the digital asset search request through the image recognition model component, and determine the search target information according to the recognized image elements;

[0147] If the current search scenario is a voice search scenario, a voice recognition model component is loaded from the cloud, so as to convert the request voice in the digital asset search request into a request text through the voice recognition model component, and determine the search target information according to the request text;

[0148] If the current search scenario is a video search scenario, a video analysis model component is obtained from the cloud, so as to perform video content recognition on the video in the digital asset search request through the video analysis model component, and determine the search target information according to the recognized video elements;

[0149] Acquire from the cloud a terminal-cloud collaboration strategy that matches the retrieval target information.

[0150] In one embodiment, the acquisition coordination strategy module 810 is also used for each model component in the image recognition model component, the speech recognition model component and the video analysis model component, and is acquired from the cloud through the following steps:

[0151] Sending a version query request of the terminal-side model component to the cloud to obtain a version query result;

[0152] If the version query result indicates that there is a version to be updated, sending a request to the cloud for obtaining a download address of a model component corresponding to the version to be updated;

[0153] Obtaining the download address sent by the cloud, and downloading the terminal-side model component from the cloud according to the download address;

[0154] Perform a validity check on the downloaded end-side model component. If the validity check passes, install the downloaded end-side model component.

[0155] In one embodiment, the acquisition coordination strategy module 810 is further used to convert the request voice in the digital asset retrieval request into a request text through the speech recognition model component, including:

[0156] Using the speech recognition model component, converting the request speech into text content corresponding to the digital asset retrieval request, and performing contextual adjustment on the text content to obtain a first recognized text;

[0157] Sending the first recognized text and text correction request to the cloud;

[0158] A second recognized text is obtained after correction of the first recognized text, and the second recognized text is determined as the request text.

[0159] In one embodiment, the search request vectorization module 830 is further used to determine the search target information under various information modalities obtained according to the digital asset search request, and send the vectorized request carrying the search target information to the cloud; the vectorized request is used to trigger the cloud to map the search target information under various information modalities to the same feature vector space through a multimodal model, and obtain the search request vector under the feature vector space;

[0160] The search request vector returned by the cloud is received.

[0161] In an exemplary embodiment, as shown in FIG9 , another digital asset retrieval device 900 is provided, which is applied to the cloud and includes: a terminal-cloud collaboration strategy module 910 and a digital asset vector retrieval module 920, wherein:

[0162] The terminal-cloud collaborative strategy module 910 is used to send the terminal-cloud collaborative strategy to the terminal in response to the terminal-cloud collaborative strategy acquisition request sent by the terminal; the terminal-cloud collaborative strategy acquisition request is sent by the terminal after receiving the digital asset retrieval request,

[0163] The digital asset vector retrieval module 920 is used to, when the end-cloud collaboration strategy indicates end-side processing, respond to the vector data synchronization request of the terminal, obtain the digital asset vector corresponding to the user's digital asset and send it to the terminal, so that the terminal updates the end-side vector database according to the digital asset vector, and vectorizes the digital asset retrieval request to obtain a retrieval request vector, matches it with the multiple digital asset vectors in the updated end-side vector data, and obtains the target digital asset corresponding to the digital asset retrieval request according to the matching result.

[0164] Each module in the above digital asset retrieval device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0165] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG10. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a digital asset retrieval method is implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0166] Those skilled in the art will understand that the structure shown in FIG. 10 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0167] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0168] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0169] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0170] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0171] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0172] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0173] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A digital asset retrieval method, characterized in that: Applied to a terminal, the method comprises: Responding to a digital asset retrieval request input by a user, obtaining a terminal-cloud collaboration strategy from the cloud; When the end-cloud collaboration strategy indicates end-side processing, a vector data synchronization request is sent to the cloud, and an end-side vector database is updated according to a digital asset vector returned by the cloud in response to the vector data synchronization request; the digital asset vector is obtained by the cloud performing vectorization processing on multiple digital assets of the user; Obtaining a retrieval request vector obtained by vectorizing the digital asset retrieval request in the cloud; The search request vector is matched with a plurality of the digital asset vectors in the updated terminal-side vector database, and a target digital asset corresponding to the digital asset search request is obtained according to the matching result.

2. The method according to claim 1, characterized in that The step of obtaining the end-cloud collaboration strategy from the cloud in response to the digital asset retrieval request input by the user includes: Determine a current search scenario according to the digital asset search request; If the current search scenario is an image search scenario, an image recognition model component is obtained from the cloud, so as to perform image content recognition on the image carried in the digital asset search request through the image recognition model component, and determine the search target information according to the recognized image elements; If the current search scenario is a voice search scenario, a voice recognition model component is loaded from the cloud, so as to convert the request voice in the digital asset search request into a request text through the voice recognition model component, and determine the search target information according to the request text; If the current search scenario is a video search scenario, a video analysis model component is obtained from the cloud, so as to perform video content recognition on the video in the digital asset search request through the video analysis model component, and determine the search target information according to the recognized video elements; Acquire from the cloud a terminal-cloud collaboration strategy that matches the retrieval target information.

3. The method according to claim 2, characterized in that Each model component in the image recognition model component, the speech recognition model component and the video analysis model component is obtained from the cloud through the following steps: Sending a version query request of the terminal-side model component to the cloud to obtain a version query result; If the version query result indicates that there is a version to be updated, sending a request to the cloud for obtaining a download address of a model component corresponding to the version to be updated; Obtaining the download address sent by the cloud, and downloading the terminal-side model component from the cloud according to the download address; Perform a validity check on the downloaded end-side model component. If the validity check passes, install the downloaded end-side model component.

4. The method according to claim 2, characterized in that: The converting the request voice in the digital asset retrieval request into a request text by the speech recognition model component includes: Using the speech recognition model component, converting the request speech into text content corresponding to the digital asset retrieval request, and performing contextual adjustment on the text content to obtain a first recognized text; Sending the first recognized text and text correction request to the cloud; A second recognized text is obtained after correction of the first recognized text, and the second recognized text is determined as the request text.

5. The method according to any one of claims 1 to 4, characterized in that: The obtaining of the retrieval request vector obtained after the cloud performs vectorization processing on the digital asset retrieval request includes: Determine the search target information under various information modalities obtained according to the digital asset search request, and send a vectorized request carrying the search target information to the cloud; the vectorized request is used to trigger the cloud to map the search target information under various information modalities to the same feature vector space through a multimodal model to obtain the search request vector under the feature vector space; The search request vector returned by the cloud is received.

6. A digital asset retrieval method, characterized in that: Applied to the cloud, the method includes: In response to a terminal-cloud collaborative strategy acquisition request sent by the terminal, the terminal-cloud collaborative strategy is sent to the terminal; the terminal-cloud collaborative strategy acquisition request is sent by the terminal after receiving the digital asset retrieval request; In the case where the end-cloud collaboration strategy indicates end-side processing, in response to the terminal's vector data synchronization request, the digital asset vector corresponding to the user's digital asset is obtained and sent to the terminal, so that the terminal updates the end-side vector database according to the digital asset vector, and vectorizes the digital asset retrieval request to obtain a retrieval request vector, which is matched with the multiple digital asset vectors in the updated end-side vector data, and the target digital asset corresponding to the digital asset retrieval request is obtained according to the matching result.

7. A digital asset retrieval device, characterized in that: The device comprises: A collaborative strategy acquisition module is used to obtain a terminal-cloud collaborative strategy from the cloud in response to a digital asset retrieval request input by a user; A vector synchronization and update module, configured to send a vector data synchronization request to the cloud when the end-cloud collaboration strategy indicates end-side processing, and update the end-side vector database according to the digital asset vector returned by the cloud in response to the vector data synchronization request; the digital asset vector is obtained by the cloud performing vectorization processing on multiple digital assets of the user; A search request vectorization module, used to obtain a search request vector obtained after the cloud performs vectorization processing on the digital asset search request; The vector matching and asset identification module is used to match the search request vector with the multiple digital asset vectors in the updated terminal-side vector database, and obtain the target digital asset corresponding to the digital asset search request according to the matching result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the processor implements the steps of the digital asset retrieval method described in any one of claims 1 to 5 or the steps of the digital asset retrieval method described in claim 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the digital asset retrieval method described in any one of claims 1 to 5 or the steps of the digital asset retrieval method described in claim 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the digital asset retrieval method described in any one of claims 1 to 5 or the steps of the digital asset retrieval method described in claim 6 are implemented.

Images