Request processing method, apparatus, device, storage medium, and program product

Abstract

The present disclosure provides a request processing method, device, equipment, storage medium and program product. The method comprises: obtaining query data related to a query request, the query data comprising at least one type of query data; for each type in the at least one type, determining, based on the query request, a sampling strategy corresponding to the query data of the type from a plurality of sampling strategies; performing sampling on the query data of the at least one type based on the determined sampling strategy to obtain sampled query data of the at least one type; and determining, based on the query request and the sampled query data of the at least one type, a response to the query request using a trained first machine learning model. In this way, by using different sampling strategies for different types of query data, the processing efficiency of the machine learning model can be optimized, thereby reducing the energy consumption of the equipment.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computers, and particularly to request processing methods, apparatus, electronic devices, computer-readable storage media, and computer program products. Background Technology

[0002] With the development of information technology, various terminal devices can provide people with various services in work and life. For example, application services can be deployed on terminal devices. Terminal devices or applications can provide users with response functions for user query requests to assist users in using the terminal devices or applications. Terminal devices can receive query requests, execute the query requests to determine the response to the query requests, and provide the response to the user. Summary of the Invention

[0003] In a first aspect of this disclosure, a request processing method is provided. The method includes: acquiring query data related to a query request, the query data including at least one type of query data; for each of the at least one type, determining a sampling strategy corresponding to that type of query data from a plurality of sampling strategies based on the query request; performing sampling on the at least one type of query data respectively based on the determined sampling strategy to obtain sampled query data of at least one type; and determining a response to the query request using a trained first machine learning model based on the query request and the sampled query data of at least one type.

[0004] In a second aspect of this disclosure, an apparatus for task processing is provided. The apparatus includes: a query data acquisition module configured to acquire query data related to a query request, the query data including at least one type of query data; a sampling strategy determination module configured to, for each of the at least one type, determine a sampling strategy corresponding to the query data of that type from a plurality of sampling strategies based on the query request; a sampled query data acquisition module configured to perform sampling on the at least one type of query data based on the determined sampling strategy to obtain sampled query data of at least one type; and a response determination module configured to determine a response to the query request based on the query request and the sampled query data of at least one type using a trained first machine learning model.

[0005] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the electronic device to perform the method of the first aspect.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The medium stores a computer program that, when executed by a processor, implements the method of the first aspect.

[0007] In a fifth aspect of this disclosure, a computer program product is provided. The product includes a computer program, wherein when executed by a processor, the computer program implements the method according to a first aspect of this disclosure.

[0008] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0011] Figures 2A to 2D A schematic diagram of an example architecture for request processing according to some embodiments of the present disclosure is shown;

[0012] Figures 3A to 3D A schematic diagram of an example architecture for obtaining a dataset for training a machine learning model is shown according to some embodiments of the present disclosure;

[0013] Figure 4 A flowchart of a method for request processing according to some embodiments of the present disclosure is shown;

[0014] Figure 5 An exemplary structural block diagram of an apparatus for request processing according to some embodiments of the present disclosure is shown; and

[0015] Figure 6 A block diagram of an electronic device in which one or more embodiments of the present disclosure may be implemented is shown. Detailed Implementation

[0016] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0017] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0018] In this document, unless explicitly stated otherwise, performing a step in response to A does not mean that the step is performed immediately after A, but may include one or more intermediate steps.

[0019] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition, use, storage or deletion of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0020] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure through appropriate means in accordance with relevant laws and regulations, and user authorization should be obtained.

[0021] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly inform the user that the requested operation will require the acquisition and use of the user's personal information, thereby enabling the user to choose whether to provide personal information to the software or hardware such as electronic devices, applications, servers or storage media that perform the operation of the technical solution disclosed herein, based on the prompt message.

[0022] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, such as a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0023] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0024] As used in this paper, the term "model" refers to a model that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs using multiple layers of processing units. A neural network model is an example of a deep learning-based model. In this paper, "model" may also be referred to as a "machine learning model," "learning model," "machine learning network," or "learning network," and these terms are used interchangeably.

[0025] A neural network is a machine learning network based on deep learning. A neural network processes input and provides a corresponding output, typically consisting of an input layer, an output layer, and one or more hidden layers between the input and output layers. Neural networks used in deep learning applications often include many hidden layers, thus increasing the network's depth. The layers of a neural network are connected sequentially, so that the output of the previous layer is provided as the input to the next layer. The input layer receives the input to the neural network, while the output layer's output serves as the final output. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each node processing the input from the layer above.

[0026] Machine learning typically comprises three phases: training, testing, and application (also known as inference). In the training phase, a given model is trained using a large amount of training data, iteratively updating parameter values ​​until the model can consistently generate inferences that meet the expected goals from the training data. Through training, the model can be considered to have learned the relationship between inputs and outputs (also known as an input-output mapping) from the training data. The parameter values ​​of the trained model are determined. In the testing phase, test inputs are applied to the trained model to test whether it can provide the correct output, thus determining the model's performance. The testing phase can sometimes be integrated into the training phase. In the application or inference phase, the trained model can be used to process actual model inputs based on the trained parameter values ​​to determine the corresponding model output.

[0027] Figure 1A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. In this example environment 100, an application 112 is installed on a terminal device 110. A user 140 can interact with the application 112 via the terminal device 110 and / or an attached device of the terminal device 110. For example, the application 112 can capture the user 140's voice via a voice capture component (e.g., a microphone) of the terminal device 110, capture images or videos of the user 140 via an image capture component (e.g., a camera) of the terminal device 110, capture the user 140's posture information via sensors (e.g., a gyroscope) of the terminal device 110, and so on.

[0028] In embodiments of this disclosure, application 112 can be any suitable application with request processing capabilities. For example, application 112 can be a social application, a chat application, a media application, and so on. In some embodiments, application 112 can provide a digital assistant for human-computer dialogue. The digital assistant supports text dialogue services, voice dialogue services, and content dialogue in other modalities with user 140. In some embodiments, application 112 or the digital assistant therein can utilize machine learning models to assist in providing one or more services. For example, application 112 or the digital assistant therein can utilize machine learning models to provide question-and-answer services to user 140. The digital assistant's response to the user can be determined based on the model output of the machine learning model.

[0029] In some embodiments, one or more machine learning models 114-1, 114-2, ..., 114-N (collectively or individually referred to as machine learning models 114) may be deployed locally on the terminal device 110. These machine learning models 114 may be configured to determine or assist in determining a response to a user. In some embodiments, one or more machine learning models 130-1, 130-2, ..., 130-M (collectively or individually referred to as machine learning models 130) may also be deployed at the server device 120. These machine learning models 130 may also be configured to determine or assist in determining a response to a user.

[0030] Both machine learning model 114 and machine learning model 130 can be based on any suitable model architecture, including but not limited to Transformer models, convolutional neural networks (CNNs), recurrent neural networks (RNNs), deep neural networks (DNNs), and so on. In some embodiments, one or more machine learning models 114 and / or machine learning model 130 can be based on language models (LMs), multimodal language models, etc. Language models, by learning from large corpora, are capable of question answering. Multimodal language models can support the processing of data from multiple modalities (e.g., text, audio, images, video, sensor data, etc.).

[0031] In some embodiments, the language model-based machine learning model is capable of receiving text-modal model input (e.g., natural language and / or machine language) and / or non-text-modal model input (e.g., images, speech, video, etc.), and is capable of generating the desired output based on the model input and prompt words. Here, the prompt words are used to guide the machine learning model to generate a model output that addresses the user's needs indicated by the model input. In application scenarios supporting user dialogue, the input of user 140 can be provided to machine learning model 114 and / or machine learning model 130 as at least a part of the model input (other parts may include prompt words).

[0032] It should be noted that both machine learning model 114 and machine learning model 130 may include one or more machine learning models. If multiple machine learning models are included, their functions, structures, and uses may be the same or different.

[0033] In environment 100, if application 112 is active, terminal device 110 can present the user interface of application 112 (e.g., interface 150). Interface 150 may include various interfaces that application 112 can provide, such as a dialogue interface between a user and a digital assistant (where the current conversation and historical conversations, including text conversation content, can be presented), and so on. In some embodiments, terminal device 110 may play voice via interface 150, and the voice may include voice questions from the user and voice responses to the questions.

[0034] In some embodiments, terminal device 110 communicates with server device 120 to provide services to application 112. For example, server device 120 may invoke machine learning model 130 to support human-computer dialogue between application 112 and user 140 based on the output of machine learning model 130.

[0035] Terminal device 110 can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio broadcast receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, terminal device 110 may also support any type of user-facing interface (such as "wearable" circuitry).

[0036] Server-side device 120 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. Server-side device 120 may include, for example, computing systems / servers such as mainframes, edge computing nodes, computing devices in a cloud environment, etc. Server-side device 120 may be implemented based on a cloud environment.

[0037] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0038] As mentioned earlier, the terminal device can receive query requests in response to a query, execute the query to determine a response, and provide that response to the user. Typically, multimodal models can be used to process query-related data, including the user's query request and other query data related to determining the response, such as images, audio, video, and context-sensitive sensor measurements. Usually, the acquired high-resolution and densely sampled raw query data is directly fed to the model for processing. While this allows the model to capture more detail, it significantly increases computational complexity, device power consumption, and the amount of data transmitted.

[0039] Furthermore, the limited computing resources can be addressed by deploying and running machine learning models locally, or by combining locally deployed machine learning models with those on server-side devices. However, with locally deployed models, the large volume of data places higher demands on local computing resources and device power consumption. In scenarios where machine learning models are used with those on server-side devices, excessive data volume can also slow down the inference speed of the server-side models. Additionally, the requirement for models to process high-resolution data presents a challenge to the model training process.

[0040] In view of the above, according to embodiments of the present disclosure, an improved scheme for request processing is provided. According to the scheme of the embodiments of the present disclosure, query data related to a query request is obtained, the query data including at least one type of query data. Further, for each of the at least one type, based on the query request, a sampling strategy corresponding to that type of query data is determined from a plurality of sampling strategies. Sampling is performed on the at least one type of query data based on the determined sampling strategy to obtain sampled query data of at least one type. Then, based on the query request and the sampled query data of at least one type, a trained first machine learning model is used to determine the response to the query request.

[0041] In this way, an appropriate sampling strategy for the query data can be dynamically determined based on the current query request. This sampling strategy can be determined to ensure the quality of the response to the query request while reducing the amount of data. Thus, when determining the response, the machine learning model only needs to process the downsampled query data instead of the original query data. This optimizes the inference efficiency of the machine learning model, improves the response speed to user query requests, and thus enhances the user experience. Processing less data also reduces device power consumption. Furthermore, since the sampling strategy is flexibly determined for different query requests, the impact on the quality of the generated query response can be minimized.

[0042] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0043] Figures 2A to 2D Schematic diagrams of example architectures 200A to 200D for request processing according to some embodiments of the present disclosure are shown. Example architectures 200A to 200D can be implemented at terminal device 110. For ease of discussion, reference will be made to... Figure 1 The example architectures 200A to 200D are described using environment 100. It should be noted that the operations performed by the aforementioned terminal device 110, and the operations performed by the terminal device 110 as described below, may specifically be performed by relevant applications (e.g., application 112) installed on the terminal device 110. In some embodiments, the operations performed on the terminal device 110 may be completed with the assistance of the server device 120.

[0044] The following description of application scenarios of this disclosure will be based on examples for ease of understanding. If terminal device 110 receives a query request (e.g., a question-and-answer request) and query data associated with the user's query request (e.g., images, videos, audio, etc.) from user 140, it can provide user 140 with a response corresponding to the query request. For example, if terminal device 110 receives the message "Please summarize the main content of video A" from user 140, it can provide user 140 with a response to the query request. The above example scenarios are merely illustrative and are not intended to limit the scope of this disclosure. Reference will be made first below to... Figures 2A to 2D Describe in detail how terminal device 110 can efficiently provide users with responses to query requests.

[0045] In embodiments of this disclosure, terminal device 110 acquires query data related to a query request, the query data including at least one type of query data. (Refer to...) Figure 2A In the example architecture 200A shown, terminal device 110 can obtain query request 202 input by user 140 and query data 204 associated with query request 202.

[0046] In some embodiments, terminal device 110 may receive query requests 202 from a user (e.g., user 140) in any suitable manner. For example, terminal device 110 may receive query requests 202 in voice form input by user 140 via a microphone. Terminal device 110 may receive query requests 202 in text form input by user 140 via an input field. In some embodiments, query request 202 may include user questions to the digital assistant. Terminal device 110 receives user questions during user interaction with the digital assistant.

[0047] In embodiments of this disclosure, multiple optional sampling strategies can be configured for different types of query data, and an appropriate sampling strategy can be selected each time based on the query request. In some embodiments, the multiple sampling strategies can be configured to sample different amounts of data from the query data. The multiple sampling strategies can be configured based on the type of query data. (Refer to...) Figure 2AThe query data 204 associated with query request 202 can be image-type query data, such as image data 204-1. The query data 204 associated with query request 202 can also be video-type query data, such as video data 204-1. The query data 204 associated with query request 202 can also be audio-type query data, such as audio data 204-3. The query data 204 associated with query request 202 can also be measurement-type query data, such as sensor measurement data 204-2 acquired via an inertial measurement unit (IMU). It is understood that image data / video data 204-1, sensor measurement data 204-2, audio data 204-3, etc., can be collectively referred to as or individually referred to as query data 204.

[0048] In some implementations, where query data 204 includes image data of the image type or video data of the video type, multiple sampling strategies can indicate multiple resolutions. For example... Figure 2B In the example architecture 200B shown, when the query data 204 includes image data of image type or video data of video type, the multiple resolutions used to determine the sampling strategy for the image type / video type query data can include 480P (e.g., ...). Figure 2B 221-1), 720P (such as...) Figure 2B 221-2), 1080P (such as...) Figure 2B 221-3), 1440P (e.g.) Figure 2B (e.g., 221-4) resolutions.

[0049] In some embodiments, where the query data 204 includes video data of the video type, the sampling strategy used to determine the query data of the video type may also indicate a sampling interval. For example, the terminal device 110 may divide the query data of the video type into X parts and randomly select one frame for each part. The terminal device 110 may also extract one frame at intervals of X frames in consecutive frames.

[0050] In other embodiments, when query data 204 includes audio data or measurement data of sensor measurement type, multiple sampling strategies can indicate multiple sampling frequencies. For example... Figure 2C In the example architecture 200C shown, when the query data 204 includes measurement data of the sensor measurement data type, the multiple sampling frequencies used to determine the sampling strategy for the sensor measurement data 204-2 may, for example, include 100Hz (e.g., Figure 2C 231-1), 250Hz (e.g.) Figure 2C 231-2), 500Hz (e.g.) Figure 2C 231-3), 1000Hz (e.g.) Figure 2CSampling frequencies such as 231-4, etc. Figure 2D In the example architecture 200D shown, when query data 204 includes audio data of audio type, multiple sampling frequencies used to determine the sampling strategy for audio data 204-3 may, for example, include 48kHz (e.g., Figure 2D 241-1), 22.05kHz (e.g.) Figure 2D 241-2), 16kHz (e.g.) Figure 2D 241-3), 8kHz (e.g.) Figure 2D Sampling frequencies such as 241-4, etc.

[0051] In some embodiments, one of a plurality of sampling strategies may instruct that no downsampling be performed on the query data. Other sampling strategies may instruct a lower resolution than the original resolution of the query data (for images or videos), or a lower sampling frequency than the default sampling frequency (for audio or sensor measurement data). These sampling strategies may instruct a downsampling strategy for the query data, thereby reducing the amount of data to be transmitted to subsequent machine learning models. In some embodiments, the plurality of sampling strategies may all include a lower resolution than the original resolution of the query data (for images or videos), or a lower sampling frequency than the default sampling frequency (for audio or sensor measurement data).

[0052] Depending on the type of query data, terminal device 110 determines the sampling strategy corresponding to that type of query data from a plurality of sampling strategies applicable to that type, based on the query request. In some embodiments, a trained machine learning model (referred to as a "second machine learning model") may be used to determine the corresponding sampling strategy for different types of query data. See also... Figure 2A In the example architecture 200A shown, for each type of query data 204, the terminal device 110 can determine the sampling strategy corresponding to that type of query data 204 based on the query request 202 by utilizing a trained machine learning model 114 from sampling strategies 211-1, 211-2, ..., 211-N (collectively or individually referred to as sampling strategies 211). For example, the terminal device 110 can determine the sampling strategy corresponding to the query data 204 (e.g., image A) associated with the query request 202 (e.g., please summarize the main content of image A) input by the user 140. For example, the terminal device 110 can determine that the sampling strategy for image A is to downsample image A to 720p.

[0053] In some embodiments, for different types of query data, different machine learning models 114 can be pre-trained to perform sampling strategy determination for the corresponding type of query data. The terminal device 110 can determine the sampling strategy corresponding to that type of query data from multiple sampling strategies in the following manner: Specifically, the terminal device 110 determines a machine learning model 114 corresponding to a given type among at least one type. Accordingly, based on the query request, the terminal device 110 uses the determined machine learning model 114 to determine the sampling strategy corresponding to the query data of the given type from multiple sampling strategies for the given type.

[0054] Understandably, terminal device 110 can invoke a machine learning model corresponding to each type of query data to determine the sampling strategy for each type of query data. For example, for image-type query data, terminal device 110 can invoke machine learning model A to determine the sampling strategy for image-type query data. For sensor measurement-type query data, terminal device 110 can invoke machine learning model B to determine the sampling strategy for sensor measurement-type query data. For audio-type query data, terminal device 110 can invoke machine learning model C to determine the sampling strategy for audio-type query data. In other embodiments, for at least one type of query data, terminal device 110 can invoke machine learning models to determine the sampling strategy corresponding to at least one type of query data respectively.

[0055] In some embodiments, machine learning model 114 can be deployed locally on terminal device 110 to determine the sampling strategy for query data. Since machine learning model 114 only needs to perform classification among multiple sampling strategies to determine the sampling strategy matching the current query request, the model size of such machine learning models is typically not large, making them suitable for running locally on the terminal device without consuming excessive resources. Furthermore, in some embodiments, by utilizing machine learning model 114 to determine the downsampling strategy for query data, the amount of data to be transmitted to subsequent machine learning model 130 can be reduced. This can improve the inference speed of subsequent machine learning models and reduce inference overhead. If subsequent machine learning model 130 is deployed on a server device rather than on the terminal device, reducing the amount of data to be transmitted over the network in advance on the local terminal device can further reduce network overhead and accelerate data transmission speed, thereby further improving the efficiency of responding to query requests.

[0056] In embodiments of this disclosure, terminal device 110 performs sampling on at least one type of query data according to a sampling strategy to obtain at least one type of sampled query data. In some embodiments, terminal device 110 uses machine learning model 114 to determine a sampling strategy for a certain type of query data from multiple sampling strategies. Subsequently, terminal device 110 can perform sampling on query data 204 by calling sampling unit 205 according to its determined sampling strategy to determine at least one type of sampled query data.

[0057] For example, in a scenario where the terminal device 110 determines that the sampling strategy for image A is to downsample image A to 720p, the terminal device 110 can send this sampling strategy as an instruction to the sampling unit 205. Accordingly, after receiving the instruction, the sampling unit 205 can downsample the received image A to 720p.

[0058] In this embodiment of the disclosure, terminal device 110 determines the response to the query request using a trained machine learning model 130 based on the query request and sampled query data of at least one type. (Refer to...) Figure 2A Terminal device 110 can determine the response 208 to query request 202 based on query request 202 and sampled query data, and by using machine learning model 206.

[0059] Continuing with the example above, terminal device 110 sends its downsampled 720p image acquired via sampling unit 205 and query request 202 (e.g., please summarize the main content of image A) to machine learning model 206. Subsequently, terminal device 110 receives a response 208 to the query request determined by machine learning model 206. It should be understood that machine learning model 206 can be machine learning model 114 deployed locally on terminal device 110, or machine learning model 130 deployed on server device 120.

[0060] In some embodiments, if the machine learning model 206 is deployed on a server device, the terminal device 110 can send a query request and sampled query data to the server device. The server device provides the query request and sampled query data to the machine learning model 130 to determine the model output. The server device can determine a response based on the model output and process the request for the terminal device, or the server device can provide the model output to the terminal device, and the terminal device can determine the response.

[0061] In some embodiments, the sampling of query data can also be implemented on the server-side device. After determining the sampling strategy, the terminal device can send the determined sampling strategy, query request, and query data to the server-side device. The server-side device can perform sampling on the query data based on the sampling strategy and provide the sampled query data and query request to the machine learning model 206 to determine the model output.

[0062] The following will be combined for ease of understanding Figures 2B to 2D The terminal device 110 determines the response to the query request, and some examples are given to illustrate this.

[0063] Reference Figure 2B The example architecture 200B is shown, and the terminal device 110 determines the response to the query request in a scenario where query data 204 is of image type. When query data 204 includes image data of image type or video data of video type, the multiple resolutions used to determine the image / video type of the query data may include 480P (e.g., ...). Figure 2B 221-1), 720P (such as...) Figure 2B 221-2), 1080P (such as...) Figure 2B 221-3), 1440P (e.g.) Figure 2B Resolutions such as 221-4) are provided. Of course, it's important to understand that these are just some examples of resolutions; in practice, any other appropriate resolution can be configured as needed. Furthermore, the number of selectable resolutions is also configurable.

[0064] Accordingly, for image data 204-1, terminal device 110 can, based on query request 202 (e.g., what content is contained in image A), utilize a trained machine learning model 114-1 to analyze the image data from multiple resolutions (e.g., 480P). Figure 2B 221-1), 720P (such as...) Figure 2B 221-2), 1080P (such as...) Figure 2B 221-3), 1440P (e.g.) Figure 2B From (221-4), it can be determined that the sampling strategy for image A is to downsample image A to 1080P (e.g., ...). Figure 2B (221-3).

[0065] Subsequently, the terminal device 110 can send the sampling strategy as an instruction to the image sampling unit 205-1. Correspondingly, after receiving the instruction, the image sampling unit 205-1 can downsample the received A image to 1080P (e.g., ...). Figure 2B(See section 221-3). The terminal device 110 sends the downsampled 1080p image acquired by the image sampling unit 205-1 and the query request 202 to the machine learning model 206. Subsequently, the terminal device 110 receives the response 224 to the query request determined by the machine learning model 206 (e.g., image A contains a certain object).

[0066] Reference Figure 2C The example architecture 200C is shown, and the terminal device 110 is described in a scenario where query data 204 is the data type of sensor measurement data to determine the response to the query request. In the case that query data 204 includes image data of the image type or video data of the video type, multiple sampling frequencies used to determine the sampling strategy for sensor measurement data 204-2 may, for example, include 100Hz (e.g., ...). Figure 2C 231-1), 250Hz (e.g.) Figure 2C 231-2), 500Hz (e.g.) Figure 2C 231-3), 1000Hz (e.g.) Figure 2C Sampling frequencies such as (e.g., 231-4) are provided. It's important to understand that this only provides a few examples of sampling frequencies; in practical applications, any other appropriate sampling frequency can be configured as needed. Furthermore, the number of selectable sampling frequencies is also configurable.

[0067] Accordingly, for sensor measurement data 204-2, terminal device 110 can, based on query request 202 (e.g., what user A's gesture indicates), utilize a trained machine learning model 114-2 to sample data from multiple sampling frequencies of 100Hz (e.g., ...). Figure 2C 231-1), 250Hz (e.g.) Figure 2C 231-2), 500Hz (e.g.) Figure 2C 231-3), 1000Hz (e.g.) Figure 2C From (231-4), it can be determined that the sampling strategy for sensor measurement data 204-2 is to downsample sensor measurement data 204-2 to 500Hz (e.g., ...). Figure 2C (231-3).

[0068] Subsequently, the terminal device 110 can send the sampling strategy as an instruction to the sensor sampling unit 205-2. Correspondingly, after receiving the instruction, the sensor sampling unit 205-2 can downsample the received sensor measurement data 204-2 to 500Hz (e.g., ...). Figure 2C(See section 231-3). The terminal device 110 sends the downsampled 500Hz sensor measurement data acquired via the sensor sampling unit 205-2 and the query request 202 to the machine learning model 206. Subsequently, the terminal device 110 receives the response 234 to the query request determined by the machine learning model 206 (e.g., user A's gesture indicating to the right).

[0069] Reference Figure 2D The example architecture 200D is shown, and the terminal device 110 determines the response to the query request in the scenario where query data 206 is audio type. When query data 204 includes audio data of audio type, the multiple sampling frequencies used to determine the sampling strategy for audio data 204-3 may, for example, include 48kHz (e.g., ...). Figure 2D 241-1), 22.05kHz (e.g.) Figure 2D 241-2), 16kHz (e.g.) Figure 2D 241-3), 8kHz (e.g.) Figure 2D Sampling frequencies such as (e.g., 241-4) are provided. Of course, it's important to understand that these are just a few examples of sampling frequencies; in practical applications, any other appropriate sampling frequency can be configured as needed. Furthermore, the number of selectable sampling frequencies is also configurable.

[0070] Accordingly, for audio data 204-3, terminal device 110 can, based on query request 202 (e.g., convert audio A to text), utilize a trained machine learning model 114-3 to extract data from multiple sampling frequencies of 48kHz (e.g., ...). Figure 2D 241-1), 22.05kHz (e.g.) Figure 2D 241-2), 16kHz (e.g.) Figure 2D 241-3), 8kHz (e.g.) Figure 2D In section 241-4, the sampling strategy for audio data 204-3 is determined to be downsampling audio data 204-3 to 22.05kHz (e.g., ...). Figure 2D (241-2).

[0071] Subsequently, the terminal device 110 can send the sampling strategy as an instruction to the audio sampling unit 205-3. Correspondingly, after receiving the instruction, the audio sampling unit 205-3 can downsample the received audio data 204-3 to 22.05kHz (e.g., ...). Figure 2D (241-2). Terminal device 110 sends the downsampled 22.05kHz audio obtained via audio sampling unit 205-3 and query request 202 to machine learning model 206. Subsequently, terminal device 110 receives response 243 to the query request determined by machine learning model 206.

[0072] Therefore, by employing different sampling strategies for different types of query data, this disclosure can optimize the processing efficiency of machine learning models, thereby reducing device energy consumption. Furthermore, this disclosure can improve the response speed of machine learning models to user query requests, thereby enhancing the user experience.

[0073] The above combination Figures 2A to 2D The application of machine learning model 114 has been described. The following section will continue to combine... Figures 3A to 3D This describes the process of obtaining the dataset used to train the machine learning model 114. Figures 3A to 3D Schematic diagrams of example architectures 300A to 300D for obtaining datasets for training machine learning models according to some embodiments of the present disclosure are shown. It should be noted that the machine learning model 114 can be trained on terminal device 110, server device 120, or any other suitable electronic device; however, in this document, the example of machine learning model 114 being trained on terminal device 110 is used only as an example.

[0074] In some embodiments, terminal device 110 can train machine learning model 114 based on a training dataset. The training dataset may include multiple training samples, each including a query request sample and a sampled query data sample. In some embodiments, the sampled query data sample is obtained by sampling the original query data sample using one of multiple sampling strategies corresponding to the type of query data sample. The following describes how terminal device 110 creates a training dataset for training machine learning model 114.

[0075] In some embodiments, during the creation of a training dataset, the terminal device 110 may use multiple query request samples to generate corresponding training samples. Each training sample includes a query data sample obtained by sampling the original query data sample under a specific sampling strategy. This sampling strategy is considered more appropriate for the current query request sample and can ensure the accuracy of the final generated response while reducing the amount of query data.

[0076] Reference Figure 3A When creating a training dataset, the terminal device 110 first obtains query request samples 311 and original query data samples 312. In some embodiments, query request samples 311 may include text query requests. In some embodiments, the terminal device 110 uses multiple sampling strategies corresponding to the types of the original query data samples to sample the original query data samples respectively, so as to obtain multiple sampled candidate query data samples.

[0077] like Figure 3AAs shown, terminal device 110 can determine multiple sampling strategies 311-1, 311-2, ..., 311-N (collectively referred to as sampling strategies 311 or individually) corresponding to a given type of original query data sample. In some examples, the multiple sampling strategies corresponding to each type of original query data sample can be pre-configured by the user. Subsequently, terminal device 110 can call sampling unit 310 to sample the given type of original query data sample according to the multiple sampling strategies 311-1, 311-2, ..., 311-N respectively, to obtain multiple sampled candidate query data samples.

[0078] Accordingly, for each candidate query data sample among multiple candidate query data samples, terminal device 110 determines a predicted response to the query request based on the query request sample and the candidate query data sample using a trained third machine learning model. In some examples, for each candidate query data sample among multiple candidate query data samples, terminal device 110 may transmit the query request sample 311 and the candidate query data sample to a trained machine learning model 313 (referred to as the "third machine learning model"). Subsequently, terminal device 110 obtains N predicted responses 314 for the query request 311 determined by machine learning model 313. In some examples, machine learning model 313 may be a trained machine learning model 130, or a different machine learning model than machine learning model 130, which is capable of determining an accurate model output based on model input.

[0079] Further, the terminal device 110 determines the quality scores of each of the multiple predicted responses corresponding to the multiple candidate query data samples. In some embodiments, the terminal device 110 can use a machine learning model 130 to determine a reference response for the query request sample based on the query request sample and the original query data sample. Then, the terminal device 110 determines the quality scores of each of the multiple predicted responses based on the differences between the multiple predicted responses and the reference response.

[0080] like Figure 3AIn the example frame 300A shown, terminal device 110 can obtain a reference response 318 by invoking machine learning model 313 based on query request sample 311 and original query data sample 312. Understandably, terminal device 110 obtains the reference response 318 based on user 140's query request and the original query data sample 312 without downsampling. In box 315, terminal device 110 can compare N predicted responses 314 with the reference response 318. In box 316, terminal device 110 determines the quality score of each of the N predicted responses 314 based on the differences between them and the reference response 318. In some examples, terminal device 110 can determine the differences between the N predicted responses 314 and the reference response 318 through semantic similarity. For example, terminal device 110 determines the differences between the N predicted responses 314 and the reference response 318 based on the similarity of the average / weighted values ​​of the text vectors.

[0081] Terminal device 110 can determine the differences between N predicted responses 314 and reference responses 318 through set comparison. For example, terminal device 110 determines the differences between N predicted responses 314 and reference responses 318 based on whether the predicted responses and reference responses belong to the correct set. Terminal device 110 can also determine the differences between N predicted responses 314 and reference responses 318 through string matching. Terminal device 110 can also determine the differences between N predicted responses 314 and reference responses 318 through accuracy.

[0082] In some embodiments, the terminal device 110 selects a sampled query data sample corresponding to the query request sample from multiple candidate query data samples based on the quality scores of each of the multiple predicted responses. In block 317, the terminal device 110 determines the predicted response with the higher quality score (e.g., predicted response A) among the N predicted responses 314 based on the quality scores of each of the N predicted responses 314. Further, the terminal device 110 can use the training sample corresponding to predicted response A as the training sample for training the machine learning model 114.

[0083] This disclosure utilizes multiple sampling strategies to sample the original query data samples separately, resulting in multiple sampled candidate query data samples. This allows the training samples in the training dataset to cover multiple sampling strategies. Using such a training dataset, training samples corresponding to different sampling strategies can be obtained, enabling a second machine learning model to learn and determine the appropriate sampling strategy for different query requests.

[0084] The following will be combined for ease of understanding Figures 3B to 3D The document describes, with some examples, how terminal device 110 creates a training dataset for training machine learning models.

[0085] Reference Figure 3B The example architecture shown is 300B, and the training dataset is created for terminal device 110 in a scenario where the original query data samples are of image type. In this case, the multiple resolutions of the image / video type query data samples can include 480P (e.g., ...). Figure 3B 324-1), 720P (such as...) Figure 3B 324-2), 1080P (such as...) Figure 3B 324-3), 1440P (such as...) Figure 3B Resolutions such as 324-4 (e.g., 424-4) are available. In some examples, users can set the original resolution of the images in each sample to the upper limit resolution of the training data set, and gradually reduce the resolution in multiple stages. Terminal device 110 can call image sampling unit 310-1 at multiple resolutions such as 480P (e.g., 424-4). Figure 3B 324-1), 720P (such as...) Figure 3B 324-2), 1080P (such as...) Figure 3B 324-3), 1440P (such as...) Figure 3B In step 324-4), the original image data sample 312-1 is sampled to obtain the sampled candidate query data samples A, B, C, and D.

[0086] Accordingly, the terminal device 110 sends the A candidate query data sample, B candidate query data sample, C candidate query data sample, D candidate query data sample, and query request sample 311 acquired via the image sampling unit 310-1 to the machine learning model 313. The terminal device 110 receives N predicted responses 325 for the query request sample 311 determined by the machine learning model 313. For example, the A predicted response corresponding to the sampled A candidate query data sample, the B predicted response corresponding to the sampled B candidate query data sample, the C predicted response corresponding to the sampled C candidate query data sample, and the D predicted response corresponding to the sampled D candidate query data sample.

[0087] Subsequently, terminal device 110 can obtain reference response 318-1 by calling machine learning model 313 based on query request sample 311 and original image data sample 312-1. In box 326, terminal device 110 can compare N predicted responses 325 with reference response 318-1. In box 327, terminal device 110 determines the quality scores of predicted responses A, B, C, and D based on the differences between the N predicted responses 325 and reference response 318-1. In box 328, terminal device 110 determines the predicted response with the higher quality score among the N predicted responses 325 (e.g., predicted response C) based on the quality scores of the N predicted responses 325. Further, terminal device 110 can use the 1080p image corresponding to predicted response C, query request sample 311, and reference response 318-1 as training samples for training machine learning model 114.

[0088] Reference Figure 3C The example architecture shown is 300C, and the scenario describes how the terminal device 110 creates a training dataset using raw query data samples as the sensor measurement data type. In this case, the query data samples of the sensor measurement data type have multiple sampling frequencies of 100Hz (e.g., Figure 3C 334-1), 250Hz (e.g.) Figure 3C 334-2), 500Hz (e.g.) Figure 3C 334-3), 1000Hz (e.g.) Figure 3C Sampling frequencies such as 334-4, etc. In some examples, the user can set the original IMU Hertz (Hz) in each sample as the upper limit Hertz (Hz) for that set of data, and gradually decrease the Hertz (Hz) in multiple stages. The terminal device 110 can call the sensor sampling unit 310-2 at multiple sampling frequencies of 100Hz (e.g., ... Figure 3C 334-1), 250Hz (e.g.) Figure 3C 334-2), 500Hz (e.g.) Figure 3C 334-3), 1000Hz (e.g.) Figure 3C In step 334-4), the original sensor data sample 312-2 is sampled to obtain the sampled AA candidate query data sample, BB candidate query data sample, CC candidate query data sample, and DD candidate query data sample.

[0089] Accordingly, the terminal device 110 sends the AA candidate query data sample, BB candidate query data sample, CC candidate query data sample, DD candidate query data sample, and query request sample 311 acquired by the sensor sampling unit 310-2 to the machine learning model 313. The terminal device 110 receives N predicted responses 335 for the query request sample 311 determined by the machine learning model 313. For example, the AA predicted response corresponding to the sampled AA candidate query data sample, the BB predicted response corresponding to the sampled BB candidate query data sample, the CC predicted response corresponding to the sampled CC candidate query data sample, and the DD predicted response corresponding to the sampled A candidate query data sample.

[0090] Subsequently, terminal device 110 can obtain reference response 318-2 by calling machine learning model 313 based on query request sample 311 and raw sensor data sample 312-2. In box 336, terminal device 110 can compare N predicted responses 335 with reference response 318-2. In box 337, terminal device 110 determines the quality scores of AA predicted response, BB predicted response, CC predicted response, and DD predicted response based on the differences between the N predicted responses 335 and reference response 318-2. In box 338, terminal device 110 determines the predicted response with the higher quality score among the N predicted responses 335 (e.g., CC predicted response) based on the quality scores of the N predicted responses 335. Further, terminal device 110 can use the 500Hz corresponding to CC predicted response, query request sample 311, and reference response 318-2 as training samples for training machine learning model 114.

[0091] Reference Figure 3D The example architecture shown is 300D, and a training dataset is created for terminal device 110 to describe a scenario where the original query data samples are of the audio type. In this case, multiple sampling frequencies of the audio type query data samples may include 48kHz (e.g., Figure 3D 344-1), 22.05kHz (e.g.) Figure 3D 344-2), 16kHz (e.g.) Figure 3D 344-3), 8kHz (e.g.) Figure 3D Sampling frequencies such as 344-4, etc. In some examples, the user can set the original Hertz (Hz) of the audio in each sample to the upper limit Hertz (Hz) of that set of training data, and gradually decrease the Hertz (Hz) in multiple stages. The terminal device 110 can call the audio sampling unit 310-3 at multiple sampling frequencies 48kHz (e.g., ... Figure 3D 344-1), 22.05kHz (e.g.) Figure 3D 344-2), 16kHz (e.g.) Figure 3D344-3), 8kHz (e.g.) Figure 3D In section 344-4), the original audio data samples 312-3 are sampled respectively to obtain the sampled candidate query data samples E, F, G, and H.

[0092] Accordingly, terminal device 110 sends the E-candidate query data samples, F-candidate query data samples, G-candidate query data samples, H-candidate query data samples, and query request sample 311 acquired via audio sampling unit 310-3 to machine learning model 313. Terminal device 110 receives N predicted responses 345 for query request sample 311 determined by machine learning model 313. For example, the E-predicted response corresponding to the sampled E-candidate query data sample, the F-predicted response corresponding to the sampled F-candidate query data sample, the G-predicted response corresponding to the sampled G-candidate query data sample, and the H-predicted response corresponding to the sampled H-candidate query data sample.

[0093] Subsequently, terminal device 110 can obtain reference responses 318-3 by calling machine learning model 313 based on query request sample 311 and original audio data samples 312-3. In box 346, terminal device 110 can compare N predicted responses 345 with reference responses 318-3. In box 347, terminal device 110 determines the quality scores of each of the E, F, G, and H predicted responses based on the differences between the N predicted responses 345 and reference responses 318-3. In box 348, terminal device 110 determines the predicted response with the higher quality score among the N predicted responses 345 (e.g., G predicted response) based on the quality scores of each of the N predicted responses 345. Further, terminal device 110 can use the 22.05kHz corresponding to G predicted response, query request sample 311, and reference response 318-3 as training samples for training machine learning model 114.

[0094] The training samples in the training dataset obtained in this way can cover multiple sampling strategies. Using such a training dataset, training samples corresponding to different sampling strategies can be obtained, enabling the second machine learning model to learn and determine the appropriate sampling strategy for different query requests.

[0095] In some embodiments, terminal device 110 trains machine learning model 114 based on a created training dataset to provide responses to user queries. In some examples, because machine learning model 114 becomes large after being trained on a large training dataset, it is difficult to use it directly as a local small model. Therefore, terminal device 110 can also perform model miniaturization (i.e., knowledge distillation, model pruning) on ​​machine learning model 114 to make the compressed model more lightweight and have more efficient inference capabilities while retaining the performance of machine learning model 114. Furthermore, terminal device 110 can also fine-tune the distilled machine learning model using a portion of the data in the training dataset to further maintain the performance of the miniaturized model, thereby improving the model's expressiveness.

[0096] In summary, according to the various embodiments of this disclosure, different sampling strategies for different types of query data can be determined by using a trained machine learning model, thereby optimizing the processing efficiency of the machine learning model and reducing device power consumption. Furthermore, through the embodiments of this disclosure, the response speed of the machine learning model to user query requests can be improved while enhancing the machine learning model's understanding ability and response quality, thereby improving the user experience.

[0097] Figure 4 A flowchart of a method 400 for request processing according to some embodiments of the present disclosure is shown. Method 400 may be implemented at terminal device 110.

[0098] In box 410, terminal device 110 acquires query data related to the query request, the query data including at least one type of query data.

[0099] In box 420, for each of at least one type, terminal device 110 determines the sampling strategy corresponding to the query data of that type from multiple sampling strategies based on the query request.

[0100] In box 430, terminal device 110 performs sampling on at least one type of query data based on the determined sampling strategy to obtain at least one type of sampled query data.

[0101] In box 440, terminal device 110 uses a trained first machine learning model to determine the response to the query request based on the query request and at least one type of sampled query data.

[0102] In some embodiments, at least one type includes one or more of the following: image type, video type, audio type, measurement data type.

[0103] In some embodiments, the query data includes image data of the image type or video data of the video type, and multiple sampling strategies indicate multiple resolutions; and / or the query data includes audio data of the audio type or measurement data of the sensor measurement type, and multiple sampling strategies indicate multiple sampling frequencies.

[0104] In some embodiments, determining the sampling strategy corresponding to the type of query data from multiple sampling strategies includes: based on the query request, using a trained second machine learning model to determine the sampling strategy corresponding to the type of query data from multiple sampling strategies.

[0105] In some embodiments, process 400 is implemented at a terminal device, and the second machine learning model is deployed locally on the terminal device.

[0106] In some embodiments, the second machine learning model is trained based on a training dataset, which includes multiple training samples. Each training sample includes a query request sample and a sampled query data sample. The sampled query data sample is obtained by sampling the original query data sample using one of multiple sampling strategies corresponding to the type of query data sample.

[0107] In some embodiments, the training samples in the training dataset are trained by: sampling the original query data samples using multiple sampling strategies corresponding to the types of the original query data samples to obtain multiple sampled candidate query data samples; for each candidate query data sample, determining the predicted response for the query request sample using a trained third machine learning model based on the query request sample and the candidate query data sample; determining the quality scores of each of the multiple predicted responses corresponding to the multiple candidate query data samples; and selecting the sampled query data sample corresponding to the query request sample from the multiple candidate query data samples based on the quality scores of each of the multiple predicted responses.

[0108] In some embodiments, determining the quality scores of the multiple predicted responses corresponding to the multiple candidate query data samples includes: using a first machine learning model to determine a reference response for the query request sample based on the query request sample and the original query data sample; and determining the quality scores of the multiple predicted responses based on the differences between the multiple predicted responses and the reference response.

[0109] In some embodiments, determining the sampling strategy corresponding to the type of query data from a plurality of sampling strategies includes: for a given type in at least one type, determining a second machine learning model corresponding to the given type; and based on the query request, using the determined second machine learning model, determining the sampling strategy corresponding to the query data of the given type from a plurality of sampling strategies corresponding to the given type.

[0110] Embodiments of this disclosure also provide corresponding apparatus for implementing the above methods or processes. Figure 5 An exemplary structural block diagram of a request processing apparatus 500 according to some embodiments of the present disclosure is shown. The apparatus 500 may be implemented as or included in a terminal device 110. Various modules / components in the apparatus 500 may be implemented by hardware, software, firmware, or any combination thereof.

[0111] like Figure 5 As shown, the apparatus 500 includes a query data acquisition module 510, configured to acquire query data related to a query request, the query data including at least one type of query data. The apparatus 500 also includes a sampling strategy determination module 520, configured to, for each of the at least one type, determine a sampling strategy corresponding to that type of query data from a plurality of sampling strategies based on the query request. The apparatus 500 further includes a sampled query data acquisition module 530, configured to perform sampling on the at least one type of query data based on the determined sampling strategy, to obtain sampled query data of at least one type. The apparatus 500 also includes a response determination module 540, configured to, based on the query request and the sampled query data of at least one type, use a trained first machine learning model to determine the response to the query request.

[0112] In some embodiments, at least one type includes one or more of the following: image type, video type, audio type, measurement data type.

[0113] In some embodiments, the query data includes image data of the image type or video data of the video type, and multiple sampling strategies indicate multiple resolutions; and / or the query data includes audio data of the audio type or measurement data of the sensor measurement type, and multiple sampling strategies indicate multiple sampling frequencies.

[0114] In some embodiments, determining the sampling strategy corresponding to the type of query data from multiple sampling strategies includes: based on the query request, using a trained second machine learning model to determine the sampling strategy corresponding to the type of query data from multiple sampling strategies.

[0115] In some embodiments, the apparatus 500 is implemented at a terminal device, and the second machine learning model is deployed locally on the terminal device.

[0116] In some embodiments, the second machine learning model is trained based on a training dataset, which includes multiple training samples. Each training sample includes a query request sample and a sampled query data sample. The sampled query data sample is obtained by sampling the original query data sample using one of multiple sampling strategies corresponding to the type of query data sample.

[0117] In some embodiments, the training samples in the training dataset are trained by: sampling the original query data samples using multiple sampling strategies corresponding to the types of the original query data samples to obtain multiple sampled candidate query data samples; for each candidate query data sample, determining the predicted response for the query request sample using a trained third machine learning model based on the query request sample and the candidate query data sample; determining the quality scores of each of the multiple predicted responses corresponding to the multiple candidate query data samples; and selecting the sampled query data sample corresponding to the query request sample from the multiple candidate query data samples based on the quality scores of each of the multiple predicted responses.

[0118] In some embodiments, the apparatus 500 further includes a quality score determination module configured to determine a reference response for the query request sample using a first machine learning model based on the query request sample and the original query data sample; and to determine the quality score of each of the multiple predicted responses based on the differences between the multiple predicted responses and the reference response.

[0119] In some embodiments, the sampling strategy determination module 520 is further configured to, for a given type among at least one type, determine a second machine learning model corresponding to the given type; and, based on a query request, use the determined second machine learning model to determine a sampling strategy corresponding to query data of the given type from a plurality of sampling strategies corresponding to the given type.

[0120] The modules included in device 500 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more modules may be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the modules in device 500 may be implemented at least partially by one or more hardware logic components. By way of example, and not limitation, exemplary types of hardware logic components that may be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chips (SoCs), complex programmable logic devices (CPLDs), and so on.

[0121] It should be understood that one or more steps in the above methods can be performed by suitable electronic devices or combinations of electronic devices. Such electronic devices or combinations of electronic devices may include, for example, […]. Figure 1 Terminal device 110 in the middle.

[0122] Figure 6 A block diagram of an electronic device 600 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that... Figure 6 The electronic device 600 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 6 The electronic device 600 shown can be used to achieve Figure 1 Terminal equipment 110 or Figure 5 The device 500.

[0123] like Figure 6 As shown, electronic device 600 is in the form of a general-purpose electronic device. Components of electronic device 600 may include, but are not limited to, one or more processors or processing units 610, memory 620, storage device 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. Processing unit 610 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 620. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 600.

[0124] Electronic device 600 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 600, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 620 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 630 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data and can be accessed within electronic device 600.

[0125] Electronic device 600 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 6As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 620 may include computer program product 625 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0126] The communication unit 640 enables communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device 600 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device 600 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0127] Input device 650 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 660 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 600 can also communicate with one or more external devices (not shown) via communication unit 640 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 600, or with any device that enables electronic device 600 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0128] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0129] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0130] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0131] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0132] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some, as newer, implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0133] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for request processing, comprising: Obtain query data related to the query request, wherein the query data includes at least one type of query data; For each of the at least one type, based on the query request, the sampling strategy corresponding to the query data of that type is determined from multiple sampling strategies; Based on the determined sampling strategy, at least one type of query data is sampled to obtain at least one type of sampled query data. as well as Based on the query request and at least one type of sampled query data, a first trained machine learning model is used to determine the response to the query request.

2. The method according to claim 1, wherein the at least one type includes one or more of the following: image type, video type, audio type, measurement data type.

3. The method of claim 2, wherein the query data includes image data of image type or video data of video type, and the plurality of sampling strategies indicate a plurality of resolutions; and / or The query data includes audio data of the audio type or measurement data of the sensor measurement type, and the multiple sampling strategies indicate multiple sampling frequencies.

4. The method of claim 1, wherein determining the sampling strategy corresponding to the type of query data from a plurality of sampling strategies comprises: Based on the query request, a trained second machine learning model is used to determine the sampling strategy corresponding to this type of query data from multiple sampling strategies.

5. The method of claim 4, wherein the method is implemented at a terminal device, and wherein the second machine learning model is deployed locally on the terminal device.

6. The method according to claim 1, wherein the second machine learning model is trained based on a training dataset, the training dataset including multiple training samples, each training sample including a query request sample and a sampled query data sample, the sampled query data sample being obtained by sampling the original query data sample using one of multiple sampling strategies corresponding to the type of the query data sample.

7. The method of claim 6, wherein the training samples in the training dataset are trained by: Using multiple sampling strategies corresponding to the type of the original query data sample, sampling is performed on the original query data sample to obtain multiple candidate query data samples after sampling; For each of the plurality of candidate query data samples, a trained third machine learning model is used to determine the predicted response for the query request sample based on the query request sample and the candidate query data sample. Determine the quality score of each of the multiple predicted responses corresponding to the multiple candidate query data samples; as well as Based on the quality scores of the multiple predicted responses, the sampled query data sample corresponding to the query request sample is selected from the multiple candidate query data samples.

8. The method of claim 7, wherein determining the quality score of each of the multiple predicted responses corresponding to the multiple candidate query data samples includes: Based on the query request sample and the original query data sample, the first machine learning model is used to determine a reference response for the query request sample; as well as Based on the differences between the multiple predicted responses and the reference response, the quality scores of the multiple predicted responses are determined.

9. The method of claim 1, wherein determining the sampling strategy corresponding to the type of query data from a plurality of sampling strategies comprises: For a given type of at least one of the types Determine the second machine learning model corresponding to the given type; as well as Based on the query request, the determined second machine learning model is used to determine the sampling strategy corresponding to the query data of the given type from multiple sampling strategies of the given type.

10. An apparatus for request processing, comprising: The query data acquisition module is configured to acquire query data related to the query request, the query data including at least one type of query data; The sampling strategy determination module is configured to determine, for each of the at least one type, the sampling strategy corresponding to the query data of that type based on the query request from a plurality of sampling strategies; The sampled query data acquisition module is configured to perform sampling on the at least one type of query data based on the determined sampling strategy to obtain sampled query data of at least one type. as well as The response determination module is configured to determine the response to the query request based on the query request and at least one type of sampled query data, using a trained first machine learning model.

11. An electronic device, comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, which, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 1 to 9.

13. A computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images