User behavior data processing method and electronic equipment

Abstract

The invention provides a user behavior data processing method and electronic equipment, and the method comprises the steps: obtaining the input information of a user; target user behavior data is determined, and the target user behavior data is stored in an area, protected by a trusted execution environment, in a video memory of the graphics processor; based on the input information and the target user behavior data, inputting an artificial intelligence model to obtain a reasoning result; and outputting a reasoning result. The privacy of the user is effectively protected.

Description

Technical Field

[0001] The present application relates to the field of data analysis, and in particular to a method and electronic device for processing user behavior data. Background Art

[0002] During the process of user use, companies can optimize product design, improve user experience, and increase user satisfaction by collecting and recording user behavior data and analyzing it. They can also identify user behavior intentions based on user behavior data. However, in this process, both the operating system and the application can observe user behavior for a long time. Once the user's privacy is leaked, it may cause extremely serious harm to the user. Summary of the invention

[0003] The present application provides a method and electronic device for processing user behavior data.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] In a first aspect, the present application provides a method for processing user behavior data, which may include: obtaining user input information; determining target user behavior data, wherein the target user behavior data is stored in a region protected by a trusted execution environment in a graphics processor's video memory; inputting an artificial intelligence model based on the input information and the target user behavior data to obtain an inference result; and outputting the inference result. The present application effectively protects the privacy of users.

[0006] In a second aspect, the present application provides a device for processing user behavior data, which may include: a first acquisition module for acquiring user input information; a first determination module for determining target user behavior data, wherein the target user behavior data is stored in an area of ​​the video memory of a graphics processor that is protected by a trusted execution environment; a first input module for inputting an artificial intelligence model based on the input information and the target user behavior data to obtain an inference result; and a first output module for outputting the inference result.

[0007] In a third aspect, the present application provides an electronic device,

[0008] Input / output components,

[0009] A memory, comprising an area protected by a trusted execution environment, wherein the area stores target user behavior data;

[0010] A processor, the processor is coupled to the input / output component and the memory, and is configured to: obtain user input information; determine target user behavior data stored in the area protected by the trusted execution environment; input an artificial intelligence model based on the input information and the target user behavior data to obtain an inference result; and output the inference result.

[0011] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0013] Figure 1 This is a schematic diagram of a first flow chart of a method for processing user behavior data in an embodiment of the present application;

[0014] Figure 2 This is a schematic diagram of a second implementation flow of the method for processing user behavior data in an embodiment of the present application;

[0015] Figure 3 A third flow chart of the method for processing user behavior data in an embodiment of the present application;

[0016] Figure 4 A fourth flow chart of the method for processing user behavior data in an embodiment of the present application;

[0017] Figure 5 This is a fifth flow chart of the method for processing user behavior data in an embodiment of the present application;

[0018] Figure 6 This is a sixth flow chart of the method for processing user behavior data in an embodiment of the present application;

[0019] Figure 7 This is a seventh flow chart of the method for processing user behavior data in an embodiment of the present application;

[0020] Figure 8 This is a schematic diagram of an eighth flow chart of the method for processing user behavior data in an embodiment of the present application;

[0021] Fig. 9 A flowchart of a method for processing user behavior data in an embodiment of the present application;

[0022] Fig.10 A schematic diagram of the structure of a device for processing user behavior data in an embodiment of the present application;

[0023] Fig.11 A schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0025] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. The terms "first / second / third" involved are merely used to distinguish similar objects and do not represent a specific order for the objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing this application and are not intended to limit this application.

[0027] In data analysis technology, the current method for processing user behavior data is usually to have the operating system or application observe and record the user's behavior for a long time, and identify the intention based on the recorded user behavior data. However, this method will lead to the leakage of user privacy and may threaten the user's personal safety and property safety.

[0028] In addition, relying solely on software to store and process user behavior data may result in poor processing of user behavior data based on the artificial intelligence model and inaccurate reasoning results obtained based on the artificial intelligence model due to incorrect configuration of the software.

[0029] In order to solve the above problems, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0030] The present application provides a method for processing user behavior data, which is implemented by an electronic device, wherein the electronic device includes an input component, an output component, and a processor. The input component is used to obtain user input; the output component is used to output to the user; the memory includes an area protected by a trusted execution environment and a storage area corresponding to a common execution environment; the storage area of ​​the common execution environment is used to store executable instructions; the processor is used to implement the following, for example Figure 1 , Figure 2 The method provided in the embodiments can effectively protect the privacy of users.

[0031] Figure 1 FIG. 1 is a schematic diagram of a first flow chart of a method for processing user behavior data in an embodiment of the present application. Figure 1 As shown, the method for processing user behavior data may include steps S100 to S400.

[0032] In step S100, user input information is obtained.

[0033] In some embodiments, the input information may be associated with user behavior data. For example, when a user is using an artificial intelligence application, such as the Wenshengtu function, the input information is: Please generate a picture: a picture of a little boy seeing the sea for the first time on a cold winter day. Although the input information does not carry artificial intelligence reasoning function information, in this scenario, since the application is associated with artificial intelligence reasoning function, it can be considered that the input information is associated with artificial intelligence reasoning function, so the stored user behavior data can be used in subsequent reasoning.

[0034] In some embodiments, the input information may include an instruction for instructing to start a reasoning task. The input information may also include an instruction for indicating a reasoning task type, so as to make inferences on the user behavior data according to the reasoning task type.

[0035] In step S200, target user behavior data is determined.

[0036] Among them, the target user behavior data is stored in the area protected by the Trusted Execution Environment (TEE) in the video memory of the graphics processor. That is to say, in the embodiment of the present application, the graphics processor can have its own trusted execution environment, and a part of the video memory of the graphics card of the image processor will be allocated to the trusted execution environment. Among them, the trusted execution environment is an independent and secure execution environment constructed by hardware and software. Even if the external environment changes or is attacked, it can ensure the security, integrity and confidentiality of the code and data running in the trusted execution environment. The functions of the trusted execution environment include: 1) Hardware isolation, the trusted execution environment is isolated from the main operating system and other applications through hardware-level technology, ensuring that even if the main system is attacked or invaded, the code and data in the trusted execution environment can still remain safe. 2) Integrity protection, the trusted execution environment ensures the integrity of the code and data running in it, and any unauthorized modifications can be detected and blocked. 3) Confidentiality protection: Only authorized access can obtain data in the trusted execution environment, effectively preventing the leakage of sensitive information.

[0037] Electronic devices with a trusted execution environment divide the system's hardware and software resources into two execution environments - the trusted execution environment and the normal execution environment (REE). The two environments are securely isolated and have independent internal data paths and storage space required for computing. In the trusted execution environment, applications interact with the trusted execution environment through the trusted execution environment API, which provides a set of secure interfaces for creating a trusted execution environment. The trusted execution environment uses hardware isolation and encryption technology to protect data and code in the trusted execution environment to prevent malware from accessing or tampering.

[0038] When storing user behavior data, it can be classified and stored according to parameters such as attribute parameters. Thus, the stored user behavior data can be filtered based on user information (such as user ID), input information, the type of user behavior data associated with the input information (audio, video, image, text, etc.), at least one of the application program identifier, etc. to obtain target user behavior data.

[0039] In step S300, an artificial intelligence model is input based on input information and target user behavior data to obtain an inference result.

[0040] Artificial Intelligence (AI) can be understood as the production of a new intelligent machine that can respond in a similar way to human intelligence. Artificial Intelligence is a very broad science and can be used in many aspects, including robotics, language recognition, image recognition, natural language processing, expert systems, machine learning, computer vision, etc. For example, artificial intelligence can achieve AI translation and AI text-to-image conversion.

[0041] Generally speaking, the reasoning results of the model are based on the problems to be solved by the user, using logic, statistics or other analysis methods to construct and execute the reasoning process to draw corresponding conclusions. It can be understood that the reasoning results need to be assisted by the artificial intelligence model. Then, when executing step S300, the input information obtained in step S100 and the target user behavior data obtained in step S200 need to be input into the artificial intelligence model to obtain the reasoning results.

[0042] In some embodiments, artificial intelligence models can be used to process massive amounts of data and complete complex tasks, etc. Among them, artificial intelligence models can include artificial intelligence large models, computer vision models, natural language processing models, etc.

[0043] In some embodiments, the input information and the target user behavior data can be input into the artificial intelligence model, and the artificial intelligence model simulates human thinking and creativity, and processes the input information based on the target user behavior data to obtain an inference result. For example, the input information and the target user behavior data can be input into the natural language processing model to perform language analysis on the target user behavior data to achieve tasks such as machine translation, text generation, and sentiment analysis.

[0044] In step S400, the inference result is output.

[0045] Understandably, the inference results need to be output to the user.

[0046] In the embodiment of the present application, the user's input information is obtained and the target user behavior data stored in the area protected by the trusted execution environment is determined. Based on the input information and the target user behavior data, the artificial intelligence model is input to obtain and output the inference result. In this way, the inference result is obtained based on the target user behavior data protected by the trusted execution environment, which prevents the data from being maliciously read or tampered with, and can effectively protect the user's privacy.

[0047] In some embodiments, the method may include step S500, storing the inference result in a storage area of ​​a common execution environment. For example, the inference result may be exported to a common memory for use by an operating system and an application program at any time.

[0048] In some embodiments, the inference results can be classified. When the inference results belong to the first classification results, the first classification results represent that the inference results are stored in the area of ​​the common execution environment, and the inference results are stored in the storage area of ​​the common execution environment; when the inference results belong to the second classification results, the second classification results represent that the inference results are stored in the area protected by the trusted execution environment, and the inference results are stored in the area protected by the trusted execution environment. For example, the judgment can be based on user needs or whether the inference results belong to privacy.

[0049] When the inference results need to be protected, the CPU can also store the inference results in an area protected by a trusted execution environment, using a combination of software and hardware. To store the inference results in an area protected by a trusted execution environment, you can use a method that combines software developed based on Intel Software Guard Extensions (Intel SGX) technology with a hardware environment or hardware system including an isolated area Enclave. Among them, Intel Software Guard Extensions is a confidential computing technology designed to strengthen security protection, enhance data privacy, and improve confidentiality. An isolated area represents a secure environment in which a portion of a system or application can be isolated in a protected, independent area, while providing protection for sensitive data and code to prevent illegal access and tampering by external attackers.

[0050] In some embodiments, the inference results are stored in an area protected by a trusted execution environment, and the code and data in the Enclave can be encrypted using the memory encryption technology in Intel SGX. The code in the Enclave is guaranteed at the hardware level to be able to securely access and modify the data inside it.

[0051] In some embodiments, code and processes may be authorized to access inference results protected by a trusted execution environment.

[0052] In some embodiments, the inference results can also be stored in a Trusted Platform Module (TPM), which provides a secure hardware module specifically created for cryptographic calculations, used to generate and store keys and certificates, physically isolated from the rest of the processing system, and provides measurement and proof of platform integrity.

[0053] In some embodiments, the TPM and SGX can also be combined to store the inference results in an area protected by a trusted execution environment. For example, the TPM can be used to measure and prove the initial state of the hardware memory to ensure the trusted startup of the hardware memory, and then the isolated environment provided by SGX can be used to store the inference results in an area protected by the trusted execution environment.

[0054] In the embodiment of the present application, the inference results are stored in an area protected by a trusted execution environment. In this way, the trusted execution environment can provide hardware-level isolation and encryption protection for the user behavior inference results, prevent the inference results from being tampered with, improve the integrity and authenticity of the inference results during transmission and storage, and effectively protect the user's privacy.

[0055] Figure 2 FIG. 1 is a second flow chart of the method for processing user behavior data in an embodiment of the present application. Figure 2 As shown, the method may also include steps S600 to S700. Figure 2 The steps shown are explained.

[0056] In step S600, user behavior data is recorded.

[0057] It is understandable that before analyzing the user behavior data, the user behavior data needs to be recorded. Therefore, in this step, the user behavior data needs to be recorded.

[0058] In step S700, the user behavior data is stored in an area protected by a trusted execution environment.

[0059] The area protected by the trusted execution environment can provide an independent processing environment with security and integrity for data. In implementation, user behavior data can be stored in the area protected by the trusted execution environment by calling the general data storage GPStorage interface. Among them, the GPStorage interface allows the trusted execution environment system to safely handle storage-related tasks, including but not limited to the initialization of storage devices, data reading and writing operations, and the allocation and release of storage space. These functions are designed to ensure the security and integrity of data and prevent unauthorized access and tampering.

[0060] In the embodiment of the present application, user behavior data is recorded and stored in an area protected by a trusted execution environment. In this way, the trusted execution environment can provide hardware-level isolation and encryption protection for user behavior data, prevent user behavior data from being tampered with, improve the integrity and authenticity of user behavior data during transmission and storage, and effectively protect user privacy.

[0061] Figure 3 FIG. 3 is a third flow chart of the method for processing user behavior data in an embodiment of the present application. Figure 3 As shown, the "determining target user behavior data" in step S200 may include steps S210 to S220. Figure 3 The steps shown are explained.

[0062] In step S210, the AI ​​service type corresponding to the input information is obtained.

[0063] It can be understood that the reasoning results of the model are tools or systems that use logic, statistics or other analytical methods to build and execute the reasoning process based on the problem to be solved by the user to draw corresponding conclusions or verify hypotheses. The service type can be used to represent the problem to be solved by the user. Then, it is necessary to obtain the AI ​​service type corresponding to the input information.

[0064] In some embodiments, AI service types may include AI translation and AI text-to-image conversion, etc. AI service types may also include factual reasoning, rule matching, and hypothetical reasoning, etc.

[0065] Among them, AI translation, or artificial intelligence translation, is the process of using artificial intelligence technology to achieve automatic translation. The AI ​​translation system can analyze and understand the content, context and grammatical structure of the source language text, and then convert it into the target language while trying to maintain the meaning, style and fluency of the original text.

[0066] Among them, AI text-to-image conversion is a process that uses generative AI technology to convert text descriptions into images. This technology can automatically convert input text into logically clear and visually beautiful images by learning the characteristics and patterns in a large amount of data. Generative AI has a wide range of applications, including advertising, art, entertainment, etc.

[0067] In step S220, based on the AI ​​service type, target user behavior data is obtained from an area protected by the trusted execution environment.

[0068] It is understandable that if a comprehensive reasoning analysis is required on a data source, then all data in the data set or data source needs to be processed, so the user behavior data used for reasoning needs to be determined in an area protected by a trusted execution environment.

[0069] In some embodiments, the target user behavior data can be obtained from the area protected by the trusted execution environment using a data cooperation mode to provide the target behavior data to a data analysis party, wherein the data analysis party only executes and analyzes the target behavior data in the trusted execution environment.

[0070] In some embodiments, before obtaining target user behavior data from an area protected by a trusted execution environment, data signatures and digital encryption techniques may be used to encrypt the target behavior data stored in the area protected by the trusted execution environment to ensure the security of the obtained target behavior data.

[0071] In the embodiment of the present application, the AI ​​service type corresponding to the input information is obtained, and based on the AI ​​service type, the target user behavior data is obtained from the area protected by the trusted execution environment. In this way, the user's service needs can be clarified through the AI ​​service type input by the user, and under the guidance of the AI ​​service type, the target user behavior data can be selected from the area protected by the trusted execution environment in a targeted manner to be input into the artificial intelligence model for processing to obtain the processing result, which can achieve a processing result that better meets the user's service needs and improves the accuracy of the artificial intelligence model in processing user behavior data.

[0072] Figure 4 FIG. 4 is a fourth flow chart of the method for processing user behavior data in an embodiment of the present application. Figure 4 As shown, the area protected by the trusted execution environment includes multiple sub-areas, and the "storing user behavior data in the area protected by the trusted execution environment" in step S700 may include steps S710 to S720. Figure 4 The steps shown are explained.

[0073] In step S710, a target sub-region is determined from among multiple sub-regions based on the timeliness status of the data.

[0074] The validity status of the user behavior data stored in the target sub-area is invalid.

[0075] In some embodiments, a performance profiler may be used to determine whether user behavior data is stored in the target sub-region. Determining whether user behavior data is stored in the target sub-region through a performance profiler may include taking a memory snapshot of the target sub-region through a performance profiler to view the objects and bytes stored in the target sub-region when each snapshot is taken, as well as the differences between memory snapshots. By analyzing the usage of these objects and bytes, it is determined whether data is stored in the memory.

[0076] In step S720, the user behavior data is written into the target sub-area.

[0077] It can be understood that the target sub-area is used to store user behavior data.

[0078] In an embodiment of the present application, a target sub-region is determined from multiple sub-regions based on the expiration status of the data, wherein the expiration status of the user behavior data stored in the target sub-region is invalid, and the user behavior data is written into the target sub-region. In this way, determining the user behavior data stored in the target sub-region as invalid can reduce the interference of the old data stored in the target sub-region on the processing results of the artificial intelligence model, and when the amount of user behavior data is relatively large, it can also reduce the demand for storage space protected by the trusted execution environment, and use less storage space to protect user privacy.

[0079] Figure 5 FIG. 5 is a fifth flow chart of the method for processing user behavior data in an embodiment of the present application. Figure 5 As shown, the step S700 of "storing the user behavior data in the area protected by the trusted execution environment" may also include steps S730 to S740. Figure 5 The steps shown are explained.

[0080] In step S730, based on the attribute information of the user behavior data, a preset storage rule is queried to determine a target sub-region from multiple sub-regions.

[0081] It can be understood that the preset storage rules constrain the mapping rules between the attribute information of the user behavior data and the target sub-region, so the target sub-region where the user behavior data is to be stored can be determined from multiple sub-regions based on the attribute information of the user behavior data and the storage rules.

[0082] In some embodiments, the attribute information of the user behavior data may also be the identification of different users. For example, the preset storage rules may include: Rule 1, the specific memory sub-area 1 in the trusted execution environment corresponds to the user behavior data of user 1; Rule 2, the memory sub-area 2 corresponds to the user behavior data of user 2; Rule i, the memory sub-area i corresponds to the user behavior data of application i; Rule N, the memory sub-area N corresponds to the user behavior data of user N.

[0083] In other embodiments, the user behavior data may be multimedia data, and the attribute information of the user behavior data may be the type of multimedia data, for example, the user behavior data includes audio information, video information, image information, and keyboard input information. The preset storage rules may include: Rule 1, the specific memory sub-area 1 corresponds to audio information; Rule 2, the memory sub-area 2 corresponds to video information; Rule 3, the memory sub-area 3 corresponds to image information (e.g., screen information); Rule 4, the memory sub-area 4 corresponds to keyboard input information.

[0084] In other embodiments, the attribute information of the user behavior data may also be the identifier of the application corresponding to N different applications used by the same user. For example, the preset storage rules may include: Rule 1, the specific memory sub-area 1 in the trusted execution environment corresponds to the user behavior data of application 1; Rule 2, the memory sub-area 2 corresponds to the user behavior data of application 2; Rule i, the memory sub-area i corresponds to the user behavior data of application i; Rule N, the memory sub-area N corresponds to the user behavior data of application N. In addition, the above-mentioned storage rules may be combined with each other.

[0085] In step S740, the user behavior data is stored in the target sub-area.

[0086] It is understandable that under the constraints of the preset storage rules, the user behavior data will be allocated to the target sub-region corresponding to the attribute information according to its own attribute information. Then, the user behavior data can be stored in the target sub-region according to the preset storage rules.

[0087] In some embodiments, storing user behavior data in the target sub-area may include: based on rule 1, storing audio information collected through MIC (Microphone Interface Connector) in the last 60 seconds into memory sub-area 1 in the trusted execution environment; based on rule 2, storing video information collected through the camera in the last 30 seconds into memory sub-area 2; based on rule 3, storing screen information obtained through screenshots in the last 30 seconds into memory sub-area 3; based on rule 4, storing keyboard input information collected through the keyboard in the last 60 seconds into memory sub-area 4.

[0088] In the embodiment of the present application, based on the attribute information of the user behavior data, the preset storage rules are queried, a target sub-region is determined from multiple sub-regions, and the user behavior data is stored in the target sub-region. In this way, the user behavior data corresponding to different attribute information can be stored in different target sub-regions according to the preset storage rules, so that when the artificial intelligence model processes different input information, it can obtain data in different target sub-regions in parallel, thereby improving efficiency.

[0089] Figure 6 FIG. 6 is a sixth flow chart of the method for processing user behavior data in an embodiment of the present application. Figure 6 As shown, the step S700 of "storing the user behavior data in the area protected by the trusted execution environment" may also include steps S750 to S760. Figure 6 The steps shown are explained.

[0090] In step S750, when the storage space in the target sub-region is sufficient to write the user behavior data, the user behavior data is stored in the target sub-region.

[0091] It is understandable that the storage space in the target sub-region is limited. Therefore, if the storage space in the target sub-region is sufficient to write the user behavior data, the user behavior data can be directly stored in the target sub-region.

[0092] In step S760, when the storage space in the target sub-region is insufficient to write the user behavior data, the user behavior data is written into the target sub-region in an overwritten manner.

[0093] It is understandable that when the storage space in the target sub-region is insufficient, the previously stored content in the target sub-region can be overwritten. Then, when the condition in step S750 "there is enough storage space in the target sub-region to write the user behavior data" is not met, the user behavior data is overwritten and written into the target sub-region.

[0094] In an embodiment of the present application, when the storage space in the target sub-region is sufficient to write the user behavior data, the user behavior data is stored in the target sub-region, and when the storage space in the target sub-region is insufficient to write the user behavior data, the user behavior data is overwritten and written into the target sub-region. In this way, when the storage space in the target sub-region is sufficient, the user behavior data is written into the storage space. When the storage space in the target sub-region is insufficient, the user behavior data is overwritten and written into the target sub-region, which can immediately release the space occupied by the old data and solve the problem of insufficient storage space. Compared with migrating data to a new storage device or performing complex storage expansion tasks, the processing efficiency is higher.

[0095] In other embodiments, when the storage space in the target sub-region is insufficient to write the user behavior data, the oldest user behavior data is determined in the target sub-region based on the timestamp of the user behavior data, and the oldest user behavior data is overwritten when the user behavior data is written to the target sub-region.

[0096] It can be seen that when the storage is insufficient, the oldest recorded information will be automatically overwritten. Generally speaking, the graphics card of the graphics processor has been loading the information (user behavior data) read by certain sensors (such as microphones, cameras, etc.) and keeping it in the video memory, but the reasoning function is not enabled to perform reasoning operations / processing on the data (so it also saves power and energy). In addition, this process also ensures that other processors (such as the CPU) cannot actually read the information read from the sensor (that is, the information of the sensor is removed from the main memory space of the central processor, transferred to the graphics card and only saved in the video memory, and only for the most recent period of time, such as a few seconds). Only when the reasoning instruction is received will the reasoning work module of the graphics card be started, and reasoning will be started for the specific saved information in the video memory; in this process, the central processor ultimately only obtains the reasoning result from the graphics processor, and there is no need to worry about user data leakage.

[0097] The electronic device in the embodiment of the present application may include more than two processors, at least one of which may be a graphics processor. This embodiment takes a central processing unit and at least one graphics processor as an example, and each step in the above embodiment may be processed by the central processing unit or executed by the graphics processor, and steps S100 to S400 are used as an example for explanation.

[0098] In some embodiments, in the first stage, the CPU obtains user behavior data, for example, the CPU obtains it from sensors such as cameras, etc., or the application collects and records user behavior. The CPU sends recent user behavior data to the GPU, and the GPU stores the recent user behavior data in an area in the video memory, which is protected by a trusted execution environment. In the second stage, the CPU obtains the user's input information, which is associated with an AI reasoning service, for example, the input information is input by the user when using the AI ​​service; the CPU sends the user's input information to the GPU, that is, the GPU obtains the user's input information from the CPU; the GPU determines the target user behavior data from the stored user behavior data, wherein the target user behavior data is stored in an area protected by a trusted execution environment in the video memory of the graphics processor; the GPU inputs the artificial intelligence model based on the input information and the target user behavior data to obtain the reasoning result; the GPU outputs the reasoning result to the CPU, and the CPU outputs the reasoning result to the user. It should be noted that the number of GPUs in this embodiment is at least one, and multiple GPUs can form a cluster.

[0099] In some other embodiments, the first stage is the same as the previous embodiment. In the second stage, the CPU obtains the user's input information, which is associated with the AI ​​reasoning service, for example, the input information is input by the user when using the AI ​​service; the CPU sends an inference request to the GPU, which can carry the above-mentioned input information. When the GPU determines that the inference service corresponding to the inference request needs to use the user behavior data, the GPU determines the target user behavior data from the stored user behavior data, wherein the target user behavior data is stored in the video memory of the graphics processor; the GPU inputs the artificial intelligence model based on the input information and the target user behavior data to obtain the inference result; the GPU outputs the inference result to the CPU, and the CPU obtains the inference result, which can be obtained by inputting the artificial intelligence model based on the input information and the target user behavior data, and then the CPU outputs the inference result to the user.

[0100] In other embodiments, in the first stage, the CPU obtains user behavior data, for example, the CPU obtains it from a sensor such as a camera, etc., or the application collects and records user behavior. The CPU stores recent user behavior data in an area of ​​the CPU that is protected by a trusted execution environment. In the second stage, the CPU obtains user input information, which is associated with an AI reasoning service, for example, the input information is input by the user when using the AI ​​service; the CPU determines target user behavior data from the stored user behavior data, wherein the target user behavior data is stored in a storage area of ​​the CPU that is protected by a trusted execution environment; the CPU inputs the artificial intelligence model based on the input information and the target user behavior data to obtain an inference result; the CPU outputs the inference result to the user.

[0101] Figure 7 FIG. 7 is a seventh flow chart of the method for processing user behavior data in an embodiment of the present application. Figure 7 As shown, the step S700 of "storing the user behavior data in the area protected by the trusted execution environment" may also include steps S770 to S780, which will be described below in conjunction with Figure 7 The steps shown are explained.

[0102] In step S770, when it is determined based on the preset writing rule that the target sub-region is writable, the user behavior data is stored in the target sub-region.

[0103] The write rule is used to indicate the timeliness of the user behavior data stored in each sub-area.

[0104] In step S780, when it is determined based on the preset writing rule that the target sub-region is not writable, the user behavior data is overwritten and written into the target sub-region.

[0105] It can be understood that when the target sub-area is full, it is determined that the target sub-area is not writable, so the data previously stored in the target sub-area needs to be overwritten. When the target sub-area is determined to be writable based on the preset write rules, the user behavior data is stored in the target sub-area, and when the target sub-area is determined to be not writable based on the preset write rules, the user behavior data is overwritten and written into the target sub-area. In this way, when the target sub-area is determined to be writable based on the preset write rules, the user behavior data is stored in the target sub-area, and when the target sub-area is determined to be not writable based on the preset write rules, the user behavior data is overwritten and written into the target sub-area. The space occupied by the old data can be released immediately, solving the problem of insufficient storage space, and the processing efficiency is higher than migrating data to a new storage device or performing complex storage expansion tasks.

[0106] In some embodiments, the storage rules are used to indicate the attribute information of the user behavior data stored in each sub-area, and the attribute information also includes at least one of the following: the collection method of the user behavior data, the type of the user behavior data, and the identifier of the tenant to which the user behavior data belongs. It can be seen that by indicating the user behavior data stored in the sub-area through attribute information, data security and processing efficiency can be improved. First, the collection method of the user behavior data can clarify the source of the user behavior data, such as the collection by the camera, the collection by the microphone, the generation of the screen recording, and the tracking of the user's usage habits. Storage based on the collection method can be classified and searched during use and storage to improve efficiency. Secondly, by distinguishing the types of user behavior data, it can include audio, video, images, text, etc., so that people can classify and process user behavior data according to type. Finally, through the identifier of the tenant to which the user behavior data belongs, the storage area corresponding to the specific tenant can be quickly located, reducing unnecessary full scanning of all sub-areas.

[0107] Figure 8 FIG. 8 is a flow chart of an eighth method for processing user behavior data in an embodiment of the present application. Figure 8 As shown, the method may also include steps S771 to S772, wherein the write rule is used to indicate the timeliness of the user behavior data stored in each sub-region. Figure 8 The steps shown are explained.

[0108] In step S771, the storage duration of the user behavior data in the sub-region is determined.

[0109] It is understandable that the data stored in the sub-region may also be stored within a preset time. During implementation, the storage time of the user behavior data in the sub-region may be determined by calculating a timestamp.

[0110] In some embodiments, determining the storage duration of user behavior data in a sub-region by calculating a timestamp may include obtaining the timestamp when the user behavior data is stored in the sub-region, and determining the duration for which the data is processed or stored in the sub-region based on the duration between the stored timestamp and the current timestamp.

[0111] In some embodiments, you can also use a custom timer to insert specific timing instructions in the application or the code for monitoring the sub-area, and read the results of these instructions in the application to determine the storage duration of the user behavior data in the sub-area.

[0112] In step S772, when the storage time does not satisfy the timeliness of the user behavior data, the data in the sub-area is deleted.

[0113] It is understandable that when the storage duration of the user behavior data in the sub-region does not meet the timeliness of the user behavior data, it will not only affect the reasoning of the user behavior data, but may also cause disk space saturation and cause the system to crash. In an embodiment of the present application, the storage duration of the user behavior data in the sub-region is determined, and when the storage duration does not meet the timeliness of the user behavior data, the data in the sub-region is deleted. In this way, deleting the user behavior data in the sub-region whose storage duration does not meet the timeliness can reduce the impact of redundant user behavior data in the sub-region on the processing results and reduce the risk of privacy data leakage.

[0114] In some embodiments, an indicator device may be added, and when reasoning about user behavior information, the indicator device is turned on to remind the user that the electronic device begins to enter the model reasoning working mode. The indicator device may be an indicator light.

[0115] In the process of accompanying users, it is necessary to record the user's behavior and identify the intention based on the user's behavior, but this mode does make users feel very worried, because both the operating system and the application can observe the user's behavior for a long time. Once leaked, it will cause great harm to the user's privacy. The protection of personal privacy in the use of large models has become a pain point in the industry. In this embodiment, the artificial intelligence model is combined with the hardware system (such as a trusted execution environment) to design a working mode and system suitable for special scenarios. This mode will greatly reduce the user's concerns about privacy leakage and improve the user experience.

[0116] The working principle of this application can be applied to other technical means for hardware protection. The technical means for hardware protection can also include Intel's graphics client hardware (Protected Xe Path, PXP) protection session technology. Among them, Intel PXP achieves the security of graphics processor computing by building an independent trusted execution environment, implementing isolation measures, and providing encryption and decryption functions. At present, Intel PXP is also protected by a non-disclosure agreement (NDA). The working principle of this application can also be applied to Intel PXP.

[0117] Fig. 9 A flowchart of a method for processing user behavior data in an embodiment of the present application is shown in FIG. Fig. 9 As shown, the method includes the following steps S901 to S905:

[0118] S901: The system for protecting personal privacy records the user behavior data generated by user interactions in real time.

[0119] Among them, the system that protects personal privacy stores user behavior data in memory protected by a trusted execution environment, rather than in a normal execution environment, so neither the operating system nor user-mode applications can obtain it.

[0120] S902: The system for protecting personal privacy stores the recorded content in an area protected by the trusted execution environment according to preset storage rules.

[0121] The multi-modal multi-user behavior data may be sent to the same area protected by a trusted execution environment, or may be sent to different areas according to the data type of the multi-modal user behavior data.

[0122] In some embodiments, the recorded content may be stored in an area protected by a trusted execution environment, in which case multimodal data including text, audio, and image data may be stored in one area.

[0123] In some embodiments, the recorded content may also be stored in an area protected by a trusted execution environment. In this case, multimodal data may be stored in different areas according to data type.

[0124] In some embodiments, multimodal data can be stored in different areas according to data type, which may include: text type data is sent to a target sub-area for storing text, audio type data is sent to a target sub-area for storing audio, and image type data is sent to a target sub-area for storing images.

[0125] In some embodiments, the correspondence between storage rules and areas protected by a trusted execution environment may include: Rule 1, corresponding to trusted execution environment memory sub-area 1, used to store audio information collected through the MIC in the last 60 seconds; Rule 2, corresponding to memory sub-area 2, used to store video information collected through the camera in the last 30 seconds; Rule 3, corresponding to memory sub-area 3, used to store screen information obtained through screenshots in the last 30 seconds; Rule 4, corresponding to memory sub-area 4, used to store keyboard input information collected through the keyboard in the last 60 seconds.

[0126] If the trusted execution environment supports multi-tenancy, the memory sub-areas 1 to 4 in the trusted execution environment can be allocated to a single tenant or to different tenants. When specific memory areas in the trusted execution environment are allocated to different tenants, the collected information can be isolated from each other.

[0127] In some embodiments, when the trusted execution environment supports multiple tenants, text-type data in the user behavior data of all tenants is sent to a target sub-region for storing text, audio-type data in the user behavior data of all tenants is sent to a target sub-region for storing audio, and image-type data in the user behavior data of all tenants is sent to a target sub-region for storing images.

[0128] In some embodiments, when the trusted execution environment supports multi-tenancy, in the user behavior data of all tenants, each tenant corresponds to a storage area, each storage area is isolated from each other, and in each storage area, text-type data is sent to a target sub-area for storing text, audio-type data is sent to a target sub-area for storing audio, and image-type data is sent to a target sub-area for storing images.

[0129] S903: The system for protecting personal privacy overwrites part or all of the old data according to a preset writing rule when the area is full or about to be full.

[0130] The write rule is used to indicate the timeliness of the user behavior data stored in each sub-area.

[0131] In some embodiments, part or all of the old data is overwritten. When storing the latest audio information collected by the MIC, the audio data 60 seconds ago will be erased; when storing the video information collected by the camera in the last 30 seconds, the video information 30 seconds ago will be erased; when storing the latest screen information, the screen information obtained by screenshot 30 seconds ago will be erased; when storing the latest keyboard input information, the keyboard input information collected through the keyboard 60 seconds ago will be erased.

[0132] S904: When the user explicitly needs to perform a reasoning task, the system for protecting personal privacy starts the reasoning model to reason about the user behavior data in the area protected by the trusted execution environment.

[0133] In some embodiments, the user explicitly needs to perform the reasoning task, which can be done by a user in user mode or a user in kernel mode pressing a specific key on the application program to explicitly need to perform the reasoning task.

[0134] In some embodiments, the user explicitly needs to perform the reasoning task, and the user may also enable the reasoning task through a standard software entry.

[0135] In some embodiments, starting the reasoning model to reason about the user behavior data in the area protected by the trusted execution environment may be unified reasoning for the multi-modal user behavior data stored in the sub-areas.

[0136] In some embodiments, the inference model is started to infer the user behavior data of the area protected by the trusted execution environment, or different inference models may be used to perform inference on the data of different sub-areas respectively.

[0137] S905: Only the inference result is exported to the common execution environment for use by the operating system and application programs.

[0138] Among them, the normal execution environment provides a complete operating system environment, allowing users to install and run various applications. This environment is powerful, but it cannot guarantee the confidentiality and integrity of running code and data, and user-installed applications may be threatened by other parts of the device.

[0139] In some embodiments, the reasoning results can be encrypted and exported from the area protected by the trusted execution environment and transmitted to the ordinary execution environment through the client application programming interface (Client Application Programming Interface, Client API) of the trusted execution environment. Among them, the client application programming interface of the trusted execution environment is an interface provided to the outside by the trusted execution environment driver in the ordinary execution environment, allowing client applications (Client Applications, CA) running in the ordinary execution environment to interact with trusted applications (Trusted Applications, TA) running in the trusted execution environment. These application programming interfaces (Application Programming Interface, API) generally include standard system calls. Among them, the API may include interfaces such as the OpenSession API, the InvokeCommand API, etc. The OpenSession API is used to establish a session and the InvokeCommand API is used to send commands.

[0140] In some embodiments, exporting the reasoning results to the normal execution environment may include initializing the OpenSession API in the trusted execution environment, calling the OpenSession API to open a connection to the trusted execution environment, performing necessary permission checks and authentication processes in the trusted execution environment, and after the authentication is passed, the trusted execution environment begins to transmit files or texts including the reasoning results to the normal execution environment. After the files or texts of the reasoning results are successfully transmitted to the normal execution environment, the connection with the OpenSession API is closed.

[0141] In some embodiments, the inference results are exported to the normal execution environment, and the target path in the normal execution environment can be set in the trusted execution environment, and the InvokeCommand API is called to copy the file in the trusted execution environment to the normal execution environment.

[0142] In the embodiment of the present application, the user's input information is obtained and the target user behavior data stored in the area protected by the trusted execution environment is determined. Based on the input information and the target user behavior data, the artificial intelligence model is input to obtain and output the inference result. In this way, the inference result is obtained based on the target user behavior data protected by the trusted execution environment, which prevents the data from being maliciously read or tampered with, and can effectively protect the user's privacy.

[0143] The advantages of adopting the technical solution provided by the embodiments of the present application are: 1) The present application implements the processing of user behavior data based on a combination of software and hardware. Compared with the processing of user behavior data only by software or only by hardware, it can be applied to special working modes and systems, greatly reducing the user's concerns about privacy leakage and improving the user's experience; 2) The method provided by the embodiments of the present application is applied to the GPU. After receiving the instruction, the corresponding model can be started according to the instruction to directly infer the collected data in the GPU, so as to respond to the user as quickly as possible and reduce waiting.

[0144] Based on the same inventive concept, an embodiment of the present application also provides a device for processing user behavior data. The modules included in the device for processing user behavior data, and the units included in each module, can be implemented by a processor of an electronic device; of course, they can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0145] Fig.10 A schematic diagram of a structure of a device for processing user behavior data in an embodiment of the present application is shown in FIG. Fig.10 As shown, the user behavior data processing device 1000 may include: a first acquisition module 1001, used to obtain user input information; a first determination module 1002, used to determine target user behavior data, wherein the target user behavior data is stored in an area protected by a trusted execution environment in the video memory of the graphics processor; a first input module 1003, used to input an artificial intelligence model based on the input information and the target user behavior data to obtain an inference result; and a first output module 1004, used to output the inference result.

[0146] In some embodiments, the user behavior data processing device 1000 may further include: a first storage module, used to store the inference result in an area protected by a trusted execution environment.

[0147] In some embodiments, the user behavior data processing device 1000 may further include: a first recording module, used to record the user behavior data; and a second storage module, used to store the user behavior data in an area protected by a trusted execution environment.

[0148] In some embodiments, the determination module 1002 may include: a first acquisition unit, used to acquire the AI ​​service type corresponding to the input information; and a second acquisition unit, used to acquire target user behavior data from an area protected by a trusted execution environment based on the AI ​​service type.

[0149] In some embodiments, the area protected by the trusted execution environment includes multiple sub-areas, and the second storage module may include: a first determination unit, used to determine a target sub-area from the multiple sub-areas based on the expiration status of the data; wherein the expiration status of the user behavior data stored in the target sub-area is invalid; and a first writing unit, used to write the user behavior data into the target sub-area.

[0150] In some embodiments, the second storage module may further include: a second determination unit, used to query preset storage rules based on attribute information of the user behavior data and determine a target sub-area from multiple sub-areas; and a first storage unit, used to store the user behavior data in the target sub-area.

[0151] In some embodiments, the second storage module may also include: a second storage unit, used to store the user behavior data in the target sub-area when the storage space in the target sub-area is sufficient to write the user behavior data; and a first output unit, used to overwrite the user behavior data in the target sub-area when the storage space in the target sub-area is insufficient to write the user behavior data.

[0152] In some embodiments, the second storage module may also include: a third storage unit, used to store user behavior data in the target sub-area when it is determined based on a preset writing rule that the target sub-area is writable; a second output unit, used to overwrite the user behavior data in the target sub-area when it is determined based on a preset writing rule that the target sub-area is not writable.

[0153] In some embodiments, the storage rules are used to indicate the attribute information of the user behavior data stored in each sub-area, and the attribute information includes at least one of the following information: the collection method of the user behavior data; the type of the user behavior data; and the identifier of the tenant to which the user behavior data belongs.

[0154] In some embodiments, the write rules are used to indicate the timeliness of the user behavior data stored in each sub-region. The second storage module may also include: a determination unit for determining the storage time of the user behavior data in the sub-region; and a deletion unit for deleting the data in the sub-region when the storage time does not meet the timeliness of the user behavior data.

[0155] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0156] In some embodiments, the method provided by the embodiment of the present application can be implemented in the form of embedded software. If the above-mentioned method for processing user behavior data is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods of each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific hardware, software or firmware, or any combination of hardware, software, and firmware.

[0157] The processor may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of processors include, but are not limited to, central processing units, graphics processing units, various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor may execute the various methods and processes described above, such as executing the resource configuration method described above.

[0158] In some embodiments, the resource configuration method described above may be implemented as a computer software program, which is stored in a machine-readable medium, such as a memory. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via a ROM and / or a communication component. When the computer program is loaded into RAM and executed by a processor, one or more steps in the resource configuration method described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform one or more steps in the resource configuration method described above by any other appropriate means (e.g., by means of firmware).

[0159] Based on the above embodiments, the present application further provides an electronic device, including:

[0160] Input / output components,

[0161] A memory, comprising an area protected by a trusted execution environment, wherein the area stores target user behavior data;

[0162] A processor, the processor is coupled to the input / output component and the memory, and is configured to: obtain user input information; determine target user behavior data stored in the area protected by the trusted execution environment; input an artificial intelligence model based on the input information and the target user behavior data to obtain an inference result; and output the inference result.

[0163] Fig.11 is a schematic diagram of the structure of an electronic device in an embodiment of the present application, such as Fig.11 As shown, the electronic device 1100 can be the electronic device in the above-mentioned embodiment. The electronic device 1100 may include a processor 1101, which can perform various appropriate steps and processes according to the computer program instructions stored in the read-only memory 1102 or the computer program instructions loaded from the memory 1108 to the random access memory (Read Access Memory, RAM) 1103. In the RAM 1103, various programs and data required for the operation of the electronic device 1100 can also be stored. The processor 1101, ROM 1102 and RAM 1103 are connected to each other via a bus 1104. An input / output (Input / Output, I / O) interface 1105 is also connected to the bus 1104.

[0164] Multiple components in the electronic device 1100 are connected to the I / O interface 1105, including: input components 1106, such as keyboards, mice, etc.; output components 1107, such as various types of displays, speakers, etc.; storage 1108, such as disks, optical disks, etc.; and communication components 1109, such as network cards, modems, wireless communication transceivers, etc. The communication components 1109 allow the electronic device 1100 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunication networks.

[0165] The processor 1101 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 1101 include, but are not limited to, a central processing unit, a graphics processing unit, various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSP), and any appropriate processors, controllers, microcontrollers, etc. The processor 1101 may perform the various methods and processes described above, such as executing the processing method for user behavior data described above. For example, in some embodiments, the processing method for user behavior data described above may be implemented as a computer software program, which is stored in a machine-readable medium, such as a memory 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1100 via the ROM 1102 and / or the communication component 1109. When the computer program is loaded into the RAM 1103 and executed by the processor 1101, one or more steps in the processing method for user behavior data described above may be executed. Alternatively, in other embodiments, the processor 1101 may be configured in any other appropriate manner (eg, by means of firmware) to execute one or more steps in the method for processing user behavior data described above.

[0166] It should be further noted that the present application may include methods, devices, systems and / or computer program products. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present application.

[0167] The embodiment of the present application provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, some or all of the steps in the above method can be implemented. The computer-readable storage medium can be transient or non-transient.

[0168] An embodiment of the present application provides a computer program, including computer-readable program instructions. When the computer-readable program instructions are executed in a computer device, a processor in the computer device executes some or all of the steps in the above method.

[0169] The present application embodiment provides a computer program product, including a computer program or an instruction, which, when executed by a processor, can implement some or all of the steps in the above method. The computer program product can be implemented by hardware, software, or a combination thereof. In some embodiments, the computer program product is embodied as a computer storage medium, and in other embodiments, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc.

[0170] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. Computer readable storage medium can include (non-exhaustive list): a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM), a flash memory, a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof. The computer-readable storage medium used in this document is not to be interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (for example, light pulses through a fiber optic cable), or an electrical signal transmitted through wires.

[0171] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or to a processing device, or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0172] The computer program instructions for performing the operation of the present application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as C language or similar programming languages. The computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, by utilizing the state information of computer-readable program instructions to personalize an electronic circuit, such as a programmable logic circuit, a field programmable gate array, or a programmable logic array (PLA), the electronic circuit can execute the computer-readable program instructions to implement various aspects of the present application.

[0173] Various aspects of the present application are described herein with reference to the flow chart and / or step diagram of the method, device (system), and computer program product according to the exemplary embodiments of the present application. It should be understood that each step of the flow chart and / or step diagram and the combination of each step in the flow chart and / or step diagram can be implemented by computer-readable program instructions.

[0174] These computer-readable program instructions can be provided to a processor in a voice interaction device, a general-purpose computer, a special-purpose computer, or a processing unit of other programmable data processing devices, thereby producing a machine, so that when these instructions are executed by a processing unit of a computer or other programmable data processing device, a device that implements the functions / steps specified in one or more steps in a flowchart and / or a step diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions enable a computer, a programmable data processing device, and / or other devices to work in a specific manner, so that a computer-readable medium storing instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / steps specified in one or more steps in a flowchart and / or a step diagram.

[0175] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / steps specified in one or more steps in the flowchart and / or step diagram.

[0176] The flow chart and step diagram in the accompanying drawings show the possible architecture, function and operation of the equipment, method and computer program product according to multiple embodiments of the present application. In this regard, each step in the flow chart or step diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the steps can also occur in a sequence different from that marked in the accompanying drawings. For example, two consecutive steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each step in the step diagram and / or flow chart, and the combination of the steps in the step diagram and / or flow chart, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0177] The above description is only an exemplary implementation of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of protection involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the technical concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for processing user behavior data, applied to an electronic device, the method comprising: Get user input information; Determining target user behavior data, wherein the target user behavior data is stored in an area protected by a trusted execution environment in a video memory of a graphics processor; Based on the input information and the target user behavior data, an artificial intelligence model is input to obtain an inference result; The inference result is outputted.

2. The method according to claim 1, wherein: The method further comprises: The inference result is stored in an area protected by the trusted execution environment.

3. The method according to claim 1 or 2, wherein: The determining of target user behavior data includes: Obtaining an artificial intelligence service type corresponding to the input information; Based on the artificial intelligence service type, target user behavior data is obtained from an area protected by a trusted execution environment.

4. The method according to any one of claims 1 to 3, wherein: The method further comprises: Record user behavior data; The user behavior data is stored in the area protected by the trusted execution environment.

5. The method according to claim 4, wherein: The area protected by the trusted execution environment includes a plurality of sub-areas, and storing the user behavior data in the area protected by the trusted execution environment includes: Determine a target sub-region from the multiple sub-regions based on the timeliness status of the data; wherein the timeliness status of the user behavior data stored in the target sub-region is invalid; The user behavior data is written into the target sub-area.

6. The method according to claim 4 or 5, wherein: The storing of the user behavior data in the area protected by the trusted execution environment further includes: Based on the attribute information of the user behavior data, a preset storage rule is queried to determine a target sub-region from multiple sub-regions; storing the user behavior data in the target sub-area; Among them, the storage rules are used to indicate the attribute information of the user behavior data stored in each of the sub-areas; the attribute information includes at least one of the following information: the collection method of the user behavior data; the type of the user behavior data; the identifier of the tenant to which the user behavior data belongs.

7. The method according to claim 6, wherein: The storing of the user behavior data in the area protected by the trusted execution environment further includes: If the storage space in the target sub-area is sufficient to write the user behavior data, storing the user behavior data in the target sub-area; When the storage space in the target sub-region is insufficient to write the user behavior data, the user behavior data is stored and written into the target sub-region.

8. The method according to claim 6 or 7, wherein: The storing of the user behavior data in the area protected by the trusted execution environment further includes: When it is determined based on a preset writing rule that the target sub-area is writable, storing the user behavior data in the target sub-area; When it is determined based on a preset writing rule that the target sub-area is not writable, the user behavior data is overwritten and written into the target sub-area.

9. The method according to claim 8, wherein: The write rule is also used to indicate the timeliness of the user behavior data stored in each sub-area; the method also includes: Determine the storage duration of the user behavior data in the sub-region; When the storage time does not satisfy the timeliness of the user behavior data, the data in the sub-area is deleted.

10. An electronic device, comprising: Input / output components, A memory, comprising an area protected by a trusted execution environment, wherein the area stores target user behavior data; A processor, the processor being coupled to the input / output component and the memory, and being configured to: obtain input information from a user; Determine target user behavior data stored in the area protected by the trusted execution environment; Based on the input information and the target user behavior data, an artificial intelligence model is input to obtain an inference result; The inference result is outputted.

Images