Model training method, service pushing method and electronic equipment

Abstract

The invention provides a model training method, a service pushing method and electronic equipment. And the cloud end trains the general intention prediction model based on the historical device event data of the group users, so that the general intention prediction model learns common behavior patterns of the group users, and a target general intention prediction model is obtained. And then, the cloud deploys the target general intention prediction model on the electronic equipment side. Due to the difference between the behavior pattern of the group user and the individual behavior pattern, the electronic equipment continues to train the target general intention prediction model on the end side based on the individual equipment event data by adopting a forgetting loss function, so that the target general intention prediction model learns the individual behavior pattern, and the user behavior is predicted. According to the invention, personalized adaptation of the target general intention prediction model is realized, so that the electronic equipment can accurately predict the user intention by using the adapted model, the accuracy of service recommendation is further ensured, and the user experience is high.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a model training method, a service push method, and an electronic device. Background Technology

[0002] With the development of machine learning technology, it has been widely applied to solve problems in various technical fields. User intent prediction, as an application of machine learning, can predict user intent based on collected user behavior data, thereby making corresponding decisions, such as recommending relevant services. Therefore, how to accurately predict user intent has become a pressing problem to be solved. Summary of the Invention

[0003] This application provides a model training method, a service push method, and an electronic device for personalized adaptation of edge models, thereby improving the accuracy of intent prediction by edge models.

[0004] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0005] Firstly, a model training method is provided, in which an electronic device receives a first intent prediction model sent by a server. This first intent prediction model is obtained by the server training a general intent prediction model based on historical event data from multiple users, enabling the general intent prediction model to learn the behavioral habits of multiple users (i.e., the behavioral patterns of a group of users). The historical event data is generated based on users' historical behavior.

[0006] Next, the electronic device collects first device event data. This first device event data is generated by the electronic device based on user behavior. Then, based on this first device event data, the electronic device trains a first intent prediction model. After training, the electronic device can determine the set of intents to be forgotten and the set of intents to be retained. The intents to be forgotten in the set of intents to be forgotten represent the preset intents that the first intent prediction model has learned to forget, i.e., the intents that need to be forgotten. The intents to be retained in the set of intents to be retained include all preset intents except for those to be forgotten. Preset intents represent the intents that the first intent prediction model can output, i.e., the intent types.

[0007] Subsequently, the electronic device can calculate a forgetting loss function value based on the set of intentions to be forgotten and the set of intentions to be retained. This value, along with the trained first intention prediction model, is used to determine the target intention prediction model. This target intention prediction model is then used to predict intentions based on device event data.

[0008] In this application, after the first intent prediction model is deployed on the electronic device, since this first intent prediction model is trained based on historical event data of a group of users, and there may still be differences between the behavior patterns of individual users and those of the group of users, the electronic device uses locally collected individual user behavior data to train the first intent prediction model to continue learning the individual user behavior patterns. Furthermore, after training, the electronic device continues to calculate loss function values ​​(i.e., forgetting loss function values) for the sets of intents to be forgotten and the sets of intents to be retained. These forgetting loss function values ​​are used to guide the first intent prediction model in forgetting intents, thereby enabling the first intent prediction model to adapt to and learn the behavior patterns of the electronic device, achieving personalized adaptation, and ultimately obtaining a target intent prediction model capable of accurately predicting intents.

[0009] In one possible design approach, after obtaining the target intent prediction model, this model can be used for service delivery. When an electronic device collects event data from a third device, it inputs this data into the target intent prediction model to obtain the intent prediction result. Subsequently, the electronic device can push corresponding services based on the intent prediction result, achieving accurate service delivery, facilitating user access to relevant services, and improving user satisfaction.

[0010] In one possible design approach, the aforementioned third-device event data includes alarm clock ringing event data. The intent prediction result includes news intent. Accordingly, the aforementioned push of corresponding services based on the intent prediction result may include, for example, the electronic device may push a morning news service based on the news intent, enabling users to quickly browse the news and facilitating user experience.

[0011] In one possible design approach, the aforementioned first intent prediction model is obtained by distilling the target general intent prediction model; that is, the first intent prediction model is the distilled target general intent prediction model, which is obtained by training the general intent prediction model. Based on this, a lightweight first intent prediction model is deployed on the electronic device side to ensure the efficiency of intent prediction and reduce the resource consumption of the electronic device.

[0012] In one possible design approach, the process of training the first intent prediction model based on the first device event data may include: the electronic device can train the first intent prediction model based on the first event data, which includes first intent event data and second device event data. The first intent event data is obtained by mapping intent to device event data other than the second device event data within the first device event data. Therefore, by mapping device event data to intent event data, the efficiency of determining training samples can be improved, and when training the model based on these training samples, the model can better learn the user's intent.

[0013] Optionally, if the application name in the first device event data belongs to a preset name set, the electronic device can map the first device event data to the intent event data (i.e., the first intent event data) corresponding to the preset intent of the preset name set, thereby realizing intent mapping.

[0014] If the application name in the first device event data does not belong to the preset name set, the electronic device may use the first device event data as the aforementioned second device event data.

[0015] In one possible design approach, the aforementioned historical event data includes first historical device event data and second intent event data. The second intent event data is obtained by performing intent mapping on the second historical device event data. The first historical device event data differs from the second historical device event data. Specifically, each historical device event data within the first historical device event data and the second historical device event data is distinct. The union of the first historical device event data and the second historical device event data constitutes the historical device event data.

[0016] In one possible design approach, the aforementioned set of intentions to be forgotten includes a first set of forgetting intentions (i.e., the set of group forgetting intentions) and / or a second set of forgetting intentions (or the set of individual forgetting intentions).

[0017] The intentions to be forgotten in the first forgetting set refer to preset intentions whose output frequency (i.e., output probability) is less than a first threshold during the training of the general intention prediction model. In other words, for each preset intention, if the output frequency corresponding to that preset intention is less than the first threshold, that preset intention can be included in the first forgetting set as an intention to be forgotten. Based on this, the first forgetting intention set represents intentions with a relatively small population distribution and a low probability of occurrence in group behavior patterns. Therefore, electronic devices can update the parameters in the first intention prediction model based on the first forgetting intention set, causing the first intention prediction model to forget the intentions in the first forgetting intention set, thereby improving the accuracy of model prediction.

[0018] The intentions to be forgotten in the second set of forgotten intentions (i.e., long-tail intentions) are preset intentions whose frequency of occurrence in historical event data is less than a second threshold, and whose frequency of occurrence in the first event data is less than a third threshold. Based on this, long-tail intentions are discovered, thereby enabling the first intention prediction model to forget the learning bias of long-tail intentions.

[0019] The third threshold mentioned above may be different from the second threshold, or they may be the same, such as both being the average proportion of intention.

[0020] In one possible design approach, the aforementioned forgetting loss function value is based on L. CE (M r ,R)-λL CE (M f The result is obtained by calculation of F, where L CE () is the cross-entropy function, M r R is the set of intentions to be retained, and M is the set of intentions to be retained. f Let λ represent the intention to be forgotten, λ be a preset value, and F be the set of intentions to be forgotten. Based on this, the sub-device can update the trained first intention prediction model using the forgetting loss function value, thereby minimizing the loss between the output of the trained first intention prediction model and the output of the original distilled first intention prediction model regarding the intention to be retained, and maximizing the loss between the two outputs regarding the intention to be forgotten.

[0021] In one possible design, if the loss function value is less than the fourth threshold, the electronic device uses the trained first intent prediction model as the target intent prediction model. If the loss function value is greater than or equal to the fourth threshold, the electronic device uses the trained first intent prediction model as the first intent prediction model and returns the step of training a general intent prediction model based on historical event data from multiple users to continue training the model and ensure the accuracy of the model's predictions.

[0022] Secondly, this application provides a service push method applied to electronic devices. The electronic device collects event data from a third device. Using the third device event data as input, the aforementioned target intent prediction model is run to obtain an intent prediction result. Based on the intent prediction result, a corresponding service is pushed.

[0023] Thirdly, this application provides an electronic device, which includes a memory and one or more processors; the memory and the processors are coupled; the memory is used to store computer program code, which includes computer instructions; when the processor executes the computer instructions, the electronic device performs the model training method or service push method as described above.

[0024] Fourthly, this application provides a chip, the chip including a communication interface and at least one processor:

[0025] The communication interface is used for inputting and / or outputting signaling or data;

[0026] The at least one processor is used to execute a computer program to implement the model training method or service push method described above.

[0027] Fifthly, this application provides a computer-readable storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform the model training method or service push method described above.

[0028] Sixthly, this application provides a computer program product that, when run on an electronic device, causes the electronic device to execute the model training method or service push method described above.

[0029] It is understood that the beneficial effects achieved by the method described in the second aspect, the electronic device described in the third aspect, the chip described in the fourth aspect, the computer-readable storage medium described in the fifth aspect, and the computer program product described in the sixth aspect can be referred to the beneficial effects in the first aspect and any of its possible design embodiments, and will not be repeated here. Attached Figure Description

[0030] Figure 1A A schematic diagram illustrating the generation of a device event, provided as an embodiment of this application;

[0031] Figure 1B A schematic diagram of an intent mapping provided for an embodiment of this application;

[0032] Figure 2 A schematic diagram illustrating a user behavior provided in an embodiment of this application;

[0033] Figure 3 A schematic diagram of an intent prediction process provided for an embodiment of this application;

[0034] Figure 4 A schematic diagram illustrating model determination provided for an embodiment of this application;

[0035] Figure 5 A schematic diagram of an intent prediction process provided in this application embodiment. Figure 2 ;

[0036] Figure 6A A schematic diagram of a service push scenario provided in an embodiment of this application;

[0037] Figure 6BThis application provides an illustration of a service push scenario. Figure 2 ;

[0038] Figure 6C This application provides an illustration of a service push scenario. Figure 3 ;

[0039] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0040] Figure 8 A flowchart illustrating an intent prediction method provided in an embodiment of this application;

[0041] Figure 9 A schematic diagram of the training process for an intent prediction model provided in this application embodiment;

[0042] Figure 10 A schematic diagram of a device event provided for an embodiment of this application;

[0043] Figure 11 A schematic diagram of a model structure provided for an embodiment of this application;

[0044] Figure 12 A schematic diagram of a model structure provided for an embodiment of this application. Figure 2 ;

[0045] Figure 13 A schematic diagram of a model structure provided for an embodiment of this application. Figure 3 ;

[0046] Figure 14 A schematic diagram of a model structure provided for an embodiment of this application. Figure 4 ;

[0047] Figure 15 A schematic diagram of the training process for an intent prediction model provided in this application embodiment. Figure 2 ;

[0048] Figure 16 This is a schematic diagram of an intent distribution provided for an embodiment of this application. Detailed Implementation

[0049] To facilitate a clear description of the technical solutions in the embodiments of this application, the terms "exemplary" or "for example" are used in the embodiments of this application to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present related concepts in a specific manner. In the embodiments of this application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple. In the embodiments of this application, "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "multiple" means two or more.

[0050] To facilitate understanding of the solutions described in the embodiments of this application, a brief introduction to some of the technical names involved in this application will be given below.

[0051] Device events are events that describe device information collected by electronic devices. Device events can reflect user behavior. Specifically, during the use of an electronic device, user actions cause changes to the device's window content, network status, geographical location, motion status, etc., generating device events. One device event represents one occurring event. For example... Figure 1AAs shown, device events include window content change events, network status change events, geolocation change events, and motion status change events. Specifically, window content change events include third-party application events, such as the start or shutdown of a third-party application process, system application events, such as the start or shutdown of a system application process, and screen-on / off events. Network status change events (referred to as network events) include Wi-Fi status events, such as Wi-Fi access, Wi-Fi disconnection, Wi-Fi auto-connection on, and Wi-Fi auto-connection off, as well as cellular mobile network (such as 4G, 5G) and Bluetooth status events. Geolocation change events include: being in an elevator, being in a train car, being on a street, being in a restaurant, etc. Motion status change events (referred to as motion status events) include: walking, running, standing still, jumping, climbing stairs, etc.

[0052] Intent events: These are normalized device events corresponding to the same intent type, that is, unifying similar different device events into the same type of event. For example, device events include the device event corresponding to opening novel application 1 at time 1 and the device event corresponding to opening novel application 2 at time 2. Both of these events correspond to the novel / reading intent; therefore, both of these device events are normalized into novel / reading intent events (e.g., ...). Figure 1B As shown in the image, novel / reading intent events were collected at time 1 and at time 2. It's understandable that intent events are still device events and can still reflect user behavior.

[0053] Intent refers to a user's actual or potential needs. Intent is a classifier that categorizes user needs into different types, with different needs corresponding to different intent types. Fundamentally, the intent predicted by an electronic device enables it to infer the actions a user wants to perform or the services they need in the coming period—that is, to predict which device events the device will perform in the near future, thus allowing the device to decide how to push services.

[0054] Base station: The base station usually refers to a public mobile communication base station, which is a radio transceiver station set up by the network side of the mobile communication system to complete the transmission of radio signals between the network and mobile terminals.

[0055] Sector: A sector is a wireless signal coverage area emitted by a base station that covers a certain geographical area. It is a division of the wireless signal coverage area of ​​the base station. Each sector uses one or more carrier frequencies with specific frequency points (carrier frequency refers to the frequency of the carrier that the base station has available, for example: N carriers, N=1, 3, 6, ...) to complete the coverage.

[0056] Cell: Also known as a cellular cell, ideally hexagonal in shape, it refers to the area covered by a base station or a portion of a base station (sector antenna) in a cellular mobile communication system, within which mobile stations can reliably communicate with the base station via a wireless channel. A cell is a logical concept under a sector; multiple cells can be defined within a single sector based on different carrier frequencies and scrambling codes; a single sector can correspond to multiple cells.

[0057] Geo-fencing is a virtual geographical boundary enclosed by a virtual fence. A geofence can include a geographical area covered by multiple preset base station cells and / or multiple preset wireless signals (e.g., Bluetooth, Wi-Fi). A geofence can be of any shape, such as a circle or a square. It should be noted that a geofence is not necessarily limited by the coverage area of ​​Wi-Fi and / or Bluetooth signals; it can also be a geographical area defined by latitude and longitude. In this way, the mobile terminal can also determine whether it has entered a certain geographical area using the Global Positioning System (GPS). However, generally, the geofence described in this application is limited by the coverage area of ​​Wi-Fi and / or Bluetooth signals. Therefore, compared to determining whether a mobile terminal has entered a certain geographical area using GPS, determining whether the mobile terminal has entered a certain geographical area using the identifiers of Wi-Fi and / or Bluetooth signals is more cost-effective.

[0058] The technical names involved in the embodiments of this application have been introduced above. The technical solutions involved in the embodiments of this application will be introduced below.

[0059] The process of users using electronic devices (such as mobile phones) generally follows a pattern. Taking the time from when a user wakes up until they arrive at work as an example, the process of a user using electronic devices is as follows: Figure 2As shown. First, upon reaching the user's set wake-up time, the electronic device's alarm rings to remind the user to get up. Then, the user watches short videos using a short video app or browses news using a news app. The user checks the weather using a weather app. Next, the user leaves home, and the electronic device leaves the home's geofence. After leaving home, the user connects Bluetooth headphones to the electronic device. Then, the user opens the electronic device's music app to play music using the Bluetooth headphones. Next, the user arrives at a subway station, and the electronic device's location is either the subway station or it enters the subway station's geofence. Then, the user opens the subway ride code to pay for transportation. Next, upon arriving at a subway station near the company, the user opens the subway station, and the electronic device leaves the subway station's geofence. Next, the user enters the company's geofence and opens the electronic device's check-in service to clock in. Next, when buying breakfast, the user opens the electronic device's payment app (i.e., an app with payment functionality) to use its payment functions, such as opening a payment code or scanning a QR code to pay. Finally, the user arrives at the company, and the electronic device connects to the company's Wi-Fi.

[0060] To better serve users, intent prediction models can be trained using user behavior data on electronic devices, i.e., device event data. This allows the intent prediction model to learn user behavior patterns, enabling it to predict user intent and accurately provide the services they need. These behavior patterns can be understood as the user's usage patterns. For example, as mentioned above... Figure 2 The example shown illustrates the process from when a user wakes up until they arrive at the office. After the alarm rings, the electronic device, based on a trained intent prediction model and user usage patterns, predicts that the user's next intention is to read news. The device can then offer a morning news service, pushing news to the user so they can quickly browse the news without having to launch the news app, thus reducing hassle. Furthermore, if the device predicts the user's next intention is to watch short videos, it can push short video services, such as displaying a quick access link for the service, or pre-loading short video apps. This pre-loaded app significantly reduces the time it takes for the device to launch short videos when the alarm rings, thus shortening the user's waiting time.

[0061] In some embodiments, the intent prediction model described above is deployed directly on the electronic device, i.e., on the edge. The electronic device records user behavior data, that is, collects device events. The electronic device then uses this device event data as training samples. Subsequently, the electronic device uses these training samples to train the intent prediction model, so that the trained intent prediction model can be used for intent prediction. In one case, there are multiple intent prediction models, with different models used to predict corresponding types of intents. For example, see... Figure 3 The intent prediction model includes a payment intent prediction model and a music listening intent prediction model. Electronic devices use collected payment events to train the payment intent prediction model, which then predicts whether a user has a payment intent. If the model predicts a payment intent, it pushes a payment code. Here, a payment event represents a device event indicating that the electronic device has performed a payment action.

[0062] Furthermore, electronic devices use collected music playback events to train a music listening intent prediction model. This trained model then predicts whether a user intends to listen to music, and if so, launches the music application. It's understood that when training the music intent prediction model, training samples can include not only payment events but also device events collected within a period prior to those payment events. Similarly, when training the payment intent prediction model, training samples can include not only music playback events but also device events collected within a period prior to those music playback events.

[0063] In another scenario, there is only one intent prediction model. This model can predict different categories of intents, essentially integrating the intent prediction models corresponding to the different intent types described above into a single model. For example, this model can predict payment intent and music listening intent.

[0064] However, relying solely on individual user device events for training samples limits the learning ability of the intent prediction model, impacting its training time and the accuracy of its predicted intent. For instance, a user might open a subway QR code upon arriving at a subway station, but the number of times a user rides the subway each day is limited, say twice. It would take a long time to collect a sufficient number of device events reflecting this behavior, preventing the intent prediction model from learning this behavioral pattern. This results in a longer training time and lower training efficiency for the intent prediction model. Similarly, if user A rarely performs a certain action, their electronic device will collect few device events related to that action, making it difficult for the intent prediction model to predict the corresponding intent. For example, if user A has never worn Bluetooth headphones before, the training samples used to train the intent prediction model will not include device events related to playing music with Bluetooth headphones. Therefore, after user A's electronic device is first connected to the Bluetooth headset, the intent prediction model does not have prior knowledge of the behavioral patterns of using the Bluetooth headset to play music. As a result, the intent prediction model does not need to predict that the user has the intention to listen to music, which limits the prediction of intent and reduces the accuracy of the intent prediction model in predicting intent.

[0065] In addition, due to the limitations of the processing power of electronic devices (i.e., the device side), the size of the model deployed on the device side is limited. Therefore, the training accuracy of the intent prediction model on the device side is limited, which affects the accuracy of intent prediction in the intent prediction mode.

[0066] Therefore, to address the above problems, this application provides a cloud-based collaborative training solution. For example... Figure 4As shown, firstly, the cloud utilizes historical device event data from a group of users—that is, different users—to train a general intent prediction model. This allows the general intent prediction model to learn common behavioral patterns of the group of users, capturing common usage patterns and obtaining a target general intent prediction model. Next, the cloud can distill this target general intent prediction model to obtain a lightweight predictor suitable for edge deployment. To further improve the model's prediction accuracy, after deployment on the edge, the electronic device can use personal device event data collected on the edge to continue training the distilled target general intent prediction model for personalized or scenario-based adaptation, obtaining a target intent prediction model. This target intent prediction model can then serve as a personalized intent prediction model, which can more accurately predict the intent of edge users. After obtaining the personalized intent prediction model, the electronic device can use it for inference. After collecting current device event data, the electronic device inputs this data into the personalized intent prediction model to obtain the intent prediction result. This allows the electronic device to accurately provide corresponding services based on the intent prediction result, improving user satisfaction. The intent prediction result includes the predicted intent.

[0067] Using the timeframe from when a user wakes up until they arrive at the company as an example, this section describes the process of using a personalized intent prediction model for intent prediction. Figure 5 As shown, after the alarm clock rings in the morning, the electronic device collects alarm clock event data and inputs this data into a personalized intent prediction model. The personalized intent prediction model outputs intent prediction result 1. Intent prediction result 1 includes the predicted intent 1. Intent 1 includes news intent, weather intent, and short video intent. Afterwards, the electronic device runs a morning news service, pushing news hotspots (such as...) Figure 6A The top 10 trending news searches shown are displayed, and electronic devices push weather information (such as...). Figure 6A (See weather alert 11). Afterwards, the user views relevant news hotspots through news trending topics 10 and checks the weather through weather alert 11. Similarly, after collecting event data from other devices, the electronic device can input this event data into a personalized intent prediction model, so that the model can provide corresponding services based on the intent prediction results. For example, after the electronic device disconnects from its home Wi-Fi, it leaves its home's geofence. The electronic device inputs the geofence-leaving event data into the personalized intent prediction model. The personalized intent prediction model outputs intent prediction result 2. Intent prediction result 2 includes the predicted intent... Figure 2 The meaning of this prediction Figure 2This includes intents related to subway travel, driving, and ride-hailing. The electronic device pushes subway service, driving service, and ride-hailing service offers. Then, the electronic device connects to Bluetooth headphones. The collected Bluetooth connection event data is input into a personalized intent prediction model to obtain the intent to listen to music. Based on this intent, the electronic device can pre-launch a music application, allowing the user to directly play music and reducing waiting time. Next, the electronic device enters the geofence of a subway station. The geofence event data is input into the personalized intent prediction model to obtain the subway intent. The electronic device can then push subway ride codes (such as...) Figure 6B As shown, the YOYO suggestion pushes the subway ride code 12, enabling subway service notifications. When a user needs to scan the code, they can click on the subway ride code 12, and the electronic device displays as shown below. Figure 6B The subway transit code 13 shown is used for quick display of the subway transit code. Afterwards, the electronic device leaves the subway station's geofence. Then, the electronic device enters the company's geofence, and the event data of entering the company's geofence is input into the personalized intent prediction model to obtain the check-in intent and payment intent. The electronic device then pushes check-in services (e.g., providing...) Figure 6C The quick access entry 14 for the check-in application shown allows users to quickly launch the application and complete check-ins, as well as push payment services (e.g., providing...). Figure 6C The quick access entry 15 for the payment code and the quick access entry 16 for the scan function of Application 1, as shown, allow users to click on the quick access entry 15 when they want to pay with the payment code, and the electronic device will display the payment code of Application 1. Alternatively, users can click on the quick access entry 16 for the scan function when they want to pay with the scan code, and the electronic device will activate the scan function of Application 1 to achieve quick scan.

[0068] It should be understood that the above Figures 6A-6C The example shown is just one instance of a predictive intent-based push service; other methods, such as... Figure 6A The top 10 trending news searches shown is just one example of a news push service; news pushes can also be achieved through methods such as preloading news apps. For example, Figure 6A The weather alert shown in Figure 11 is just one example; weather services can also be pushed through methods such as preloading weather apps.

[0069] For example, the electronic device may be a mobile phone, wearable device (such as a smartwatch or smart bracelet), tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as mobile terminals such as cellular phone, personal digital assistant (PDA), and media player. The embodiments of this application do not impose any special restrictions on the specific form of the electronic device.

[0070] For example, Figure 7 A schematic diagram of the structure of electronic device 200 is shown. For example... Figure 7 As shown, the electronic device 200 may include a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a charging management module 211, a power management module 212, a battery 213, an antenna 1, an antenna 2, a mobile communication module 240, a wireless communication module 250, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a headphone jack 270D, a sensor module 280, buttons 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a subscriber identification module (SIM) card interface 295, etc.

[0071] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 200. In other embodiments of this application, the electronic device 200 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0072] Processor 210 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0073] The controller can be the nerve center and command center of the electronic device 200. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0074] The processor 210 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. This memory can store instructions or data that the processor 210 has just used or that are used repeatedly. If the processor 210 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 210, and thus improves the efficiency of the system.

[0075] In some embodiments, the processor 210 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0076] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 200. In other embodiments of this application, the electronic device 200 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0077] The charging management module 211 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 211 receives charging input from the wired charger via a USB interface 230. In some wireless charging embodiments, the charging management module 211 receives wireless charging input via the wireless charging coil of the electronic device 200. While charging the battery 213, the charging management module 211 can also supply power to the electronic device via the power management module 212.

[0078] The wireless communication function of electronic device 200 can be implemented through antenna 1, antenna 2, mobile communication module 240, wireless communication module 250, modem processor, and baseband processor.

[0079] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 200 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0080] The mobile communication module 240 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 200. The mobile communication module 240 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 240 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 240 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 240 may be housed in the processor 210. In some embodiments, at least some functional modules of the mobile communication module 240 and at least some modules of the processor 210 may be housed in the same device.

[0081] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 270A, receiver 270B, etc.) or displays images or videos through the display screen 294. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 210 and may be housed in the same device as the mobile communication module 240 or other functional modules.

[0082] The wireless communication module 250 can provide solutions for wireless communication applications on the electronic device 200, including wireless local area networks (WLAN) (such as wireless fidelity (WIFI) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 250 can be one or more devices integrating at least one communication processing module. The wireless communication module 250 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 210. The wireless communication module 250 can also receive signals to be transmitted from processor 210, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0083] Electronic device 200 implements display functions through a GPU, a display screen 294, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 294 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 210 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0084] The display screen (or screen) 294 is used to display images, videos, etc. The display screen 294 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 200 may include one or N displays 294, where N is a positive integer greater than 1.

[0085] The external storage interface 220 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 200.

[0086] Internal memory 221 can be used to store computer executable program code, which includes instructions. Processor 210 executes various functional applications and data processing of electronic device 200 by running the instructions stored in internal memory 221. Internal memory 221 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 200 (such as audio data, phonebook, etc.). Furthermore, internal memory 221 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0087] Electronic device 200 can implement audio functions such as music playback and recording through audio module 270, speaker 270A, receiver 270B, microphone 270C, headphone jack 270D, and application processor.

[0088] Button 290 includes a power button, volume buttons, etc. Button 290 can be a mechanical button or a touch button.

[0089] Indicator 292 can be an indicator light, which can be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0090] The sensor module 280 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc.

[0091] As mentioned earlier, during the use of the aforementioned electronic devices, user behavior causes changes in the window content, network status, geographical location, motion status, and device status, generating device events. Device events, ordered chronologically, reflect user behavior patterns, i.e., usage patterns. However, due to the limited nature of individual user device event data, and considering that behavioral patterns may be general (e.g., most users habitually browse news after an alarm rings, or open a subway QR code to ride the subway after entering a subway area), to improve training efficiency, the cloud can train a general intent prediction model using device event data from a group of users. This allows the general intent prediction model to discover common behavioral patterns among the group's users, resulting in a target general intent prediction model capable of accurate intent prediction. Simply put, there may be causal relationships between device events; some device events may lead to others. The general intent prediction model discovers these causal relationships, enabling the target general intent prediction model to predict the resulting events on the electronic device based on the causal events that have already occurred, outputting the intent prediction result. Subsequently, the cloud can simplify the general target intent prediction model to obtain a target intent prediction model, thereby improving prediction speed. The cloud can then deploy this target intent prediction model on the electronic device side, enabling the device to use it for reasoning and output the predicted intent. This provides a reference for the electronic device, allowing it to offer push services related to the predicted intent, enabling users to quickly access these services.

[0092] The following is combined Figure 8 This section describes the process by which electronic devices use a target intent prediction model to perform inference in order to provide the aforementioned morning news service.

[0093] S301. Electronic devices receive target intent prediction models sent from the cloud. These target intent prediction models are used to predict user intent based on learned behavioral patterns of a group of users. The target intent prediction model includes a mapping layer, a change module, an intent attention network layer, and a prediction layer.

[0094] Among them, such as Figure 8As shown, the target intent prediction model includes an embedding layer, a transformer block, an intent attention network layer, and a prediction layer. The embedding layer, transformer block, intent attention network layer, and prediction layer are connected sequentially.

[0095] The embedding layer maps device event data into an initial vector of predefined dimension 1, representing the meaning of the device event data numerically. The device event data includes the device event itself and its corresponding spatiotemporal information. The spatiotemporal information includes the time and location of the event.

[0096] For example, the embedding layer maps device events to initial vectors of a preset dimension of 2, so as to represent the device event using the initial vectors corresponding to the device events. The embedding layer converts the occurrence time of the device event into an initial vector of a preset dimension of 3, so as to represent the occurrence time using the initial vectors. And the embedding layer maps the occurrence location of the device event to an initial vector of a preset dimension of 4, so as to represent the occurrence location using the initial vectors.

[0097] Subsequently, the embedding layer determines the initial vector corresponding to the device event data based on the initial vector corresponding to the device event, the initial vector corresponding to the occurrence time of the device event, and the initial vector corresponding to the occurrence location of the device event. This can be achieved by concatenating the initial vectors corresponding to the device event, the occurrence time, and the occurrence location, or by adding these three initial vectors together.

[0098] Optionally, the occurrence time can include the moment of occurrence and the date of occurrence. The date of occurrence can be calculated on a weekly basis, such as the first day of the week, or it can include weekdays (Monday to Friday) and non-weekdays (Saturday and Sunday). Accordingly, the embedding layer maps the moment of occurrence to an initial vector corresponding to the moment of occurrence, and maps the date of occurrence to an initial vector corresponding to the date of occurrence.

[0099] Among them, preset dimension 2, preset dimension 3, and preset dimension 4 can be the same or different. However, generally speaking, preset dimension 2, preset dimension 3, and preset dimension 4 are the same.

[0100] In addition, preset dimension 1 can be the same as or different from preset dimension 2, preset dimension 3, and preset dimension 4.

[0101] Optionally, the above-mentioned location indicates the identifier of the community, or the location indicates specific latitude and longitude, etc.

[0102] Specifically, the embedding layer can first convert device event data into identifiers according to preset conversion rules, and then map these identifiers to the corresponding initialization vectors of the device event data. For example, the preset conversion rules define identifiers for different device events, such as the identifier 2 for an alarm clock ringing event and the identifier 5 for a geofence event entering a subway station. In addition, the preset conversion rules can also define how to map the occurrence time to identifiers, and how to map the occurrence location to identifiers.

[0103] In some embodiments, the embedding layer can perform bucketing of occurrence times according to preset conversion rules to map occurrence times to identifiers. For example, a day (i.e., 24 hours) can be divided into 30-minute intervals to obtain 48 intervals, in order to determine the interval to which the occurrence time belongs and the identifier corresponding to the interval to which the occurrence time belongs.

[0104] The above article introduced the role of the embedding layer. The following article will continue to introduce the content about the change module.

[0105] The transformation module processes the initial vector corresponding to the device event data to obtain vector 1 with a preset dimension of 5. Compared to the initial vector, vector 1 better expresses the correlation between device events and their corresponding spatiotemporal information, and it more accurately represents the device event data. For example, the transformation module can modify the initial vector corresponding to the device event data based on learned user behavior patterns, such as the historical behavior before and after the device event data within those patterns, i.e., the historical device event data situation. This modification ensures that vector 1 accurately reflects the specific behavior corresponding to the device event data.

[0106] For example, when an app launches, its name might be "Name A," but Name A could have other meanings, such as being a fruit name. Correspondingly, the device event data for this app launch is mapped by the embedding layer to "Name A" of type "fruit name," such as (1,1,1). However, after correction by the change module, the vector 1 corresponding to this device time data (such as (1,2,1)) can express the app launch, thus ensuring the accuracy of intent prediction.

[0107] For example, the aforementioned preset intents include one or more of the following intent types: long video intent, music listening intent, photo taking intent, weather check intent, note taking intent, exercise intent, shopping intent, meeting intent, audio-visual editing intent, food delivery intent, novel / reading intent, express delivery intent, ride-hailing intent, driving intent, subway intent, news intent, short video intent, game intent, learning and education intent, audiobook & podcast intent, and renting / buying a house intent. Of course, the intent types listed here are merely examples; the specific intents included in the preset intents can be set according to needs, and this application does not limit them. For instance, the intent types listed here do not include short video intent and news intent, but include entertainment intent; simply put, entertainment intent includes short video intent and news intent.

[0108] In summary, the initial vector corresponding to the device event data output by the above embedding layer is essentially randomly initialized, and it may not accurately represent the true meaning of the corresponding device event data. Therefore, it is necessary to use the change module to further correct this initial vector.

[0109] The behavioral patterns of the group of users learned by the aforementioned change module can be learned during the training phase. For details, please refer to the following description of the training process of the general intent prediction model using historical device event data.

[0110] The preset dimension 5 is less than the preset dimension 1 mentioned above. In other words, the change module will perform dimensionality reduction processing on the initial vector corresponding to the device event data.

[0111] The above has introduced the role of the change module. The following will continue to introduce the content about the intent attention network layer.

[0112] The aforementioned intent attention network layer is used to calculate the similarity between vector 1 corresponding to the device event data output by the change module and the target intent prediction vector corresponding to each preset intent, thereby determining the probability corresponding to each preset intent and outputting the probability corresponding to each preset intent. The target intent prediction vector corresponding to the preset intent is obtained by training the initial intent prediction vector corresponding to the preset intent based on historical device event data. For details, please refer to the following description of the training process of the general intent prediction model using historical device event data.

[0113] The role of the intent attention network layer has been introduced above. The following section will continue with the content about the prediction layer.

[0114] The prediction layer is used to filter the probabilities corresponding to each preset intent based on the output of the intent attention network layer, to obtain the intent prediction result. The intent prediction result may include the predicted intent. Optionally, the intent prediction result may include the predicted intent and the probability corresponding to each predicted intent. The probability corresponding to the predicted intent represents the probability that the predicted intent will occur. The higher the probability that the predicted intent will occur, the higher the probability that the user behavior corresponding to the predicted intent will appear.

[0115] The above describes the model deployed on the device side. After deploying this model, electronic devices will push corresponding services after collecting device event data (here referred to as third-party device event data). The following uses the collection of an alarm clock ringing event as an example to illustrate the service push process.

[0116] S302. When the alarm clock rings, the electronic device collects alarm clock ringing event data. The alarm clock ringing event data includes the alarm clock ringing event and the corresponding spatiotemporal information.

[0117] The spatiotemporal information corresponding to the alarm ringing event includes the time and location of the alarm ringing event. The alarm ringing event may include user identifier us_id, data item number item_type_id, event collection time event_time, event content content, data item event item_event, and event collection date event_datatime, etc.

[0118] The user identifier us_id is used to distinguish different users. That is, the user identifier us_id of the device events corresponding to the same user is the same, while the user identifier us_id of the device events corresponding to different users is different.

[0119] The item number (item_type_id) refers to the number of each device event. The item number (item_type_id) is different for different device events.

[0120] The data item event (item_event) represents the specific event type of the device event, as shown below. Figure 10 The types shown include screen on / off, geofence trigger (leaving the geofence, entering the geofence), application opening, application opening and exiting, subway QR code (i.e., subway ride code) opening, and motion status.

[0121] The event content is unique to device events of different event types. The event content includes multiple fields and their corresponding values, meaning it contains multiple feature data items. The number and specific values ​​of the feature data may differ between different event content types.

[0122] The event acquisition time (event_time) represents the time when the device event was acquired, that is, the time when the device event occurred. The event acquisition time can be the same as or different from the occurrence time of the corresponding device event.

[0123] The event collection date (event_datatime) represents the date on which the device event was collected. The event collection time can be the same as or different from the occurrence time of the corresponding device event.

[0124] It is understood that the fields, i.e. data items, included in the alarm clock ringing event listed above are only one example of the data items included in a device event. A device event may include only some of the data items listed above, or it may include more data items. This application does not limit this.

[0125] The above provides a brief overview of alarm clock ringing events, which are the components of device events. For more details on device events, please refer to the section on historical device events discussed during the training phase.

[0126] S303. When the electronic device determines that the alarm clock ringing event is a preset triggering event, it inputs the alarm clock ringing event data into the target intent prediction model.

[0127] Among them, the preset trigger event refers to the device event that triggers the target intent prediction model to predict intent.

[0128] In this embodiment, after collecting device event data, such as alarm clock ringing event data, the electronic device determines whether the alarm clock ringing event belongs to a preset trigger event. If the alarm clock ringing event belongs to a preset trigger event, indicating that reasoning needs to be performed using the target intent prediction model, the electronic device inputs the alarm clock ringing event data into the target intent prediction model for intent prediction.

[0129] In addition, if the alarm ringing event is not a preset trigger event, it indicates that there is no need to use the target intent prediction model for reasoning. In this case, the electronic device can continue to collect device event data without inputting the alarm ringing event into the target intent prediction model.

[0130] It should be noted that the above-mentioned determination of whether the alarm ringing event belongs to the preset trigger event is only one example of the triggering condition for the electronic device to use the target intent prediction model to predict intent. The electronic device can also determine whether to use the target intent prediction model to predict intent based on other triggering conditions, such as determining to use the target intent prediction model to predict intent when the current time belongs to a fixed trigger event. In other words, when the preset triggering conditions are met, the electronic device inputs the currently collected device event data into the target intent prediction model. The preset triggering conditions include whether the device event in the currently collected device event data belongs to a preset trigger event, or whether the current time belongs to a preset trigger time.

[0131] In some embodiments, when a preset triggering condition is met, the electronic device can not only input the currently collected device event data (such as the alarm clock ringing event data mentioned above) into the target intent prediction model, but also input the device event data within a time period 1 before the currently collected device event data into the target intent prediction model.

[0132] Optionally, the device event data within a duration of 1 preceding the currently collected device event data can be read from a cache queue. After collecting device event data, the electronic device can insert the device event data into the event cache queue. The cache queue is checked if the time difference between the event collection time of the device event data at the head of the queue and the event collection time of the device event data at the tail of the queue is greater than a duration of 1. Here, the device event data at the head of the queue refers to the earliest inserted device event data, and the device event data at the tail of the queue refers to the latest inserted device event data.

[0133] Optionally, after a device event is collected, the electronic device can perform a cleaning process on the device event. This cleaning process may include removing unnecessary field information from the device event, and / or discarding the device event if it is deemed unnecessary, thereby avoiding interference from irrelevant field information or irrelevant device events with the intended prediction. The unnecessary fields refer to information about fields other than the preset fields corresponding to the event type of the device event (i.e., fields and field values, i.e., feature data). The unnecessary device event refers to the device event corresponding to the preset event type. The specific process for cleaning device events can be found in the process of cleaning historical device events described below.

[0134] In some embodiments, after collecting a device event (such as the alarm clock ringing event mentioned above), the electronic device can determine whether it needs to map the device event to an intent event. After mapping the device event to an intent event, the electronic device can input the intent event into the target intent prediction model. The process by which the electronic device determines whether it needs to map the device event to an intent event can be referred to the mapping process of relevant intent events in the training phase below.

[0135] S304. The embedding layer in the target intent prediction model maps the alarm ringing event data into the initial vector corresponding to the alarm ringing event data, and outputs the initial vector corresponding to the alarm ringing event data.

[0136] The process of mapping alarm ringing event data into corresponding initial vectors can be referred to in the previous section on mapping device event data into corresponding initial vectors in the embedding layer, and will not be repeated here.

[0137] S305. The block module in the target intent prediction model processes the initial vector corresponding to the alarm ringing event data to obtain vector 1 corresponding to the alarm ringing event, and outputs vector 1 corresponding to the alarm ringing event. This vector 1 represents the specific meaning of the alarm ringing event data.

[0138] For example, the processing in S305 above includes correlation processing and / or correction processing to adjust the initial vector corresponding to the alarm ringing event data. Taking this processing including correlation processing and correction processing as an example, the block module performs comprehensive processing on the initial vector based on the alarm ringing event and the spatiotemporal information corresponding to the alarm ringing event to enhance the correlation between the alarm ringing event and the spatiotemporal information corresponding to the alarm ringing event. Furthermore, the block module corrects the initial vector corresponding to the alarm ringing event based on the learned group behavior patterns, such as the historical before and after the alarm ringing event (i.e., the behavior that occurred before the alarm rings and the behavior that occurred after the alarm rings). Simply put, the alarm ringing event is corrected using the historical before and after device events of the alarm ringing event so that vector 1 can more accurately represent the alarm ringing event.

[0139] S306. The Intent Attention Network layer obtains the probability of each preset intent based on the similarity between the vector 1 corresponding to the alarm clock ringing event and the target intent prediction vector corresponding to the preset intent, and outputs the probability of each preset intent.

[0140] In S306, the probability corresponding to the preset intention represents the probability that the preset intention will occur at the next moment in the case of an alarm clock event.

[0141] In the inference phase, the process by which the intent attention network layer determines the probability of the preset intent based on vector 1 and the target intent prediction vector can be found in the section below on determining the probability of the preset intent by the intent attention network layer during the training phase.

[0142] S307. The prediction layer outputs the intention prediction result 1 based on the probability corresponding to each preset intention.

[0143] The intention prediction result 1 may include the predicted intention 1 mentioned above. This predicted intention 1 represents a preset probability that is greater than a preset probability 1. For example, for each preset intention, the prediction layer can determine whether the probability corresponding to that preset intention is greater than the preset probability 1. If the probability corresponding to the preset intention is greater than the preset probability 1, the prediction layer can use that preset intention as the predicted intention 1. If the probability corresponding to the preset intention is less than or equal to the preset probability 1, the prediction layer will not use that preset intention as the predicted intention 1.

[0144] Optionally, the intent prediction result 1 may further include the predicted intent 1 and the probability corresponding to each predicted intent 1. The probability corresponding to the predicted intent 1 represents the probability that the predicted intent 1 may occur, so the electronic device can further determine whether to provide the corresponding service based on the probability corresponding to the predicted intent 1. For example, if the probability corresponding to the predicted intent 1 is greater than a preset probability value 2, it indicates that the predicted intent 1 has a high probability of occurring, and the electronic device can provide the corresponding service based on the predicted intent 1, improving the accuracy of service push. If the probability corresponding to the predicted intent 1 is less than or equal to the preset probability value, it indicates that the predicted intent 1 has a low probability of occurring, and the electronic device may not provide the service corresponding to the predicted intent 1, avoiding providing unnecessary services, reducing the resource consumption of unnecessary services on the electronic device, and avoiding interference with the user.

[0145] For example, intention prediction result 1 includes news intention, weather check intention, and short video intention, with a probability of 0.5 for news intention, 0.4 for weather check intention, and 0.2 for short video intention. The electronic device determines that the probability of news intention and weather check intention are both greater than the preset probability value 2, and the probability of short video intention is less than the preset probability value 2. Therefore, the electronic device provides news services (such as morning news) and weather services, but not short video services.

[0146] S308. The electronic device recommends corresponding services based on the intent prediction result 1 output by the target intent prediction model and in combination with preset recommendation rules.

[0147] Since the target intent prediction model in this application outputs the intent, rather than the specific launch of XX application / XX interface, the electronic device needs to use preset recommendation rules to push the service corresponding to the predicted intent 1.

[0148] In one scenario, preset recommendation rules indicate recommended services based on user habits. For example, if the predicted intent 1 includes a payment intent, and the electronic device has applications with payment functionality installed, then the electronic device can determine the user's frequently used applications with payment functionality based on the user's habits, and prioritize displaying the payment code (or payment code) corresponding to those applications to facilitate user use.

[0149] In another scenario, preset recommendation rules indicate application-based priority recommendation services. Taking the predicted intent 1, which includes a payment intent, as an example, different applications with payment functions have corresponding priorities. An electronic device has applications A and B installed, both of which have payment functions. Application A has a higher priority than application B; therefore, the electronic device can prioritize recommending the payment code corresponding to application A. The application priorities are set by the user or by the electronic device manufacturer based on business needs; this application does not restrict these settings.

[0150] In some embodiments, electronic devices can push services to users through one or more push formats, providing quick access to services and ensuring that information effectively reaches users. For example, push formats include cards, icons, capsules, etc. Taking a payment service as an example, cards and capsules can display text content related to the payment service (such as displaying key information about the payment service) or graphic icons (identifying graphic icons related to the payment service, such as the icon of a payment application).

[0151] Optionally, the aforementioned cards can be displayed on the main interface, the -1 screen, the always-on display (AOD) interface, the lock screen, the notification bar, and other application interfaces. It should be understood that the AOD interface refers to the interface displayed on an electronic device with AOD enabled after the screen is locked or turned off. Cards displayed on the AOD interface can be called AOD cards, and users can view information pushed by the electronic device through these cards when the device is locked.

[0152] The aforementioned graphic icons can display graphic symbols related to the payment service. For example, the icon could be an application icon that provides payment services.

[0153] The capsules described above can be displayed on any display interface. Optionally, the capsules may include status bar capsules and floating capsules.

[0154] In other embodiments, the electronic device may also preload the application corresponding to the predicted intent 1 to enable service push and shorten the time users wait for the application to start.

[0155] It is understandable that during the inference phase, the operations performed by the target intent prediction model in the electronic device, such as the operations performed by the mapping layer, change module, intent attention network layer, and prediction layer in the target intent prediction model, are actually still operations performed by the electronic device.

[0156] The preceding section introduced the process of reasoning using a target intent prediction model. This target intent prediction model is obtained by training a general intent prediction model. The training process of the aforementioned general intent prediction model can be executed in the cloud. The following section will further describe the cloud-based training process of the general intent prediction model. For example, ... Figure 9 As shown, the training process can be as follows:

[0157] S401. For each electronic device in electronic devices 1-N, the electronic device performs cleaning processing on the collected historical device events to obtain the cleaned historical device events corresponding to the electronic device.

[0158] For example, the above cleaning process may include: the electronic device discarding the historical device events corresponding to the preset event type in the historical device events (in other words, retaining the required historical device events), and / or removing information fields other than the preset fields corresponding to the event type of the historical device event in the historical device events (in other words, retaining the required fields), to obtain the cleaned historical device events, thereby reducing the processing volume in the cloud and reducing the impact of irrelevant time or fields on training.

[0159] For example, such as Figure 10 The subway fence trigger events shown in the multiple historical device events include city and station fields. However, user behavior at subway stations is generally consistent across all subway stations, not specific to any particular station. For example, a user opening the subway QR code after entering a subway station is a common action at any subway station, not just at a specific one. Therefore, the city field can be removed from the subway fence trigger events. It should be understood that... Figure 10 Each row of data in the table represents a historical device event.

[0160] In some embodiments, the aforementioned historical device events refer to historical device events collected by the electronic device before the deployment of the aforementioned target intent prediction model. The collection of these historical device events is triggered by the cloud. The cloud sends a sampling task to the electronic device, which describes the sampling object. For example, if the sampling object is a network state change event, then the electronic device only collects network state change events. Alternatively, the sampling task also describes the start and end times of the sampling task. Upon receiving the sampling task, the electronic device collects device events at the start time of the sampling task and stops collecting device events at the end time of the sampling task. Alternatively, the collection of historical device events is automatically triggered by the electronic device. The electronic device can trigger the collection of event streams and the sending of event streams to the cloud within a preset time period, such as the period after network connection, the sleep period, the charging and screen-off period, or the user's rest period (e.g., 2:00-5:00).

[0161] The aforementioned historical device events were determined based on log records.

[0162] S402. For each cleaned historical device event corresponding to the electronic device, if the name of the application in the cleaned historical device event belongs to a preset name set, the electronic device maps the cleaned historical device event to an intent event.

[0163] The preset name set includes one or more preset application names. Each preset name set has a corresponding intent, that is, a corresponding intent type. For example, in this embodiment, the electronic device can determine whether the application name in the cleaned historical device events belongs to any preset name set. If it does, it indicates that the cleaned historical device event can be mapped to an intent event corresponding to the intent type of the preset name set to which the application name belongs, thus achieving intent mapping. This intent event also represents user behavior, that is, it is also a specific device event, but different device events are normalized into one type of device event. For example, compared to a historical device event indicating opening application 1, the intent event obtained by mapping the intent of this historical device event has different field values ​​from the data item event of the historical device event, while the rest of the content can be the same. For example, the historical device event indicates opening application 1, while this intent event indicates a short video intent.

[0164] Additionally, if the application name in the cleaned historical device event does not belong to all preset name sets, the electronic device does not need to map the cleaned historical device event to an intent event.

[0165] Optionally, the name of the application can be the application's package name (packName), such as... Figure 10The application shown has the package name "com.XX".

[0166] It should be noted that if the cleaned historical device event does not have an application name, the electronic device can determine whether to map the cleaned historical device event as an intent event based on the fact that the application name of the cleaned historical device event does not belong to all preset name sets.

[0167] In some embodiments, mapping cleaned historical device events to intent events by application name is only one possible implementation of mapping cleaned historical device events to intent events. Other implementations can also be used to map cleaned historical device events to intent events, such as mapping the cleaned historical device events to intent events corresponding to the intent type corresponding to the preset field when the cleaned historical device events include preset fields.

[0168] In this embodiment, the electronic device normalizes similar historical device events to map them to intent events, thereby increasing the accumulation speed of device events corresponding to similar intent types. This, in turn, improves the accumulation speed of training samples when using historical device events as training samples. Furthermore, since intents better align with general behavioral patterns, it facilitates the learning of user behavior patterns by a general intent prediction model. For example, regardless of whether a user opens "XX Novel" or "XX Reading," it essentially means opening a novel / reading application, enabling the general intent prediction model to learn that the user has a need to open the novel / reading application.

[0169] S403. The electronic device sends historical event data to the cloud. This historical event data includes historical events and their corresponding spatiotemporal information. Historical events include the aforementioned intent events and cleaned historical device events that were not mapped to intent events.

[0170] The spatiotemporal information corresponding to the aforementioned historical events includes the time and location of the events. For descriptions of the events and their locations, please refer to the preceding text.

[0171] In this embodiment of the application, as described above, for each cleaned historical device event, if the name of the application in the cleaned historical device event belongs to any preset name set, the cleaned historical device event (here described as a historical device event in the second historical device event data) is mapped to an intent event (here described as an intent event in the second intent event data), and the electronic device can treat the intent event as a historical event. If the name of the application in the cleaned historical device event does not belong to all preset name sets, there is no need to map the cleaned historical device event (here described as a historical device event in the first historical device event data) to an intent event, and the electronic device can directly treat the cleaned historical device event as a historical event.

[0172] Optionally, the aforementioned historical event data may also include only intent event data, or only cleaned historical device event data that has not been mapped to intent events. In other words, all cleaned historical device event data has not been mapped to intent event data.

[0173] It should be noted that the historical device event cleaning process described in S401 above is an optional step. The operation performed on the cleaned device events is changed to directly operating on the historical device events. For example, mapping the cleaned historical device events to intent events is changed to mapping the historical device events to intent events. Accordingly, the historical events include the intent. Furthermore, mapping historical device events to intent events is also an optional step.

[0174] S404, The cloud uses historical event data sent by various electronic devices as training samples.

[0175] The training samples in S404 can also be called the overall training samples or the group training samples.

[0176] S405. The cloud inputs training samples into the general intent prediction model to train the general intent prediction model and obtain the target general intent prediction model. The target general intent prediction model is used for intent prediction.

[0177] In this embodiment, during the training phase, the cloud uses training samples to train a general intent prediction model. This allows the model to learn the correlation between historical events and spatiotemporal information in the training samples, as well as the causal relationships between historical events, thereby learning the behavioral patterns of a group of users. This enables the trained general intent prediction model to predict user intent based on these behavioral patterns. After training, if the prediction accuracy of the trained general intent prediction model does not meet the requirements, the cloud can continue to train the model using training samples until the prediction accuracy meets the requirements. Afterward, the cloud can use the general intent prediction model with the satisfactory prediction accuracy as the target general intent prediction model.

[0178] Specifically, the cloud can determine whether the prediction accuracy of the general intent prediction model meets the requirements using a loss function. If the loss function value is less than a preset loss value of 1, the cloud determines that the prediction accuracy of the general intent prediction model meets the requirements. However, if the loss function value is greater than or equal to the preset loss value of 1, the cloud determines that the prediction accuracy of the general intent prediction model does not meet the requirements.

[0179] Alternatively, since the prediction of intent is essentially a classification problem, the loss function mentioned above can be the cross-entropy function.

[0180] The training phase of the general intent prediction model has been briefly introduced above. Below, we will use an example where the number of historical event data points in the training samples is 30, each historical event includes the event itself, its location, time of occurrence, and date of transmission, and the number of preset intents is 10. Figure 11 This section provides a detailed introduction to this training phase.

[0181] First, for each of the 30 historical event data points, the embedding layer in the general intent prediction model converts the historical event data into an initial vector, resulting in a sequence of initial vectors. Then, the embedding layer outputs the initial vectors corresponding to each of the 30 historical event data points. The process of mapping historical event data to initial vectors is similar to the process of mapping device events to initial vectors described earlier, and will not be repeated here.

[0182] Subsequently, the aforementioned 30 initial vectors serve as input to the change module in the general intent prediction model. Based on these 30 initial vectors, the change module learns the associations between historical events and their location, time, and date of occurrence, as well as the causal relationships (i.e., correlations) between historical events, thereby learning the behavioral patterns of the user group. Furthermore, for each initial vector, the change module adjusts it to obtain the corresponding historical sequence (historyembedding), which is the historical sequence of the historical event data corresponding to that initial vector. This historical sequence is equivalent to vector 1 described above.

[0183] Each historical sequence has 128 dimensions. Here, 128 can be understood as the preset dimension 2 mentioned above. After obtaining the vector 1 corresponding to each historical event data, the transformation module outputs 30 historical sequences, which the intent attention network layer uses to continue learning the behavioral patterns of the group of users.

[0184] The above-mentioned adjustment process can refer to the changes in the module described above for dimensionality reduction, correction, and integration (i.e., correlation).

[0185] Optionally, the aforementioned change module can employ the GPT-2 model to leverage the knowledge and reasoning capabilities of a large model to learn group behavior patterns.

[0186] Subsequently, the aforementioned 30 historical sequences serve as input to the intent attention network layer. The intent prediction vector calculates the similarity between the 30 historical sequences and the intent prediction vector corresponding to each of the 10 preset intents, and determines the probability corresponding to each preset intent based on the similarity. Preset intents are pre-defined intents, encompassing all types of intents that the general intent prediction model can output. The initial value of the intent prediction vector (intent embedding) is a randomly initialized vector, a process similar to the process described above of mapping the aforementioned historical event data to initial vectors through the embedding layer. There is a one-to-one correspondence between preset intents and intent prediction vectors; the intent prediction vector corresponding to each preset intent represents the specific meaning of that preset intent.

[0187] Subsequently, the aforementioned prediction layer outputs an intent prediction result based on the probability corresponding to the preset intent. This result is then used by the cloud to calculate the loss function value 1, and the parameters in the general intent prediction model are updated through backpropagation based on this loss function value 1, thereby training the intent prediction model. For example, if the loss function value 1 is greater than or equal to the preset loss value 1, the cloud updates the intent prediction vector corresponding to the preset intent, making the intent prediction vector more accurately represent the corresponding preset intent, thus training the intent attention network layer.

[0188] In addition, if the loss function value 1 is less than the preset loss value 1, it indicates that the training of the general intent prediction model can end. Accordingly, the trained intent prediction vector corresponding to the preset intent can be used as the target intent prediction vector corresponding to the preset intent.

[0189] In some embodiments, such as Figure 12 As shown, the aforementioned intent attention network layer can sequentially include an outproduct layer, a contact layer, a PReLU activation function layer, and a linear layer. The outproduct layer is used to capture the correlation between group user behavior and the preset intent. Specifically, during the training phase, the outproduct layer captures the relationship between historical sequences and the intent prediction vector, thereby capturing the correlation between historical user behavior and the preset intent.

[0190] The contact layer concatenates the device event-corresponding vector 1 and the inner product vector between the device event-corresponding vector and the intent prediction vector, thereby combining different feature information into the device event-corresponding vector 1 and enhancing its expressive power. Specifically, during the training phase, the contact layer concatenates the historical sequence and the inner product vector between the historical sequence and the intent prediction vector to form a feature vector containing more information, enhancing the expressive power of the historical sequence.

[0191] The PRelu layer is used to average the feature vectors corresponding to device events to obtain activation features.

[0192] Linear layers are used to map activation features to the output space.

[0193] To better understand the role of each layer in the intent attention network, let's take the training phase as an example, where the number of preset intents is 10, the intent prediction vector corresponding to each preset intent is 128-dimensional, and the historical sequence is also 128-dimensional. Figure 12 This section describes the operations performed in each layer of the intention attention network.

[0194] The cloud inputs all intent prediction vectors (128*10) and historical sequences (30*128) corresponding to all historical device event data into the out product layer. The out product layer calculates the inner product (30*128)*(128*10) between each historical sequence and each intent prediction vector to calculate the similarity between each historical sequence and each intent prediction vector, resulting in an inner product vector C of dimension 30*10 between the historical sequence and the intent prediction vector. Here, Cij in the inner product vector C represents the similarity between the i-th historical sequence and the j-th intent prediction vector.

[0195] Next, the cloud layer concatenates the inner product vector C with the historical sequences through a concatenation layer. That is, for each historical sequence, the inner product between the historical sequence and the corresponding intent prediction vector is concatenated to obtain 30 feature vectors (30*(128+10)). Then, the cloud layer inputs the 30 feature vectors into the PRuelu layer.

[0196] The PRelu layer calculates the mean of 30 feature vectors. Simply put, for each column, it calculates the average of the sums of the elements corresponding to that column in the 30 feature vectors, obtains a 1*138 dimensional activation feature, and outputs the 1*138 dimensional activation feature.

[0197] Next, the cloud inputs the 1*138 dimensional activation features into the Linear layer. The Linear layer calculates the inner product between the activation features and the weight vectors corresponding to each preset intent, i.e., (1*138)*(138*10), to obtain the probability (1*10) corresponding to each preset intent. Each element in 1*10 represents the probability corresponding to each preset probability.

[0198] As can be seen from the above, the weight vector corresponding to each preset intent is 138-dimensional, and the initial value of the weight vector corresponding to the preset intent can be randomly generated, similar to the intent prediction vector corresponding to the preset intent.

[0199] In some embodiments, the aforementioned change module may be a reused conventional model structure (such as...) Figure 13 As shown, training can be performed in the conventional manner. Alternatively, to enable the change module to better learn the sequential relationships between historical events, it can be trained through event reconstruction. For example, the event reconstruction layer in the intent prediction model can reconstruct the masked historical sequence corresponding to a historical event and return the reconstructed historical sequence to the change module. The change module compares the reconstructed historical sequence corresponding to the historical event with the actual historical sequence corresponding to the historical event to update the change module, aiding its training and enhancing the learning of the conversion pattern from device events to intents.

[0200] For example, such as Figure 14As shown, after the mapping layer outputs initial vectors corresponding to 30 historical event data, the cloud layer masks 10 of these 30 initial vectors. Then, the cloud layer inputs the 10 masked initial vectors and 20 complete initial vectors (i.e., the 20 unmasked initial vectors) into the transformation module. The transformation module adjusts the 10 masked initial vectors and 20 complete initial vectors to obtain 10 masked historical sequences and 20 complete historical sequences, and outputs them. Next, the event reconstruction layer, based on the sequential relationship between the 10 masked historical sequences and the 20 complete historical sequences, restores the 10 masked historical sequences to obtain 10 restored historical sequences, and returns these 10 restored historical sequences to the transformation module.

[0201] Next, the transformation module compares the historical sequences corresponding to the 10 initial vectors (i.e., the actual historical sequences) with the 10 reconstructed historical sequences. If they are inconsistent, the transformation module can update its own parameters to more accurately adjust the masked initial vectors, enabling the event reconstruction layer to accurately reconstruct the historical sequences and allowing the general intent prediction model to understand the correlation before and after the event. If they are consistent, the transformation module does not need to be updated.

[0202] The transformation module compares the historical sequence corresponding to the 10 initial vectors with the restored historical sequence corresponding to the 10 initial vectors to determine whether the transformation module needs to be trained further. This is equivalent to calculating the loss function.

[0203] It should be noted that during the training phase, the operations performed by the general intent prediction model are actually executed in the cloud. Optionally, the cloud can be a device with high processing power, such as a server.

[0204] Understandably, if the time difference between the collection times of two consecutive events in the aforementioned historical device events is greater than a duration of 2, it indicates a low correlation between the two events, meaning a low probability of a causal relationship. Correspondingly, during the inference phase, when collecting the preceding event, the electronic device inputs that preceding event into the target intent prediction model. The target intent prediction model outputs a null value, indicating no intent.

[0205] In some embodiments, after training a general intent prediction model to obtain a target general intent prediction model, the cloud can directly use this target general intent prediction model as the target intent prediction model. Alternatively, to improve prediction speed, the cloud can distill the target general intent prediction model to simplify it. The distilled target general intent prediction model is a lightweight prediction model, suitable for deployment on the edge. This distilled target intent prediction model can be used as the target intent prediction model. Simply put, the cloud can streamline a large model (i.e., the target general intent prediction model) into a smaller model (the target intent prediction model), thereby reducing the resources occupied by the model on electronic devices. This ensures both the accuracy and speed of the target general intent prediction model in predicting intent.

[0206] Optionally, the above distillation can be teacher-student distillation. The above target general intent prediction model can be understood as the teacher model, and the distilled target general intent prediction model can be the student model. The student model can learn fine-grained information from the output of the teacher model, as well as learn basic classification ability.

[0207] In some embodiments, the target intent prediction model is not directly sent from the cloud; instead, the cloud sends a distilled general target intent prediction model to the electronic device. Since there may still be differences between individual user behavior patterns and group user behavior patterns, to improve prediction accuracy, after deploying the distilled general target intent prediction model, the electronic device can use user-specific behavior data collected on the device side—i.e., device events—to further train the distilled general target intent prediction model, achieving fine-tuning and obtaining a target intent prediction model that can efficiently and accurately predict user intent. The following will combine... Figure 15 Continue the fine-tuning process of the target intent prediction model.

[0208] S501. After deploying the distilled target general intent prediction model, the electronic device collects personal device event data.

[0209] Personal device event data (or first device event data) may include personal device events and the spatiotemporal information corresponding to the personal device events. For details, please refer to the relevant introduction of device event data mentioned above.

[0210] In this embodiment of the application, after the distilled target general intent prediction model is deployed on the device side, the electronic device continues to collect device event data (i.e., personal device event data) so as to use the personal device event data to incrementally train the distilled target general intent prediction model, so that the distilled target general intent prediction model can use the personalized user behavior data collected locally on the device side to capture the user's personalized behavior pattern, thereby realizing the personalized adaptation of the model.

[0211] S502. The electronic device maps personal device event data to obtain individual intent event data. Each individual intent event has a corresponding intent type.

[0212] The process of mapping personal device event data to obtain intent event data (i.e., individual intent event data) can refer to the relevant intent event mapping content introduced above, such as the content related to mapping cleaned historical device events to intent events described in S402 above. Furthermore, similar to the previous description, the steps in S502 above are also optional.

[0213] S503. Electronic devices use personal event data as individual training samples. This personal event data includes individual intent event data and unmapped personal device event data.

[0214] In this embodiment of the application, when the collected personal device event data meets the requirements, such as when the collected personal device event data reaches a preset quantity, or when the duration of collected personal device event data reaches a duration of 3, the electronic device can use the mapped individual intent event data (or first intent event data) and the personal device event data not mapped to individual intent events (or second device event data) as personal event data (or first event data). This personal event data can be used as a sample for training and deploying the distilled target general intent prediction model, i.e., individual training samples.

[0215] The process for determining personal event data can refer to the relevant content on historical event data described in S403 above.

[0216] The process of determining the individual training samples required to train the model deployed on the edge has been described above. The process of training the model using individual training samples will be described below.

[0217] S504. The electronic device trains the distilled target general intent prediction model based on the individual training sample to obtain the trained target general intent prediction model.

[0218] S505. After this training is completed, the electronic device determines the set of intentions to be forgotten and the set of intentions to be retained. The union of the set of intentions to be forgotten and the set of intentions to be retained is the set of preset intentions.

[0219] The set of preset intentions includes all preset intentions. The set of intentions to be forgotten, F, includes one or more intentions to be forgotten, which represent the intentions that need to be forgotten on the end side.

[0220] The set of intentions to be retained, R, includes one or more intentions to be retained. Intentions to be retained represent those that need to be retained on the end side, and are intentions other than those to be forgotten. For example, if there are 10 preset intentions, and the set of intentions to be forgotten, F, includes 3 preset intentions, then the set of intentions to be retained, R, includes the remaining 7 preset intentions excluding these 3 preset intentions.

[0221] Optionally, the aforementioned set of intentions to be forgotten, F, may include the set of collective forgetting intentions, F. sta and / or the set of individual forgetting intentions F dyn The set of intentions to be forgotten, F, includes the set of intentions to be forgotten by the group, F. sta The set of individual forgetting intentions F dyn In this case, it shows that F = F sta ∪F dyn .

[0222] The above-mentioned group forgetting set F sta This includes at least one pre-defined intention to be forgotten by the group. For each pre-defined intention, the electronic device can determine whether to treat it as a pre-defined intention to be forgotten by the group based on the proportion of the group distribution corresponding to that pre-defined intention. Since the general intention prediction model in the cloud also outputs predicted intentions during the training phase, the cloud can statistically analyze the probability of each pre-defined intention appearing from the start of training to the end of training, and obtain the proportion of each pre-defined intention in the group distribution (i.e., the percentage), that is, obtain the proportion of the group distribution corresponding to each pre-defined intention.

[0223] For example, the preset number of intents is 4: intent a, intent b, intent c, and intent d. The general intent prediction model is trained twice. After the first training, the general intent prediction model outputs predicted intents including intent a, intent b, and intent c. After the second training, the general intent prediction model outputs predicted intents including intent a, intent b, and intent d. Intent a appears 2 times, intent b appears 2 times, and intent c and intent d each appear 1 time. For intent a, its corresponding group distribution percentage is 2 / (2+2+1+1). Here, the numerator 2 represents the number of times intent a appears, the two 2s in the denominator represent the number of times intent a appears, and the two 1s represent the number of times intent c appears, and the two 1s represent the number of times intent d appears.

[0224] Similarly, for intention b, the corresponding group distribution proportion is 2 / (2+2+1+1). For intention c or intention d, the corresponding group distribution proportion is 1 / (2+2+1+1).

[0225] The larger the proportion of the group distribution corresponding to the preset intent, the more times the cloud-based general intent model outputs that preset intent during the training phase, meaning the higher the probability of that preset intent appearing. Conversely, the smaller the proportion of the group distribution corresponding to the preset intent, the fewer times the cloud-based general intent model outputs that preset intent during the training phase, meaning the lower the probability of that preset intent appearing.

[0226] Therefore, for each preset intention, if the proportion of the group distribution corresponding to the preset intention (or the proportion of output times) is less than the preset proportion 1 (or the first threshold), the electronic device can regard the preset intention as a group forgetting intention, i.e., I i ∈F sta ifP out (I i )<∈。where, I i P represents the i-th preset intention. out (I i ) represents the population distribution percentage corresponding to the i-th preset intention. ∈ indicates a preset percentage of 1. When the population distribution percentage corresponding to a preset intention is greater than or equal to a preset percentage of 1, it indicates that the preset intention has a high probability of occurrence. Therefore, the electronic device does not treat this preset intention as a population forgotten intention. It should be understood that for population forgotten intentions, all electronic devices that have deployed the distilled target general intention prediction model need to forget them.

[0227] The above set of individual forgetting intentions F dynThis includes at least one individual forgetting intention. For each preset intention, if the preset intention appears infrequently during actual group use but appears more frequently during actual use on the electronic device side (i.e., less frequently in the group but more frequently in the individual), it indicates insufficient model learning, and the electronic device can treat this preset intention as an individual forgetting intention. For example, if the proportion of the group training sample corresponding to the preset intention is less than the average proportion of intentions, and the proportion of the individual training sample corresponding to the preset intention is greater than the average proportion of intentions, the electronic device can treat this preset intention as an individual forgetting intention (e.g., ...). Figure 16 The preset meaning shown Figure 3 ). That is I i ∈F dyn ifP pop (I i )<1 / N andP in (I i P > 1 / N. Where P pop (I i ) represents the proportion of group training samples corresponding to the i-th preset intention. in (I i ) represents the proportion of individual training samples corresponding to the i-th preset intent. 1 / N represents the average proportion of intents, and N represents the number of preset intents.

[0228] The process of determining the proportion of individual training samples corresponding to the i-th preset intent is as follows: Count the number of individual intent events corresponding to each preset intent in the individual training samples. The proportion of individual training samples corresponding to the i-th preset intent (or the proportion of occurrences in the first event data) is equal to the ratio of the number of individual intent events corresponding to the i-th preset intent to the sum of the number of individual intent events corresponding to all preset intents. Alternatively, the proportion of individual training samples corresponding to the i-th preset intent is equal to the ratio of the number of individual intent events corresponding to the i-th preset intent to the total number of individual training samples (i.e., the total number of events in the individual training samples).

[0229] Similarly, the process of determining the proportion of the group training samples corresponding to the i-th preset intent is as follows: count the number of intent events corresponding to each preset intent (i.e., intent type) in the group training samples. The proportion of the group training samples corresponding to the i-th preset intent (or the proportion of occurrences in historical event data) is equal to the ratio of the number of intent events corresponding to the i-th preset intent to the sum of the number of intent events corresponding to all preset intents. Alternatively, the proportion of the group training samples corresponding to the i-th preset intent is equal to the ratio of the number of individual intent events corresponding to the i-th preset intent to the total number of group training samples.

[0230] It is understandable that the aforementioned set of group forgetting intentions and set of individual forgetting intentions can be determined collaboratively by electronic devices and the cloud. For example, the cloud can directly send the set of group forgetting intentions and the proportion of group training samples corresponding to each preset intention to the electronic device. Alternatively, the cloud can send relevant data for determining the set of group forgetting intentions to the electronic device, so that the electronic device can determine the set of group forgetting intentions.

[0231] S506. The electronic device calculates the forgetting loss function value based on the set of intentions to be forgotten and the set of intentions to be retained.

[0232] In this embodiment of the application, after determining that the electronic device needs to forget a preset intention, that is, after determining the intention to be forgotten corresponding to the electronic device, in order to enable the electronic device to effectively forget the intention to be forgotten, a preset forgetting loss function (i.e., L) can be used. CE (M r ,R)-λL CE (M f The loss function value is calculated to obtain the forgetting loss function value, so that the electronic device can use the forgetting loss function value to update the trained target general intent prediction model, thereby minimizing the loss between the output of the trained target general intent prediction model and the output of the original distilled target general intent prediction model with respect to the intent to be retained, and maximizing the loss between the two outputs with respect to the intent to be forgotten.

[0233] Among them, L CE () denotes the cross-entropy function, M r This indicates a specific intention to retain the item. M f This represents the specific intention to be forgotten. λ represents a hyperparameter used to balance the trade-off between the intention to be forgotten and the intention to be retained; it is a preset value. λ was determined experimentally. If λ is too small, the model focuses more on learning the intention to forget, while if it is too large, the model focuses more on learning the intention to retain.

[0234] S507. The electronic device determines whether the value of the forgetting loss function is less than the preset loss value 2.

[0235] In this embodiment of the application, if the forgetting loss function value is less than the preset loss value 2 (or the fourth threshold), it indicates that the prediction accuracy of the trained target general intent prediction model meets the requirements, and training can be stopped. Then the electronic device can execute S508.

[0236] If the forgetting loss function value is greater than or equal to the preset loss value of 2, it indicates that the prediction accuracy of the trained target general intent prediction model does not meet the requirements and needs to be trained again. In this case, the electronic device can execute S509.

[0237] S508, the electronic device uses the trained target general intent prediction model as the target intent prediction model.

[0238] After obtaining the target intent prediction model, the electronic device can use this model to perform reasoning to realize the predicted intent. This reasoning process can be referenced from the reasoning process described above, and will not be repeated here. Furthermore, it is understood that compared to the aforementioned general intent prediction model, the target intent prediction model has the same structure, but the parameters may differ.

[0239] S509, Electronic device returns to S504.

[0240] In this embodiment, after deploying a lightweight, distilled, general-purpose intent prediction model on the electronic device side, it can be fine-tuned using personalized user behavior data collected from the electronic device side. Furthermore, considering the differences in the distribution of behavior data between group users and individual users—that is, the differences in behavior patterns—a model trained solely on group user behavior data may not effectively capture the long-tail distribution of data on the electronic device side. Therefore, an intent forgetting method adapted to the electronic device side is used to train the distilled, general-purpose intent prediction model, enabling it to adapt to and learn edge-side behavior patterns, achieving personalized enhancement of the edge-side model, and thus enabling accurate intent prediction.

[0241] In some embodiments, the structure of the general intent prediction model described above is merely an example, and other types of classification models may also be used; this application does not limit them. Correspondingly, the training process of the general intent prediction model described above is also merely an example.

[0242] In some embodiments, to improve the accuracy of intent prediction, the target general intent prediction model can also be directly deployed on the electronic device side in the cloud. Accordingly, the description of the distilled target general intent prediction model in S501-S508 above can be replaced with the target general intent prediction model.

[0243] It should be noted that the collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solutions disclosed in this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals. Furthermore, the personal information (such as user behavioral data) used in the technical solutions of this application is limited to information for which separate consent has been obtained, including but not limited to notifying and reminding users to read the relevant user agreement (notification) and sign the agreement (authorization) which includes authorization of relevant user information before users use the function.

[0244] In some embodiments, this application provides a computer-readable storage medium including computer instructions that, when executed by an electronic device, cause the electronic device to perform the method performed by the electronic device as described above.

[0245] In some embodiments, this application provides a computer-readable storage medium including computer instructions that, when executed on a server, cause the server to perform the server-executed method as described above.

[0246] In some embodiments, this application provides a computer program product that, when run on an electronic device, causes the electronic device to perform the method performed by the electronic device as described above.

[0247] In some embodiments, this application provides a computer program product that, when run on a server, causes the server to execute the server-executed method as described above.

[0248] In some embodiments, this application provides a chip including a communication interface and at least one processor: the communication interface for inputting and / or outputting signaling or data; and at least one processor for executing a computer program to implement the method described above.

[0249] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0250] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0251] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0252] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0253] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0254] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A model training method, characterized in that, Applied to electronic devices, the method includes: Receive a first intent prediction model; wherein the first intent prediction model is obtained by training a general intent prediction model based on historical event data of multiple users, so that the general intent prediction model learns the behavioral habits of multiple users; the historical event data is generated based on the user's historical behavior. Collect first device event data; the first device event data is generated by the electronic device based on user behavior; The first intent prediction model is trained based on the first device event data; After training, the forgetting loss function value is determined based on the set of intentions to be forgotten and the set of intentions to be retained. The intentions to be forgotten in the set of intentions to be forgotten represent the preset intentions that the first intention prediction model learns to forget. The intentions to be retained in the set of intentions to be retained include all preset intentions except for the intentions to be forgotten. The preset intentions represent the intentions that the first intention prediction model can output. Based on the forgotten loss function value and the trained first intent prediction model, a target intent prediction model is determined; wherein, the target intent prediction model is used to predict intent based on device event data.

2. The method according to claim 1, characterized in that, The step of training the first intent prediction model based on the first device event data includes: The first intent prediction model is trained based on the first event data; wherein, the first event data includes first intent event data and second device event data; the first intent event data is obtained by performing intent mapping on device event data other than the second device event data in the first device event data.

3. The method according to claim 1 or 2, characterized in that, The set of intentions to be forgotten includes a first set of forgetting intentions; The intents to be forgotten in the first forgetting set represent the preset intents whose output frequency is less than a first threshold during the training of the general intent prediction model.

4. The method according to claim 2 or 3, characterized in that, The historical event data includes first historical device event data and second intent event data, wherein the second intent event data is obtained by performing intent mapping on the second historical device event data; the first historical device event data is different from the second historical device event data. The set of intentions to be forgotten includes a second set of forgetting intentions; The intentions to be forgotten in the second set of forgetting intentions represent preset intentions that have a lower percentage of occurrence in the historical event data than a second threshold and a lower percentage of occurrence in the first event data than a third threshold.

5. The method according to any one of claims 1 to 4, characterized in that, The forgetting loss function value is based on L CE (M r ,R)-λL CE (M f The result is obtained by calculating F; where L is the value of L. CE () is the cross-entropy function, M r R is the set of intentions to be retained, and M is the set of intentions to be retained. f Let λ represent the intention to be forgotten, λ be a preset value, and F represent the set of intentions to be forgotten.

6. The method according to any one of claims 1 to 5, characterized in that, The step of determining the target intent prediction model based on the forgetting loss function value and the trained first intent prediction model includes: If the value of the forgetting loss function is less than the fourth threshold, the trained first intent prediction model is used as the target intent prediction model.

7. The method according to any one of claims 1 to 6, characterized in that, The first intent prediction model is obtained by distilling the target general intent prediction model; the target general intent prediction model is obtained by training the general intent prediction model.

8. The method according to any one of claims 1 to 7, characterized in that, After determining the target intent prediction model, the method further includes: Collect event data from third-party devices; The third device event data is input into the target intent prediction model to obtain the intent prediction result; Based on the intent prediction results, corresponding services are pushed.

9. The method according to claim 8, characterized in that, The third device event data includes alarm clock ringing event data; the intent prediction result includes news intent. The step of pushing corresponding services based on the intent prediction result includes: Based on the aforementioned news intent, a morning news service is pushed out.

10. A service push method, characterized in that, The method further includes: Collect event data from third-party devices; Using the third device event data as input, the target intent prediction model according to any one of claims 1-9 is run to obtain the intent prediction result; Based on the intent prediction results, corresponding services are pushed.

11. An electronic device, characterized in that, The electronic device includes a memory and one or more processors; the memory and the processors are coupled; the memory is used to store computer program code, the computer program code including computer instructions; when the processor executes the computer instructions, the electronic device performs the model training method as described in any one of claims 1 to 9, or the service push method as described in claim 10.

12. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the model training method as described in any one of claims 1 to 9, or the service push method as described in claim 10.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the model training method as described in any one of claims 1 to 9, or the service push method as described in claim 10.

Images