Data deduplication methods, electronic devices, storage media and chips

Abstract

This application discloses a data deduplication method, electronic device, storage medium, and chip. The method includes: acquiring data to be deduplicated, the data including M usage records, the M usage records including L event usage records, the L events including S1 first events, all usage records of each first event being located in at least one time period, and each time period within at least a portion of the time periods corresponding to at least two usage records of the first event; for each first event, deleting the remaining usage records other than the first usage record from the at least two usage records corresponding to the first event in each of the at least a portion of the time periods, to obtain a deduplication result, wherein the first usage record is the usage record with the earliest usage time among the at least two usage records of the first event corresponding to each of the at least a portion of the time periods. This method can improve the accuracy of the data deduplication result, thereby improving the quality of the deduplicated data.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a data deduplication method, electronic device, storage medium, and chip. Background Technology

[0002] With the continuous development of computer and network technologies, the amount of data stored has also increased exponentially. This massive amount of data (such as usage records generated by electronic devices) can be stored in databases.

[0003] Currently, with the surge in the amount of data stored in databases, if the data volume is not reduced, redundant and dirty data will be stored, resulting in poor data quality. Therefore, how to accurately deduplicate data is an urgent problem to be solved. Summary of the Invention

[0004] This application provides a data deduplication method, electronic device, storage medium, and chip, which can improve the accuracy of data deduplication results, thereby improving the quality of deduplicated data.

[0005] Firstly, this application provides a data deduplication method, which includes:

[0006] Obtain the data to be deduplicated, wherein the data to be deduplicated includes M usage records, the M usage records include L event usage records, the L events include S1 first events, all usage records of each first event are located in at least one time period, and each time period of at least some time periods within at least one time period corresponds to at least two usage records of the first event, M is an integer greater than 1, and L and S1 are positive integers;

[0007] For each first event, in at least two usage records corresponding to each time period of the first event, delete the remaining usage records other than the first usage record to obtain the deduplication result. The first usage record is the usage record with the earliest usage time in the at least two usage records corresponding to each time period of the first event. The deduplication result includes the usage records in M ​​usage records other than the remaining usage records corresponding to each time period of the first event.

[0008] Optionally, multiple time periods can correspond to multiple time buckets, and the time period of each time bucket is the corresponding time period.

[0009] In the above technical solution, when performing deduplication on M usage records, for each first event, in the case of at least two usage records corresponding to each time period of the first event, it is necessary to delete the remaining usage records other than the usage record with the earliest usage time (i.e., the first usage record) among the at least two usage records corresponding to the first event in each time period. In other words, based on the data deduplication method provided in this application, for each first event, in the case of multiple usage records for each first event within each time period, only the usage record of each first event that first appears in each time period (i.e., the first usage record) is retained. Therefore, this method can avoid the problem of missed deduplication in traditional techniques when performing data deduplication. In summary, the data deduplication method provided in this application can improve the accuracy of data deduplication results, thereby improving the quality of the deduplicated data.

[0010] In one possible implementation, the L events in the deduplication result include S2 second events, wherein the deduplication result specifically includes multiple usage records for each second event, and S2 is a positive integer; and, after deleting the remaining usage records other than the first usage records from at least two usage records corresponding to each time period of the first event for each first event to obtain the deduplication result, the method further includes:

[0011] If the interval between the usage times of at least two usage records in multiple usage records of each second event does not exceed a first threshold, delete the usage record with the latest usage time among the at least two usage records of each second event.

[0012] The deduplication result is obtained by performing the above deduplication process on the M usage records of the data to be deduplicated. It can be seen that the L events in the deduplication result are the same as the L events in the M usage records, but the number of usage records corresponding to the L events in the deduplication result is different from the number of usage records in the M usage records of the data to be deduplicated.

[0013] The multiple usage records of each second event in S2 events are located in multiple time periods, and there are no specific restrictions on the mapping relationship between the multiple usage records and the multiple time periods.

[0014] The L events in the deduplication result include S2 second events, each of which may be the same as or different from each of the S1 first events mentioned above.

[0015] The value of the first threshold is not specifically limited and can be set according to actual needs. For example, the first threshold can be, but is not limited to, 2 seconds or 3 seconds.

[0016] In the above technical solution, after performing data deduplication on each of the S1 first events among the M usage records included in the deduplication data, the resulting deduplication result includes multiple usage records of each second event located in multiple time periods. Based on this, a second short-term repetitive behavior data deduplication can be performed on each second event. That is, among at least two usage records with a small interval between their usage times, the usage record with the latest usage time among at least two usage records of each second event is deleted. This method can remove some events of the same type that happen to be distributed at the edge of adjacent time periods, minimizing the data noise in the deduplication result obtained after the second deduplication, improving the accuracy of the data deduplication result, and thus improving the quality of the deduplicated data.

[0017] In another possible implementation, the L events in the deduplication result include S3 third events, wherein each third event corresponds to an associated event, multiple usage records of each third event are consecutive, and multiple usage records of each third event are adjacent to the usage records of the corresponding associated events, and S3 is a positive integer; and, after deleting the remaining usage records other than the first usage records from at least two usage records of the first event corresponding to each time period for each first event to obtain the deduplication result, the method further includes:

[0018] If the duration between the usage time of the usage record of the associated event corresponding to each third event and the usage time of the multiple usage records of each third event does not exceed the first preset duration, delete the usage records of each third event except for the earliest usage record.

[0019] Multiple usage records for each third event are located in multiple time periods, and the mapping relationship between these multiple usage records and multiple time periods is not specifically limited.

[0020] The third event corresponds to related events. These two related events can be two events that have a relative connection or obvious correlation; there are no specific limitations on the two related events. For example, the two related events could include a "plugging in" event and a "plugging out" event. Similarly, the two related events could include a "device connected" event and a "device disconnected" event.

[0021] The deduplication result includes the usage records of the associated events corresponding to each third event and multiple consecutive adjacent usage records of each third event.

[0022] For example, if the third event is the "unplug" event and the associated event is the "plug" event, then the three consecutive usage records of the "unplug" event and the adjacent usage records of the "plug" event can be described as: plug, unplug, unplug, unplug; or unplug, unplug, unplug, plug.

[0023] In the above technical solution, after performing data deduplication based on time features (i.e., the time period mentioned above) on each of the S1 first events in the M usage records to be deduplicated, the resulting deduplication result includes each third event corresponding to a related event. Subsequently, deduplication is performed based on the correlation between events. That is, if the usage time of the usage record of the related event corresponding to each third event is short, the usage records of each third event are deleted except for the earliest usage record. This method performs data deduplication from the two dimensions of time features and correlation between events. This method has high versatility and robustness, which can improve the accuracy of data deduplication results and thus improve the quality of the deduplicated data.

[0024] In another possible implementation, the L events in the deduplication result include S4 fourth events and there are no associated events corresponding to each fourth event, where S4 is a positive integer; and, after deleting the remaining usage records other than the first usage record from at least two usage records corresponding to each of the first events in at least some time periods to obtain the deduplication result, the method further includes:

[0025] If the total usage duration corresponding to all usage records for each fourth event exceeds the second preset duration, delete all usage records for each fourth event.

[0026] In the above technical solution, after performing data deduplication based on time characteristics (i.e., the time period mentioned above) on each of the S1 first events in the M usage records to be deduplicated, the resulting deduplication result includes each fourth event corresponding to a related event, but does not include the related events corresponding to each fourth event. Subsequently, deduplication is performed based on the correlation between events; that is, if the usage duration corresponding to all usage records of each fourth event is large, then all usage records of that fourth event are deleted. In other words, this method performs deduplication on the data from two dimensions: time characteristics and the correlation between events. This method has high versatility and robustness, and can improve the accuracy of the data deduplication results, thereby improving the quality of the deduplicated data.

[0027] In another possible implementation, after deleting the remaining usage records other than the first usage record from at least two usage records corresponding to each of at least a portion of the time period for each first event to obtain the deduplication result, the method further includes:

[0028] If there is a logical error in the event result of one of the two usage records included in the deduplication result, delete the usage record with the logical error.

[0029] Two usage records correspond to two different events, or two usage records correspond to the same event.

[0030] In the above technical solution, after performing data deduplication based on time features (i.e., the time period mentioned above) on each of the S1 first events in the M usage records to be deduplicated, if there is a logical error between two usage records included in the deduplication result, the usage record with the logical error is deleted. This method performs deduplication on the data from the two dimensions of time features and event logic. This method has high versatility and robustness, which can improve the accuracy of the data deduplication result, thereby improving the quality of the deduplicated data.

[0031] In another possible implementation, after deleting the remaining usage records other than the first usage record from at least two usage records corresponding to each of at least a portion of the time period for each first event to obtain the deduplication result, the method further includes:

[0032] If the deduplication result contains multiple usage records corresponding to a single event that contain logical errors, delete at least one usage record with a logical error from among the multiple usage records corresponding to the single event; or...

[0033] If, in the deduplication result, at least part of the usage record of one of the two related events contains a logical error, the usage record of the one event containing the logical error in the usage record of the other event shall be deleted.

[0034] In the above technical solution, after performing data deduplication based on time features (i.e., the time period mentioned above) for each of the S1 first events in the M usage records to be deduplicated, if there are logical errors in multiple usage records corresponding to one or two events in the deduplication result, the usage records with logical errors will be deleted. This method performs deduplication on the data from the two dimensions of time features and the logic of related events. This method has high versatility and robustness, which can improve the accuracy of data deduplication results, thereby improving the quality of the deduplicated data.

[0035] In another possible implementation, the deduplication result includes at least one usage record of the fifth event in the two related events; and, after deleting the remaining usage records other than the first usage record from at least two usage records of the first event corresponding to each time period for at least a portion of the time period to obtain the deduplication result, the method further includes:

[0036] If no usage record for the sixth event in the two related events is detected, delete at least one usage record for the fifth event; or,

[0037] If at least one usage record of the sixth event is detected, and the minimum interval between the usage time of at least one usage record of the fifth event and the usage time of at least one usage record of the sixth event exceeds a second threshold, the usage record of the fifth event associated with the minimum interval is deleted.

[0038] In the above technical solution, if both related events (i.e., the fifth event and the sixth event) occur within a short period of time, these two related events can be considered as real events, and the usage records of these real events need to be retained during data deduplication. If only one of the two related events occurs within a short period of time, or if the interval between the occurrence of the two related events is long, then these two related events are considered as not real events, and the usage records of these not real events need to be deleted during data deduplication.

[0039] Secondly, this application provides a data deduplication method, which includes:

[0040] Obtain the data to be deduplicated, wherein the M usage records in the data to be deduplicated include S events, each event corresponds to an associated event, multiple usage records of each event are consecutive, and multiple usage records of each event are adjacent to the usage records of the corresponding associated events, M is an integer greater than 2, and S is a positive integer;

[0041] If the time between the usage record of the associated event corresponding to each event and the usage time of multiple usage records for each event does not exceed the preset time, delete the usage records of each event except for the earliest usage record.

[0042] Thirdly, this application provides a data deduplication method, which includes:

[0043] Get the data to be deduplicated, where the M usage records in the data to be deduplicated include S events and there is no associated event for each event. Each event has an associated event. M is an integer greater than 1 and S is a positive integer.

[0044] If the total usage duration for all usage records of each event exceeds a preset duration, delete all usage records for that event.

[0045] Fourthly, this application provides a data deduplication method, which includes:

[0046] Get the data to be deduplicated, which includes M usage records, where M is an integer greater than 1;

[0047] If multiple usage records corresponding to one event among M usage records contain logical errors, delete at least one usage record with a logical error among the multiple usage records corresponding to one event; or...

[0048] If, in a pair of M usage records, at least a portion of the usage record for one of the two related events contains a logical error, then delete the usage record containing the logical error in at least a portion of the usage record for one of the events and the usage record containing the logical error in the usage record for the other event.

[0049] Fifthly, this application provides a data deduplication apparatus, which includes a processing unit for performing any one of the methods in the first to fourth aspects.

[0050] Sixthly, this application provides an electronic device including a unit for performing any one of the methods of the first to fourth aspects. The device may be a terminal device or a chip within a terminal device. The device may include an input unit and a processing unit.

[0051] When the device is a terminal device, the processing unit may be a processor, and the input unit may be a communication interface; the terminal device may also include a memory for storing computer program code, which, when the processor executes the computer program code stored in the memory, causes the terminal device to execute any one of the data deduplication methods in the first to fourth aspects.

[0052] When the device is a chip within a terminal device, the processing unit can be an internal processing unit of the chip, and the input unit can be an output interface, pin, or circuit, etc.; the chip can also include a memory, which can be an internal memory of the chip (e.g., registers, cache, etc.) or an external memory (e.g., read-only memory, random access memory, etc.); the memory is used to store computer program code, and when the processor executes the computer program code stored in the memory, the chip performs any of the data deduplication methods in the first to fourth aspects.

[0053] In one possible implementation, the memory is used to store computer program code; the processor executes the computer program code stored in the memory, and when the computer program code stored in the memory is executed, the processor is used to perform any one of the data deduplication methods in the first to fourth aspects.

[0054] In a seventh aspect, this application provides a computer-readable storage medium storing computer program code that, when executed by a data deduplication device, causes the data deduplication device to perform any one of the data deduplication methods in the first to fourth aspects.

[0055] Eighthly, this application provides a computer program product comprising: computer program code, which, when run by a data deduplication device, causes the data deduplication device to perform any one of the data deduplication methods in the first to fourth aspects.

[0056] It is understood that the beneficial effects of aspects two through eight above can be found in the relevant descriptions in aspect one above, and will not be repeated here.

[0057] It should be understood that the descriptions of technical features, technical solutions, beneficial effects, or similar language in this application do not imply that all features and advantages can be achieved in any single embodiment. Rather, it is understood that the description of a feature or beneficial effect means that a specific technical feature, technical solution, or beneficial effect is included in at least one embodiment. Therefore, the descriptions of technical features, technical solutions, or beneficial effects in this specification do not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions, and beneficial effects described in this embodiment can be combined in any suitable manner. Those skilled in the art will understand that embodiments can be implemented without one or more specific technical features, technical solutions, or beneficial effects of a particular embodiment. In other embodiments, additional technical features and beneficial effects may be identified in specific embodiments that do not embody all embodiments. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of the hardware system of the electronic device 100 provided in the embodiments of this application.

[0059] Figure 2 This is a software structure block diagram of the electronic device 100 according to an embodiment of this application.

[0060] Figure 3 This is a schematic diagram of a data deduplication method provided in an embodiment of this application.

[0061] Figure 4 This is a schematic diagram of data to be deduplicated provided in an embodiment of this application.

[0062] Figure 5 The embodiments provided in this application are for the above-mentioned Figure 4 This is a diagram illustrating the result obtained after deduplication of the data to be deduplicated.

[0063] Figure 6 The embodiments provided in this application are for the above-mentioned Figure 5 This is a diagram showing the result obtained after performing a second round of deduplication on the original deduplication result.

[0064] Figure 7 This is a schematic diagram of a data deduplication method provided in an embodiment of this application.

[0065] Figure 8 This is a schematic diagram of a deduplication result provided in an embodiment of this application.

[0066] Figure 9 This is a schematic diagram of a deduplication result provided in an embodiment of this application.

[0067] Figure 10 This is a schematic diagram of a deduplication result provided in an embodiment of this application.

[0068] Figure 11 This is a schematic diagram of a deduplication result provided in an embodiment of this application.

[0069] Figure 12 This is a schematic diagram of a data deduplication device provided in an embodiment of this application. Detailed Implementation

[0070] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0071] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. "And / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone; A and B at the same time; and B alone.

[0072] In the description of the embodiments of this application, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first," "second," "third," etc., may explicitly or implicitly include one or more features. In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0073] In the description of the embodiments of this application, the term "comprising" indicates the presence of the described feature, integral, step, operation, element, and / or component, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0074] The technical solution provided in this application will now be described in detail.

[0075] The data deduplication method provided in this application can be applied to electronic devices. The hardware and software structures of the electronic devices will be described in detail below with reference to the accompanying drawings.

[0076] Figure 1 This is a schematic diagram of the hardware system of the electronic device 100 provided in the embodiments of this application.

[0077] The type of electronic device 100 is not specifically limited and can be selected according to the actual scenario. For example, electronic device 100 can be a mobile phone, smart screen, tablet computer, wearable electronic device, in-vehicle electronic device, augmented reality (AR) device, virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), projector, in-vehicle equipment, etc. This application embodiment does not impose any restrictions on the specific type of electronic device 100.

[0078] Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0079] It should be noted that, Figure 1 The structure shown does not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include... Figure 1 The components shown may include more or fewer components, or the electronic device 100 may include... Figure 1 The components shown may be a combination of certain components, or the electronic device 100 may include... Figure 1 Sub-components of some of the components shown. Figure 1 The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0080] Processor 110 may include one or more processing units. For example, processor 110 may include at least one of the following processing units: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and neural network processing unit (NPU). These different processing units may be independent devices or integrated devices.

[0081] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0082] The processor 110 may also include a memory for storing instructions and data. For example, the processor 110 may store instructions for executing the data deduplication method provided in the embodiments of this application. For example, the processor 110 may store data to be deduplicated or the deduplication result obtained by executing the data deduplication method provided in the embodiments of this application. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system. In some embodiments, the processor 110 may include one or more interfaces. For example, processor 110 may include at least one of the following interfaces: an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and a USB interface.

[0083] Figure 1 The connection relationships between the modules shown are merely illustrative and do not constitute a limitation on the connection relationships between the modules of the electronic device 100. Optionally, the modules of the electronic device 100 may also adopt a combination of various connection methods described in the above embodiments.

[0084] The hardware system of electronic device 100 has been described in detail above. The software system of electronic device 100 is described below. The software system can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment takes a layered architecture as an example to exemplarily describe the software system of electronic device 100.

[0085] For example, Figure 2 This is a schematic diagram of the software system of the electronic device 100 provided in an embodiment of this application. See also... Figure 2The software system adopts a layered architecture. This layered architecture divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom: the application layer 210, the application framework layer 220, the Android Runtime and core library layer 230, the hardware abstraction layer (HAL) 240, and the kernel layer 250.

[0086] Application layer 210 may include a series of application packages. For example, application packages may include applications such as voice assistant, video editing, camera, gallery, chat, calling, map, navigation, calendar, Bluetooth, music, and video.

[0087] The applications mentioned above can include more specific functional modules. For example, a gallery can include business modules and notification modules. A camera can include a photo-taking module.

[0088] The aforementioned applications can be used to generate application data; for example, the gallery can be used to generate photo data.

[0089] The application framework layer 220 provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer 220 includes some predefined functions.

[0090] like Figure 2 As shown, the application framework layer 220 may include a window manager, notification manager, activity manager, input manager, view system, content provider, resource manager, etc.

[0091] The window manager provides a window management service (WMS), which can be used for window management, window animation management, surface management, and as a relay station for the input system.

[0092] Content providers store and retrieve data, making that data accessible to applications. This data can include videos, images, audio, phone calls made and received, browsing history and bookmarks, phone books, etc.

[0093] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text message notification icon can include views for displaying text and views for displaying images. For example, a display interface can be, but is not limited to, […]. Figure 4 or Figure 6 The user interface shown.

[0094] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and so on.

[0095] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0096] The Activity Manager Service (AMS) can be used to start, switch, and schedule system components (such as activities, services, content providers, and broadcast receivers), as well as manage and schedule application processes.

[0097] The input manager can provide an input management service (IMS), which can be used to manage system inputs, such as touchscreen input, keypad input, and sensor input. IMS retrieves events from input device nodes and, through interaction with the WMS, distributes these events to the appropriate windows.

[0098] The Android Runtime consists of core libraries and a virtual machine. The Android Runtime is responsible for scheduling and managing the Android system.

[0099] The core library consists of two parts: one part is the functionalities that need to be called by programming languages ​​(e.g., Java), and the other part is the Android core library.

[0100] Application layer 210 and application framework layer 220 run in a virtual machine. The virtual machine executes the programming files (e.g., Java files) of application layer 210 and application framework layer 220 as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0101] The core library layer 230 can include multiple functional modules. For example: surface manager, media framework, libc, SQLite, OpenGL ES, Webkit, etc.

[0102] The Surface Manager is used to manage the display subsystem and provides the fusion of two-dimensional (2D) and three-dimensional (3D) layers for multiple applications.

[0103] The media framework supports playback and recording of various commonly used audio and video formats, as well as still image files.

[0104] libc (the C library) is the standard library for the C programming language. libc is one of the lowest-level libraries in the system, implemented through Linux system calls. For example, libc can be used to connect or disconnect camera services, set camera shooting parameters, start and stop previewing, and take photos.

[0105] The Hardware Abstraction Layer (HAL) is an interface layer located between the operating system kernel and upper-level software, designed to abstract hardware. The HAL is an abstract interface for device kernel drivers, providing application programming interfaces (APIs) that allow access to the underlying device to higher-level Java API frameworks. The HAL contains multiple library modules, such as the Camera HAL (e.g., aperture, TOF sensor, lens, or focus motor), the Vendor repository, the display, Bluetooth, and audio. Each library module implements an interface for a specific type of hardware component. For example, the Camera HAL provides the camera firmware (FWK) with an interface to access hardware components like the camera lens. The Vendor repository provides the media firmware (FWK) with an interface to access hardware components like the encoder. When the system framework layer API requires access to the portable device's hardware, the Android operating system loads the library module for that hardware component.

[0106] The kernel layer 250 is the foundation of the Android operating system; all the final functions of the Android operating system are implemented through the kernel layer. The kernel layer can contain display drivers, camera drivers, audio drivers, and sensor drivers.

[0107] It should be noted that the application provides Figure 2 The illustrated software architecture diagram of the electronic device is merely an example and does not limit the specific module divisions within different layers of the Android operating system. For details, please refer to the descriptions of the Android operating system software architecture in conventional technologies. Furthermore, the data deduplication method provided in this application can also be implemented on other operating systems (e.g., iOS or HarmonyOS), which will not be listed here.

[0108] Next, combined Figures 3 to 7 The data deduplication method provided in the embodiments of this application will be described in detail.

[0109] Figure 3This is a schematic diagram illustrating a data deduplication method provided in an embodiment of this application. The data deduplication method provided in this embodiment can be executed by an electronic device. It is understood that the electronic device can be implemented as software, or a combination of software and hardware. For example, the electronic device in this embodiment can be, but is not limited to, […]. Figure 1 The electronic device 100 is shown. (For example...) Figure 3 As shown, the data deduplication method provided in this application includes steps S310 to S340. The steps S310 to S340 will be described in detail below.

[0110] S310, the electronic device acquires user service data 1, wherein the user service data 1 includes multiple usage records 1 corresponding to multiple applications, and each usage record 1 includes at least the application identifier, the event corresponding to the application, and the time information of the application usage.

[0111] There is a one-to-one correspondence between multiple applications and multiple usage records 1, and each usage record 1 records information about the user's use of the corresponding application.

[0112] Each usage record 1 includes at least the application's identifier, the event corresponding to the application, and the time information when the application was used.

[0113] An application identifier is used to identify an application, and there are no specific limitations on the identifier. For example, an application identifier can be the application's package name or the application's name.

[0114] The events associated with an application are used to indicate the purpose of that application. For example, for a chat application, the event associated with that application is "chat". For a weather application, the event associated with that application is "check the weather".

[0115] The application's time information is used to indicate when the event corresponding to the application occurred. For example, the time when the event occurs can be a time period or a moment, without specific limitations. In one example, the application's time information may include the time the application was used (i.e., the time when the event corresponding to the application occurred).

[0116] It should be noted that each usage record 1 described above may also include other information, without specific limitations. For example, other information may be the user's identity document (id), where the user's id refers to the account used to log in to the application.

[0117] For example, the user service data 1 in step S310 above includes multiple usage records 1 corresponding to multiple applications, which can be as follows: Figure 4 The table shown contains multiple usage records 1. See also... Figure 3Each usage record for each application includes the user's ID, the application's package name, the event associated with the application, and the time the application was used. For example, using... Figure 3 Taking the first row of the table shown as an example, the usage record 1 includes the user's ID as "123", the application's package name as "package name A", the event corresponding to the application as "chat", and the time of application use as "2023-06-02 21:41:30.560".

[0118] In this embodiment of the application, the method for the electronic device to obtain user service data 1 is not specifically limited. For example, the electronic device obtaining user service data 1 may include the following steps: in response to the electronic device running each of multiple applications, the electronic device records each usage record 1 corresponding to each application in the storage space allocated to each application by the electronic device; the electronic device obtains user service data 1, including multiple usage records 1 corresponding to multiple applications, by accessing multiple storage spaces allocated to multiple applications.

[0119] S320, the electronic device obtains multiple time buckets based on multiple usage records 1, wherein the multiple time buckets correspond to multiple consecutive time periods, each time bucket includes at least one usage record 1 from the multiple usage records 1, and the time indicated by the usage time information included in at least one usage record 1 is located within the time period corresponding to each time bucket.

[0120] A series of consecutive time periods refers to a series of time periods in which there is no time interval between any two adjacent time periods. For example, if time period 1 is "20:00:00 to 20:05:59", time period 2 is "20:06:00 to 20:07:59", and time period 3 is "20:08:00 to 20:09:59", then time period 1, time period 2, and time period 3 are three consecutive time periods, while time period 1 and time period 3 are two non-consecutive time periods.

[0121] The duration of each time segment within a series of consecutive time periods is not specifically limited and can be set according to the actual situation. For example, each time segment can be, but is not limited to, 5 minutes, 8 minutes, or 10 minutes.

[0122] For example, continuing with the multiple usage records 1 corresponding to the multiple applications in step S320 above, Figure 4 Taking the table shown as an example, with each time period being 10 minutes, the multiple usage records corresponding to these multiple applications can be divided into the following categories: Figure 4The six time buckets shown are as follows: Time bucket 1 corresponds to the time period "21:41:00-21:50:00", Time bucket 2 corresponds to the time period "21:51:00-22:00:00", Time bucket 3 corresponds to the time period "22:01:00-22:10:00", Time bucket 4 corresponds to the time period "22:11:00-22:20:00", Time bucket 5 corresponds to the time period "22:21:00-22:30:00", and Time bucket 6 corresponds to the time period "22:31:00-22:40:00".

[0123] For example, taking time bin 1 as an example, time bin 1 includes Figure 4 The table shown contains four usage records in the first four rows. The usage time information in each usage record indicates a time within the time period corresponding to time bucket 1.

[0124] In this embodiment of the application, the execution of the above-described S320 steps by the electronic device is not specifically limited. The following is an example describing the process of the electronic device executing the above-described S320 steps. It should be understood that the process described below is only illustrative and does not constitute any limitation on the process by which the electronic device obtains multiple time buckets based on multiple usage records 1.

[0125] In one example, an electronic device obtains multiple time buckets based on multiple usage records 1, which may include the following steps: The electronic device obtains a start time, an end time, and a preset time interval 1, wherein the start time is the usage time of the earliest used application among the multiple applications corresponding to the multiple usage records 1, and the end time is the usage time of the latest used application among the multiple applications corresponding to the multiple usage records 1; The electronic device obtains multiple consecutive time periods based on the start time, end time, and preset time interval 1, wherein the length of the preset time interval 1 is the same as the length of the time period corresponding to each time bucket; The electronic device obtains multiple time buckets based on the multiple consecutive time periods and the usage time of the application recorded in each of the multiple usage records 1.

[0126] S330, the electronic device performs deduplication processing on at least one usage record 1 included in each time bucket according to data deduplication strategy 0, and obtains an initial deduplication result, wherein the initial deduplication result includes multiple deduplicated time buckets corresponding to multiple time buckets.

[0127] There is a one-to-one correspondence between multiple time buckets and multiple deduplicated time buckets. Each deduplicated time bucket is the result obtained by the electronic device performing deduplication on the corresponding time bucket according to data deduplication strategy 0.

[0128] Data deduplication strategy 0 includes the following: if the identifiers of multiple applications corresponding to multiple usage records 1 in each time bucket are the same, and the events corresponding to multiple usage records 1 are the same, only the first usage record 1 in each time bucket is retained. The time of the application recorded in the first usage record 1 is earlier than the time of the application recorded in the other usage records 1 in each time bucket.

[0129] For example, continue using the multiple time buckets from step S330 above. Figure 4 Taking the six time-divided data buckets shown as an example, the electronic device follows the data deduplication strategy 0 described above to... Figure 4 After deduplication of each of the six time buckets shown, the following can be obtained: Figure 5 The six deduplicated time buckets are shown.

[0130] Specifically Figure 4 The time buckets shown are i and Figure 6 The deduplicated time bins shown correspond to bins i, where i = 1, 2, 3, 4, 5, 6. For example, using... Figure 4 The time buckets shown are 1 and 2 Figure 6 Taking the deduplicated time bucket 1 as an example, time bucket 1 includes 4 usage records 1. Among these 4 usage records 1, the second and third usage records 1 have the same application package name (i.e., package name B), and the second and third usage records 1 in time bucket 1 correspond to the same event (i.e., community). Therefore, the deduplicated time bucket 1 should include the following: Figure 5 The three usage records shown are 1, and these three usage records are... Figure 4 The first usage record 1, the second usage record 1, and the fourth usage record 1 are shown in time bucket 1.

[0131] In step S330 above, the electronic device performs deduplication processing on each time bucket obtained in step S320 according to data deduplication strategy 0. That is, when each time bucket includes multiple duplicate usage records, only the first usage record in each time bucket is retained. In this way, the problem of missing deduplication caused by chained data processing (the same event occurs continuously and will be interrupted by the insertion of other events) can be solved.

[0132] S340, the electronic device performs deduplication processing on the usage records 1 included in multiple deduplicated time buckets according to the data deduplication strategy 1, and obtains the target deduplication result 1.

[0133] Data deduplication strategy 1 includes the following: if the application identifiers corresponding to the two usage records 1 in two adjacent deduplicated time buckets are the same, and the events corresponding to the two usage records 1 are the same, and if the time interval between the two usage applications corresponding to the two usage records 1 is less than a preset time interval 2, then delete the usage record corresponding to the latest usage application time among the two usage application times corresponding to the two usage records 1.

[0134] Two adjacent deduplicated time buckets correspond to two consecutive time periods. For example, using... Figure 5 Taking the multiple deduplicated time bins shown as an example, deduplicated time bin 1 and deduplicated time bin 2 are two adjacent deduplicated time bins. Deduplicated time bin 2 and deduplicated time bin 3 are two adjacent deduplicated time bins. Deduplicated time bin 2 and deduplicated time bin 4 are not two adjacent deduplicated time bins.

[0135] The preset time interval 2 is shorter than the time interval corresponding to each deduplicated time bucket. The preset time interval 2 can be set according to the actual situation. For example, the preset time interval 2 can be, but is not limited to, 1 minute, 2 minutes or 3 minutes.

[0136] It should be noted that the time length corresponding to each deduplicated time bucket, the time length corresponding to each time bucket, and the time length of the preset time interval 1 mentioned above are all the same.

[0137] In one example, one of the two usage records 1 corresponding to the two adjacent deduplicated time buckets mentioned above is the first usage record 1 in one of the two adjacent deduplicated time buckets, and the other usage record 1 is the last usage record 1 in the other deduplicated time bucket, and the time period corresponding to the other deduplicated time bucket is the time period before the time period corresponding to the first deduplicated time bucket.

[0138] For example, the following uses the multiple deduplicated time bins in step S340 above as an example. Figure 5 Taking the six deduplicated time buckets shown as an example, this describes the target deduplication result 1 obtained by the electronic device after performing the above S340.

[0139] See Figure 5The rectangle 601 shown includes usage record 1A recorded in deduplicated time bucket 2 and usage record 1B recorded in deduplicated time bucket 3. It can be seen that the application package names (i.e., package name A) corresponding to these two usage records 1 (i.e., usage record 1A and usage record 1B) are the same, and the events (i.e., chats) corresponding to these two usage records 1 are also the same. Based on this, when the interval between the times of the two applications corresponding to these two usage records 1 is less than 5 minutes (i.e., an example of the preset time interval 2), based on the data deduplication strategy 1 described above, usage record 1B recorded in deduplicated time bucket 3 needs to be deleted.

[0140] See Figure 5 The rectangle 602 shown includes usage record 1C recorded in deduplicated time bucket 4 and usage record 1D recorded in deduplicated time bucket 5. It can be seen that the application package names (i.e., package name H) corresponding to these two usage records 1 (i.e., usage record 1C and usage record 1D) are the same, and the events (i.e., local life) corresponding to these two usage records 1 are also the same. Therefore, based on the data deduplication strategy 1 described above, usage record 1D recorded in deduplicated time bucket 5 needs to be deleted. Based on this, since the time interval between the two usage applications corresponding to these two usage records 1 exceeds 5 minutes (i.e., an example of the preset time interval 2), based on the data deduplication strategy 1 described above, it is not necessary to delete usage record 1D recorded in deduplicated time bucket 3.

[0141] Based on this, the electronic device is processed according to data deduplication strategy 1. Figure 5 The usage record 1 in the 6 deduplicated time buckets shown is used for deduplication to obtain the target deduplication result 1, as shown. Figure 6 The contents recorded in the table shown.

[0142] In step S340 above, the electronic device performs another short-term (i.e., preset time interval 2) deduplication of the initial deduplication result (i.e., multiple deduplicated time buckets) obtained by performing step 330 above, and removes some duplicate usage records 1 that happen to be located at the edge of two adjacent deduplicated time buckets. In this way, data noise can be minimized.

[0143] It should be understood that the above Figure 3 The data deduplication method shown is for illustrative purposes only and does not constitute any limitation on the data deduplication method provided in this application.

[0144] Figure 7This is a schematic diagram illustrating a data deduplication method provided in an embodiment of this application. The data deduplication method provided in this embodiment can be executed by an electronic device. It is understood that the electronic device can be implemented as software, or a combination of software and hardware. For example, the electronic device in this embodiment can be, but is not limited to, […]. Figure 1 The electronic device 100 is shown. (For example...) Figure 7 As shown, the data deduplication method provided in this application embodiment includes steps S710 and S720. The steps S710 and S720 will be described in detail below.

[0145] S710, the electronic device acquires the data to be deduplicated, wherein the data to be deduplicated includes M usage records, the M usage records include L event usage records, the L events include S1 first events, all usage records of each first event are located in at least one time period, and each time period of at least some time periods within at least one time period corresponds to at least two usage records of the first event, M is an integer greater than 1, and L and S1 are positive integers.

[0146] Each of the M usage records corresponds to one event, meaning each usage record is used to record the corresponding event. There are no specific limitations on the type of event recorded by each usage record. For example, the event recorded by each usage record can be, but is not limited to, any of the following: chat, messaging, community, charging, power outage, connecting via Bluetooth, disconnecting via Bluetooth, watching video, listening to music, payment, or office work, etc.

[0147] In this embodiment of the application, as described above, each usage record is used to record the corresponding event. The content of each usage record among the M usage records is not specifically limited and can be determined according to the actual situation.

[0148] In one example, each usage record may include the usage time of the event (i.e., when the event occurred) and the event itself.

[0149] In another example, each usage record may include the user identifier that triggered the event (e.g., user's id), the usage time of the event (i.e., the time when the event occurred), and the event.

[0150] In another example, each usage record may include a user identifier (e.g., a user account) that triggered the event, an application identifier associated with the event, the usage time of the event (i.e., the time the event occurred), and the event itself. The application identifier associated with the event indicates the application used to generate the corresponding event, where the application identifier associated with the event can be the application name (also known as the application package name). For example, if the event is chat, the application identifier associated with the event could be the application identifier of the chat application. Similarly, if the event is listening to music, the application identifier associated with the event could be the application identifier of the music application.

[0151] Optionally, the usage log for each of the above examples may also include the event result. For example, taking the event as charging, the event result could be the amount of electricity.

[0152] The data to be deduplicated includes M usage records, and the total number of events of all types included in these M usage records is L, where M is an integer greater than 1 and L is a positive integer. There is no specific limit to the number of events (i.e., L) included in the M usage records; for example, L can take values ​​such as, but is not limited to, 1, 2, 5, 7, or 10.

[0153] For example, taking the above M (i.e., M=4) usage records as... Figure 4 The example shown is of the four usage records included in time bucket 1. Figure 4 As shown, the event of the first usage record is "Chat", the events of the second and third usage records are both "Community", and the event of the fourth usage record is "Note Intent". Therefore, the four usage records include three (i.e., L=3) events (i.e., "Chat", "Community" and "Note Intent").

[0154] In this embodiment of the application, the M usage records include L events, including S1 first events, wherein all usage records of each first event are located in at least one time period, and each time period of at least a portion of the time period within the at least one time period corresponds to at least two usage records of the first event, and S1 is a positive integer.

[0155] All usage records for each first event are located in at least one time period, that is, all usage records for each first event can be located in one or more time periods, and this is not specifically limited in the embodiments of this application.

[0156] Each of the at least partial time periods within at least one time period corresponds to at least two usage records of the first event. In one example, when at least one time period is a single time period, at least partial time periods within at least one time period refers to that single time period. In another example, when at least one time period is multiple time periods, at least partial time periods within at least one time period refers to one or more of those multiple time periods.

[0157] Optionally, each time period corresponds to a time bucket, meaning the time period of each time bucket is the corresponding time period. When there are multiple time periods, each of these time periods corresponds one-to-one with a different time bucket.

[0158] M is an integer greater than 1, L and S1 are positive integers, and S1 is a positive integer less than or equal to L. The values ​​of M, L, and S1 can be set according to the actual situation and are not specifically limited. The following example describes the M usage records, L events, and S1 first events mentioned above.

[0159] In one example, when S1 is a positive integer equal to 1, the M usage records include only one first event among the L events, all usage records of the first event are located in at least one time period, and each time period of at least a portion of the time period within the at least one time period corresponds to at least two usage records of the first event.

[0160] For example, with M (i.e., M=4) usage records as... Figure 4 Taking the four usage records in time bucket 1 as an example, time bucket 1 corresponds to a time period (i.e., 21:41:00-21:50:00). Within the time period corresponding to time bucket 1, the four usage records include L events: "Chat", "Community", and "Note Intent" (i.e., L=3). Within the time period corresponding to time bucket 1, the four usage records include two usage records for "Community". Since the number of usage records for other events (i.e., "Chat" or "Note Intent") within the time period corresponding to time bucket 1 is only one, therefore, when M usage records are... Figure 4 When the time bucket 1 shown includes 4 usage records, only 1 of these M usage records has a first event of "community" (i.e., S1 = 1).

[0161] In another example, when S1 is a positive integer greater than 1, the M usage records include L events, which include two or more first events. All usage records for each first event are located in at least one time period, and each time period within at least a portion of the time periods corresponds to at least two usage records for a single first event. That is, when a time period includes two usage records for an event, that event can be called a first event.

[0162] For example, with M (i.e., M=11) usage records including Figure 4 Taking the 4 usage records in time bucket 1 and the 7 usage records in time bucket 2 as examples, based on the description of the first event mentioned above, it can be seen that the L events included in these 11 usage records are "Chat", "Community", "Note Intent" and "Office Software". Since the time period corresponding to time bucket 1 includes 2 usage records of the Community, and the time period corresponding to time bucket 2 includes 2 usage records of the Community, the Community is a first event; since the time period corresponding to time bucket 2 includes 2 usage records of the Office Software, the Office Software is a first event; since the time periods corresponding to time bucket 1 and time bucket 2 each include only 1 usage record of Chat, Chat is not a first event; since the time period corresponding to time bucket 2 includes only 1 usage record of Browser, Browser is not a first event; in summary, these 11 usage records include 2 first events (i.e., S1 = 2), namely "Community" and "Office Software".

[0163] It should be noted that the M usage records, L events, and S1 first events mentioned above are for illustrative purposes only and do not constitute any limitation on the M usage records, L events, and S1 first events, nor do they limit the specific values ​​of M, L, and S1.

[0164] S720, for each first event, the electronic device deletes the remaining usage records other than the first usage record from at least two usage records corresponding to each time period of the first event in at least a portion of the time period to obtain a deduplication result, wherein the first usage record is the usage record with the earliest usage time among the at least two usage records corresponding to each time period of the first event in at least a portion of the time period, and the deduplication result includes the usage records other than the remaining usage records corresponding to each time period of the first event from the M usage records.

[0165] In step S720 above, for each first event, if the electronic device has multiple usage records for each first event within each time period, it retains only the usage record of each first event that first appears in each time period (i.e., the first usage record).

[0166] The first usage record is the earliest usage record among at least two usage records of the first event corresponding to each time period of at least some time periods.

[0167] For example, taking the above M (i.e., M equals 4) usage records as... Figure 4 Taking the four usage records included in time bucket 1 as an example, when the first event is "community", the first usage record is the second usage record included in time bucket 2, namely: "123, package name B, community, 2023-06-02 21:43:31.693"; the remaining usage records other than the first usage record are the third usage records included in time bucket 1, namely: "123, package name B, community, 2023-06-02 21:48:24.231".

[0168] Below, we will use an example to describe M usage records and the deduplication results.

[0169] For example, taking the above M (i.e., M=4) usage records as... Figure 4 Taking the four usage records included in time bin 1 as an example, after the electronic device performs the above S720 step, the deduplication result can be found in [reference needed]. Figure 5 The deduplicated time bucket 1 shown includes 3 events (i.e., "Chat", "Community" and "Note Intent") and 3 usage records corresponding to them.

[0170] For example, with M (i.e., M equals 11) usage records including Figure 4Taking the example of time bin 1 containing 4 usage records and time bin 2 containing 7 usage records, after the electronic device performs the above S720 step, the deduplication result obtained includes... Figure 5 The deduplicated time bucket 1 includes 3 events (i.e., "Chat", "Community", and "Note Intent") and 5 usage records corresponding to them, and the deduplicated time bucket 2 includes 5 events (i.e., "Office Software", "Note Intent", "Browser", "Community", and "Chat") and 5 usage records corresponding to them.

[0171] For example, with M usage records including Figure 4 Taking the usage records included in time bins 1 to 6 as an example, after the electronic device performs the above S720 step, the deduplication result obtained includes... Figure 5 The deduplication time bins 1 to 6 shown contain all usage records.

[0172] For example, an example of the deduplication method described in steps S710 and S720 above can be found in the above text. Figure 3 The deduplication method provided in steps S310 to S330 of the provided method is as follows: the user business data 1 in step S310 is an example of the data to be deduplicated; the multiple usage records 1 in step S310 is an example of the M usage records; the multiple usage buckets in step S320 is a specific example of at least one time period; the data deduplication strategy 0 in step S330 is an example of the deduplication strategy described in step S720; and the initial deduplication result in step S330 is an example of the deduplication result. For details not elaborated here, please refer to the relevant descriptions in steps S310 to S330 above.

[0173] The electronic device, based on the aforementioned time-based deduplication method, performs deduplication processing on the M usage records included in the data to be deduplicated. This avoids the problem of missed deduplication that occurs when performing deduplication on chained data (i.e., data interrupted by other events when the same event occurs consecutively) based on time intervals in traditional technologies. In summary, the data deduplication method provided in this application can improve the accuracy of data deduplication results, thereby improving the quality of the deduplicated data.

[0174] In practical applications, certain events exhibit clear correlations in some scenarios. For example, there are events such as an electronic device disconnecting and an electronic device connecting. Similarly, there are events such as an electronic device charging and an electronic device unplugging. Therefore, if the data contains only a single event (e.g., charging) while corresponding events related to that single event (e.g., unplugging) are missing, or if the data contains consecutive single events (e.g., charging), there may be data omissions or mis-collections, requiring deduplication of the erroneous data. In other scenarios, some events recorded in the data may have obvious logical errors with subsequent events (e.g., the electronic device's battery level increases after being unplugged), necessitating deduplication of the data with logical errors. Therefore, in this embodiment, after the electronic device executes the time-based data deduplication method described in steps S710 and S720 to obtain the deduplication result, the electronic device can further perform deduplication processing on the obtained result based on event correlation, i.e., the deduplication methods shown in Implementation Methods 1 to Implementation Methods 5 below. Implementation Methods 1 to Implementation Methods 5 will be described below.

[0175] Implementation method 1:

[0176] In implementation method one, the L events in the deduplication result include S2 second events, wherein the deduplication result specifically includes multiple usage records for each second event, and S2 is a positive integer; and, for each first event, after deleting the remaining usage records other than the first usage record from at least two usage records corresponding to each time period of at least some time periods to obtain the deduplication result, the electronic device further performs the following steps: if the interval between the usage times corresponding to at least two usage records in the multiple usage records of each second event does not exceed a first threshold, delete the usage record with the latest usage time from the at least two usage records of each second event.

[0177] The deduplication result is obtained by performing the above deduplication process on the M usage records of the data to be deduplicated. In other words, the L events in the deduplication result are the same as the L events in the M usage records, but the number of usage records corresponding to the L events in the deduplication result is different from the number of usage records in the M usage records of the data to be deduplicated.

[0178] The multiple usage records of each second event in S2 events are located in multiple time periods, and there are no specific restrictions on the mapping relationship between the multiple usage records and the multiple time periods.

[0179] For example, multiple usage records correspond one-to-one with multiple time periods. That is, each time period corresponds to one usage record among the multiple usage records, and each usage record is a usage record for the corresponding time period, meaning that the usage time for each usage record falls within the corresponding time period. Alternatively, multiple usage records can correspond to multiple time periods, with each time period corresponding to at least one usage record among the multiple usage records, and at least one time period corresponding to a subset of the multiple usage records.

[0180] The L events in the deduplication result include S2 second events, each of which may be the same as or different from each of the S1 first events mentioned above.

[0181] The value of the first threshold is not specifically limited and can be set according to actual needs. For example, the first threshold can be, but is not limited to, 2 seconds or 3 seconds.

[0182] In this application, "exceeding" means "greater than", and "not exceeding" means "less than" or "equal to". The meanings of "exceeding" or "not exceeding" in the following text are the same as those here, and will not be repeated here.

[0183] For example, an example of the deduplication method described in Implementation Method 1 above can be found in the text above. Figure 3 The data deduplication strategy 1 described in step S340 of the provided method, wherein the usage record 1 included in the multiple deduplicated time buckets in step S340 is an example of the deduplication result in implementation method one, and the preset time interval 2 in step S340 is an example of the first threshold in implementation method one. For details not elaborated here, please refer to the relevant description in step S340 above.

[0184] In the first implementation described above, after performing data deduplication on each of the S1 first events among the M usage records to be deduplicated, the resulting deduplication result includes multiple usage records for each second event located in multiple time periods. Based on this, a second short-term repetitive behavior data deduplication can be performed on each second event. That is, among at least two usage records with a small interval between their usage times, the usage record with the latest usage time is deleted from at least two usage records of each second event. This method can remove some events of the same type that happen to be distributed at the edge of adjacent time periods, minimizing the data noise in the deduplication result obtained after the second deduplication, thus improving the accuracy of the data deduplication result and improving the quality of the deduplicated data.

[0185] Implementation Method Two:

[0186] In implementation method two, the L events in the deduplication result include S3 third events, where each third event corresponds to an associated event, multiple usage records of each third event are consecutive, and multiple usage records of each third event are adjacent to the usage records of the corresponding associated events, and S3 is a positive integer; and, for each first event, after deleting the remaining usage records other than the first usage record from at least two usage records of the first event corresponding to each time period in at least some time periods to obtain the deduplication result, the electronic device further performs the following steps: if the duration between the usage time of the usage record of the associated event corresponding to each third event and the usage time corresponding to the multiple usage records of each third event does not exceed a first preset duration, delete the usage records of the multiple usage records of each third event except for the usage record with the earliest usage time.

[0187] Multiple usage records for each third event are located in multiple time periods, and the mapping relationship between these multiple usage records and multiple time periods is not specifically limited. For example, there is a one-to-one correspondence between the multiple usage records and multiple time periods for each third event.

[0188] "Multiple usage records consecutive for each third event" means that the multiple usage records corresponding to each third event are consecutive records among all usage records included in the deduplication result.

[0189] For example, the usage records of the L events included in the above deduplication result can be as follows: Figure 8 Taking the three usage records in the table shown in (1) as an example, Figure 8 The two usage records of the “power-off” event shown in (1) are consecutive.

[0190] The third event corresponds to related events. These two related events can be two events that have a relative connection or obvious correlation; there are no specific limitations on the two related events. For example, the two related events could include a "plugging in" event and a "plugging out" event. Similarly, the two related events could include a "device connected" event and a "device disconnected" event.

[0191] The deduplication result includes the usage records of the associated events corresponding to each third event and the adjacent consecutive usage records of each third event. In other words, the usage records of the associated events corresponding to each third event, as well as the consecutive usage records of each third event, are consecutive usage records. The usage time of the usage record of the associated events corresponding to each third event can be earlier or later than the usage time of the earliest usage record among the consecutive usage records of each third event.

[0192] For example, the usage records of the L events included in the above deduplication result can be as follows: Figure 8 Taking the three usage records in the table shown in (1) as an example, Figure 8 In (1), the two usage records for the plug-in event and the unplug event are adjacent.

[0193] For example, the usage records of the L events included in the above deduplication result can be as follows: Figure 8 The table shown in (1) contains three usage records, which correspond to two (i.e., L=2) events (i.e., "plugging in" and "plugging out"). Based on this, the electronic device performs deduplication processing on these three usage records according to the above implementation method two. That is, it is necessary to delete the usage records of the two consecutive power-outs except for the first power-out (i.e., the usage record of the second power-out). The result after deletion is as follows: Figure 8 (2) shows two usage records recorded in the table.

[0194] Each of the three third events mentioned above, each of the two second events mentioned above, and each of the three first events mentioned above can be the same or different. In this embodiment of the application, no specific limitation is made.

[0195] For example, each of the S3 third events included in the L events in the above deduplication result, each of the S2 second events included in the L events in the above deduplication result, and each of the S1 first events included in the L events in the above deduplication result are the same event.

[0196] For example, each of the S3 third events included in the L events in the above deduplication result, each of the S2 second events included in the L events in the above deduplication result, and each of the S1 first events included in the L events in the above deduplication result are three different events.

[0197] In the above technical solution, after performing data deduplication based on time features (i.e., the time period mentioned above) on each of the S1 first events in the M usage records to be deduplicated, the resulting deduplication result includes each third event corresponding to a related event. Subsequently, deduplication is performed based on the correlation between events. That is, if the usage time of the usage record of the related event corresponding to each third event is short, the usage records of each third event are deleted except for the earliest usage record. This method performs data deduplication from the two dimensions of time features and correlation between events. This method has high versatility and robustness, which can improve the accuracy of data deduplication results and thus improve the quality of the deduplicated data.

[0198] Implementation method three:

[0199] In implementation method three, the L events in the deduplication result include S4 fourth events and there is no associated event corresponding to each fourth event, where S4 is a positive integer; and, for each first event, after deleting the remaining usage records other than the first usage record from at least two usage records corresponding to each time period of at least some time periods to obtain the deduplication result, the electronic device further performs the following steps: if the usage duration corresponding to all usage records of each fourth event exceeds the second preset duration, delete all usage records of each fourth event.

[0200] The second preset duration is a shorter time period. There is no specific limitation on the length of the second preset duration; it can be set according to actual needs. For example, the length of the second preset duration can be, but is not limited to, 20 minutes, 30 minutes, 1 hour, or 2 hours.

[0201] The second preset duration is shorter than the first preset duration. For example, the first preset duration can be 24 hours, and the second preset duration can be 2 hours.

[0202] For example, the usage records of the L events included in the above deduplication result can be as follows: Figure 9 The table in (1) records 6 usage records. The events in the 3rd and 4th usage records are related events. The event in the 6th usage record (i.e., "unplug") should have a corresponding related event (i.e., "plug in"), but... Figure 9 In (1) shown, none of the six usage records have an event related to the event in the sixth usage record. Therefore, based on the data deduplication method described in implementation method three above, the sixth usage record needs to be deleted. The result after deletion is as follows: Figure 9 (2) shows the five usage records recorded in the table.

[0203] Each of the four fourth events mentioned above, each of the three third events mentioned above, each of the two second events mentioned above, and each of the one first events mentioned above can be the same or different. In this embodiment of the application, no specific limitation is made.

[0204] In the above technical solution, after performing data deduplication based on time characteristics (i.e., the time period mentioned above) on each of the S1 first events in the M usage records to be deduplicated, the resulting deduplication result includes each fourth event corresponding to a related event, but does not include the related events corresponding to each fourth event. Subsequently, deduplication is performed based on the correlation between events; that is, if the usage duration corresponding to all usage records of each fourth event is large, then all usage records of that fourth event are deleted. In other words, this method performs deduplication on the data from two dimensions: time characteristics and the correlation between events. This method has high versatility and robustness, and can improve the accuracy of the data deduplication results, thereby improving the quality of the deduplicated data.

[0205] Implementation Method 4:

[0206] In implementation method four, after deleting the remaining usage records other than the first usage record from at least two usage records corresponding to each of the first events in at least a portion of the time period to obtain the deduplication result, the electronic device further performs the following steps: if there is a logical error in multiple usage records corresponding to one event included in the deduplication result, delete at least one usage record with a logical error among the multiple usage records corresponding to one event; or, if there is a logical error in at least a portion of the usage records of one event and the usage record of another event in two related events included in the deduplication result, delete at least a portion of the usage records of one event and the usage record with a logical error among the usage records of the other event.

[0207] The data deduplication method described in the above implementation method 4 can delete the usage record with logical error if there is a logical error between one usage record and another usage record in the multiple usage records included in the deduplication result (e.g., usage record of one event, or multiple usage records of two related events).

[0208] A logical error exists between one usage record and another usage record, which could be that the event result recorded in one usage record does not conform to normal logic compared to the event result of the other usage record.

[0209] A logical error exists where the usage time of one record is earlier or later than that of another record. For example, when the usage time of one record is earlier than that of another, the event of the first record can be called the preceding event, and the event of the second record can be called the following event.

[0210] In one example, if the deduplication result includes multiple usage records corresponding to an event that contain logical errors, delete at least one usage record with a logical error among the multiple usage records corresponding to the event.

[0211] In the above implementation, if there is only one logical error among multiple usage records corresponding to an event, then that one usage record is deleted; if there are multiple logical errors among multiple usage records, then all of those usage records are deleted.

[0212] For example, the usage records of L events included in the deduplication result can be as follows: Figure 10 (1) shows four usage records. The first usage record includes a usage time of "2023-08-25 21:26:11", an event of "power off", and an event result of "94%". The second usage record includes a usage time of "2023-08-25 21:57:36", an event of "power off", and an event result of "100%". The event in the first usage record can be the preceding event described above, and the event in the second usage record can be the following event described above. According to normal logic, when an electronic device is in a power-off state, the power of the electronic device remains unchanged or decreases (for example, the power decreases due to the user's use of the electronic device). However, based on Figure 10 As shown in (1), the battery level corresponding to the first and second usage records indicates that the battery level of the electronic device increases from 90% to 95% when the device is unplugged. This contradicts normal logic and is not consistent with reality. Therefore, it is necessary to delete the second usage record, or delete both usage record 1 and usage record 2 simultaneously. Therefore, based on the above implementation method... Figure 10 The deduplication results obtained after performing deduplication on the four records shown in (1) can be found in [reference]. Figure 10 The three usage records shown in (2) are as follows.

[0213] In another example, if the deduplication result includes two related events, and at least a partial usage record of one event and a usage record of the other event contain a logical error, delete the usage record of at least a partial usage record of one event and the usage record of the other event containing the logical error.

[0214] In the above implementation, the number of usage records containing logical errors in at least part of the usage records of one event and the usage records of another event can be one or more, without specific limitation.

[0215] In the above implementation, the deduplication result may include one or more usage records of one of the two related events, and at least some of the usage records of that event may be one or more usage records of that event.

[0216] For example, the deduplication result might include two related events: a "charging" event and a "power off" event. The deduplication result might include the following related events: Record 1: "March 7, 2024, 10:43:53, Charging, 90%"; Record 2: "March 7, 2024, 11:43:53, Power off, 70%". Based on this deduplication result, it can be seen that after charging the device and then unplugging it, the device's battery level dropped from 90% to 70%. Logically, the battery level should not decrease after charging and then unplugging the device. Therefore, according to the deduplication method described above, when performing data deduplication on this result, record 2, which contains a logical error, needs to be deleted.

[0217] In the above technical solution, after performing data deduplication based on time features (i.e., the time period mentioned above) on each of the S1 first events in the M usage records to be deduplicated, if there are logical errors in multiple usage records corresponding to one or two events in the deduplication result, the usage records with logical errors will be deleted. This method performs deduplication on the data from the two dimensions of time features and the logic of related events. This method has high versatility and robustness, which can improve the accuracy of data deduplication results, thereby improving the quality of deduplicated data.

[0218] Implementation Method 5:

[0219] In practical applications, when an electronic device executes a target event, it will execute multiple events associated with that target event (e.g., music playback event) within a short period of time (e.g., music application opening event and audio event). Simultaneously, the electronic device will also record information about the execution of these associated events. In this situation, an accurate determination cannot be made based solely on a single event (e.g., music application opening event or audio event) among the multiple events associated with the target event. In other words, in this scenario, it is necessary to combine the multiple events associated with the target event. Only when these multiple events meet specific conditions can the multiple usage records corresponding to these multiple events be considered valid. In this case, there is no need to deduplicate the usage records including these multiple events. If the multiple events do not meet the specific conditions, the multiple usage records corresponding to these multiple events are considered invalid, and the usage records including these multiple events need to be deleted.

[0220] The specific condition mentioned above can be that the interval between the occurrence times of multiple events associated with a target event is less than a preset threshold. That is, in this embodiment, multiple events are determined to be actual events only when they occur simultaneously within a short time interval; in this case, the multiple events are not deleted. If only one event among the multiple events associated with a target event is triggered within a short time interval, then the multiple events need to be deleted. The short time interval is not specifically limited; for example, it can be, but is not limited to, one minute.

[0221] Based on this, in the above implementation method five, the deduplication result includes at least one usage record of the fifth event in the two associated events; and, for each first event, after deleting the remaining usage records other than the first usage record from at least two usage records of the first event corresponding to each time period of at least some time periods to obtain the deduplication result, the electronic device may further perform the following steps: if at least one usage record of the sixth event in the two associated events is not detected, delete at least one usage record of the fifth event, wherein the two associated events trigger the electronic device to execute the target event; or, if at least one usage record of the sixth event is detected, and the minimum interval between the usage time of at least one usage record of the fifth event and the usage time of at least one usage record of the sixth event exceeds a second threshold, delete the usage record of the fifth event associated with the minimum interval; if at least one usage record of the sixth event is detected, and the minimum interval between the usage time of at least one usage record of the fifth event and the usage time of at least one usage record of the sixth event does not exceed the second threshold, do not delete the usage record of the fifth event associated with the minimum interval.

[0222] The aforementioned minimum interval is the difference in usage time between the usage record of the fifth event associated with the minimum interval and at least one usage record of the sixth event.

[0223] There is a minimum interval between the usage time of at least one usage record of one event and the usage time of at least one usage record of another event.

[0224] For example, if at least one usage record of another event includes usage record A, and at least one usage record of another event is usage record B, where the usage time of usage record A is 10:00 and the usage time of usage record B is 10:02, then the minimum interval between the usage time of one usage record of the first event and the usage time of at least one usage record of the second event is 2 minutes.

[0225] For example, if at least one usage record of another event includes usage record 1 and usage record 2, and at least one usage record of an event is usage record 3, wherein the usage time of usage record 1 is 10:00, the usage time of usage record 2 is 10:05, and the usage time of usage record 3 is 9:55, then the minimum interval between the usage time of one usage record of the event and the usage time of at least one usage record of the other event is 5 minutes (i.e., the interval between the usage time of usage record 1 and the usage time of usage record 3 being 9:55).

[0226] The implementation method for determining whether the electronic device detects a usage record of the other event in the two related events in the above steps is not specifically limited. For example, the electronic device checks the deduplication results, and if it determines that the deduplication results do not include a usage record of the other event in the two related events, it determines that a usage record of the other event in the two related events has not been detected. For example, the electronic device checks its own stored data (i.e., including the deduplication results and the data excluding the deduplication results), and if it determines that its own stored data does not include a usage record of the other event in the two related events, it determines that a usage record of the other event in the two related events has not been detected.

[0227] The value of the second threshold is not specifically limited and can be set according to actual needs. For example, the second threshold can be, but is not limited to, 5 seconds or 1 minute.

[0228] For example, the deduplication result above is Figure 11 In example (1), two usage records of music application opening events (i.e., music intents) are shown, and the second threshold is 1 minute. The electronic device also stores Figure 11 (2) shows two usage records of the audio event. In this case, the deduplication method described in Implementation Method 5 above is used for... Figure 11 In (1), when deduplicating the two records shown, because... Figure 11 The usage time of record 1 shown in (1) is Figure 11 (2) shows that the interval of usage time for record #1 is the smallest, and this interval is 20 seconds. Therefore, record 1 can be considered as valid data and does not need to be deleted. Figure 11 The usage time of record 2 shown in (1) is Figure 11 In (2), record #2 has the shortest usage interval, which is 30 minutes. Therefore, record #2 can be considered invalid data and needs to be deleted. In summary, based on the above implementation method five, for Figure 11 The result shown in (1) is obtained by deduplicating the records. Figure 11(3) shows one usage record, which is Figure 11 Record 1 is shown in (1) of the document.

[0229] In the above technical solution, after performing data deduplication based on time characteristics (i.e., the time period mentioned above) for each of the S1 first events in the M usage records to be deduplicated, deduplication is then performed based on the correlation between events. That is, if the deduplication result only includes the usage record of one of the two related events, or if the interval between the usage times of the two related events is long, then it is necessary to delete the usage record of one of the two related events, or the usage time of the two related events. In other words, this method performs deduplication on the data from two dimensions: time characteristics and the correlation between events. This method has high versatility and robustness, and can improve the accuracy of the data deduplication result, thereby improving the quality of the deduplicated data.

[0230] It should be noted that the above description uses any one of the above implementation methods one to five as an example, which is executed by the electronic device after performing the above steps S710 and S720. Optionally, in some other implementation methods, the electronic device can also directly execute any one of the above implementation methods one to five on the data to be deduplicated (for example, the M usage records mentioned above).

[0231] It should be understood that the above Figure 7 The data deduplication method shown is for illustrative purposes only and does not constitute any limitation on the data deduplication method provided in this application. For example, the electronic device may execute only S710 and S720 above; the electronic device may execute only one of the above implementation methods one to five; the electronic device may execute multiple implementation methods one to five.

[0232] The above text combined Figures 3 to 11 This document describes in detail the application scenarios and methods of the data deduplication method according to embodiments of this application. The following will combine... Figure 12 This document describes in detail the device embodiments of this application. It should be understood that the data deduplication device in the embodiments of this application can execute the various data deduplication methods described in the foregoing embodiments of this application. That is, the specific working processes of the various products described below can be referred to the corresponding processes in the foregoing method embodiments.

[0233] Figure 12 This is a schematic diagram of a data deduplication device provided in an embodiment of this application. For example, Figure 12 The data deduplication device 1200 shown includes a processing unit 1210.

[0234] In one example, processing unit 1210 is used for:

[0235] Obtain the data to be deduplicated, wherein the data to be deduplicated includes M usage records, the M usage records include L event usage records, the L events include S1 first events, all usage records of each first event are located in at least one time period, each time period of at least some time periods within the at least one time period corresponds to at least two usage records of the first event, M is an integer greater than 1, and L and S1 are positive integers;

[0236] For each of the first events, among the at least two usage records of the first event corresponding to each of the at least some time periods, the remaining usage records other than the first usage record are deleted to obtain a deduplication result, wherein the first usage record is the usage record with the earliest usage time among the at least two usage records of the first event corresponding to each of the at least some time periods, and the deduplication result includes the usage records among the M usage records other than the remaining usage records corresponding to each of the at least some time periods for each of the first events.

[0237] In one possible implementation, the L events in the deduplication result include S2 second events, wherein the deduplication result specifically includes multiple usage records for each second event, and S2 is a positive integer; and the processing unit 1210 is further configured to: after deleting the remaining usage records other than the first usage records from at least two usage records corresponding to each of the at least some time periods for each first event to obtain the deduplication result, perform the following steps:

[0238] If the interval between the usage times of at least two usage records in the plurality of usage records of each second event does not exceed a first threshold, delete the usage record with the latest usage time among the at least two usage records of each second event.

[0239] In another possible implementation, the L events in the deduplication result include S3 third events, wherein each third event corresponds to an associated event, multiple usage records of each third event are consecutive, and multiple usage records of each third event are adjacent to the usage records of the corresponding associated event, and S3 is a positive integer; and the processing unit 1210 is further configured to: after deleting the remaining usage records other than the first usage record from at least two usage records of the first event corresponding to each of the at least some time periods to obtain the deduplication result, perform the following steps:

[0240] If the time between the usage record of the associated event corresponding to each third event and the usage time corresponding to the multiple usage records of each third event does not exceed a first preset time, delete the usage records of each third event except for the earliest usage record.

[0241] In another possible implementation, the L events in the deduplication result include S4 fourth events and there are no associated events corresponding to each of the fourth events, where S4 is a positive integer; and the processing unit 1210 is further configured to: after deleting the remaining usage records other than the first usage record from at least two usage records corresponding to the first event in each of the at least some time periods for each first event to obtain the deduplication result, perform the following steps:

[0242] If the usage duration corresponding to all usage records of each fourth event exceeds the second preset duration, all usage records of each fourth event are deleted.

[0243] In another possible implementation, the processing unit 1210 is further configured to: after deleting the remaining usage records other than the first usage record from at least two usage records corresponding to each of the at least partial time periods for each first event to obtain a deduplication result, perform the following steps:

[0244] If a logical error exists in multiple usage records corresponding to an event in the deduplication result, delete at least one usage record containing the logical error from among the multiple usage records corresponding to the event; or...

[0245] If, in the deduplication result, at least a portion of the usage record of one of the two related events contains a logical error, and the usage record of the other event contains a logical error, then delete the usage record of the at least portion of the usage record of the one event and the usage record of the other event containing the logical error.

[0246] In another possible implementation, the deduplication result includes at least one usage record of the fifth event in the two associated events; and the processing unit 1210 is further configured to:

[0247] After deleting the remaining usage records other than the first usage record from at least two usage records corresponding to each of the at least partial time periods for each first event to obtain a deduplication result, the following steps are performed:

[0248] If no usage record for the sixth event in the two associated events is detected, delete at least one usage record for the fifth event; or,

[0249] If at least one usage record of the sixth event is detected, and the minimum interval between the usage time of at least one usage record of the fifth event and the usage time of at least one usage record of the sixth event exceeds a second threshold, the usage record of the fifth event associated with the minimum interval is deleted.

[0250] In another example, processing unit 1210 is used for:

[0251] Obtain the data to be deduplicated, wherein the M usage records in the data to be deduplicated include S events, each event corresponds to an associated event, multiple usage records of each event are consecutive, and multiple usage records of each event are adjacent to the usage records of the corresponding associated event, where M is an integer greater than 2 and S is a positive integer;

[0252] If the time between the usage record of the associated event corresponding to each event and the usage time corresponding to the multiple usage records of each event does not exceed a preset time, delete the usage records of each event except for the earliest usage record.

[0253] In yet another example, processing unit 1210 is used for:

[0254] Obtain the data to be deduplicated, wherein the M usage records in the data to be deduplicated include S events and there is no associated event corresponding to each event, and each event corresponds to an associated event, where M is an integer greater than 1 and S is a positive integer;

[0255] If the usage duration corresponding to all usage records for each event exceeds a preset duration, all usage records for each event will be deleted.

[0256] In yet another example, processing unit 1210 is used for:

[0257] Obtain the data to be deduplicated, wherein the data to be deduplicated includes M usage records, where M is an integer greater than 1;

[0258] If a logical error exists in multiple usage records corresponding to one event among the M usage records, delete at least one usage record containing the logical error among the multiple usage records corresponding to the one event; or...

[0259] If at least a portion of the usage record for one of the two related events included in the M usage records contains a logical error, delete the usage record containing the logical error in at least a portion of the usage record for one event and the usage record for the other event.

[0260] In yet another example, processing unit 1210 is used for:

[0261] Obtain the data to be deduplicated, wherein the data to be deduplicated includes M usage records, and the M usage records include at least one usage record of the fifth event in two related events, where M is a positive integer;

[0262] If no usage record for the sixth event in the two associated events is detected, delete at least one usage record for the fifth event; or,

[0263] If at least one usage record of the sixth event is detected, and the minimum interval between the usage time of at least one usage record of the fifth event and the usage time of at least one usage record of the sixth event exceeds a second threshold, the usage record of the fifth event associated with the minimum interval is deleted.

[0264] It should be noted that the aforementioned data deduplication device 1200 is embodied in the form of a functional unit. The term "unit" here can be implemented in software and / or hardware, without specific limitations.

[0265] For example, a "unit" can be a software program, a hardware circuit, or a combination of both that implements the above functions. The hardware circuit may include an application-specific integrated circuit (ASIC), electronic circuitry, a processor (e.g., a shared processor, a proprietary processor, or a group processor) and memory for executing one or more software or firmware programs, integrated logic circuitry, and / or other suitable components that support the described functions.

[0266] Therefore, the units of the various examples described in the embodiments of this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0267] This application also provides a computer program product that, when executed by a processor, implements the data deduplication method described in any of the method embodiments of this application.

[0268] The computer program product can be stored in memory, for example, it is a program. The program is eventually converted into an executable object file that can be executed by the processor after processes such as preprocessing, compilation, assembly and linking.

[0269] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer, implements the data deduplication method described in any of the method embodiments of this application. The computer program may be a high-level language program or an executable object program.

[0270] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0271] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0272] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0273] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0274] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and other division methods may exist in actual implementation; for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0275] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0276] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0277] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data deduplication method, characterized in that, The method includes: Obtain the data to be deduplicated, wherein the data to be deduplicated includes M usage records, the M usage records include L event usage records, the L events include S1 first events, all usage records of each first event are located in at least one time period, each time period of at least some time periods within the at least one time period corresponds to at least two usage records of the first event, M is an integer greater than 1, and L and S1 are positive integers; For each first event, in at least two usage records corresponding to the first event in each of the at least partial time periods, the remaining usage records other than the first usage record are deleted to obtain a deduplication result. The first usage record is the usage record with the earliest usage time in the at least two usage records corresponding to the first event in each of the at least partial time periods. The deduplication result includes usage records in the M usage records other than the remaining usage records corresponding to each of the at least partial time periods for each first event. The L events in the deduplication result include S2 second events. The deduplication result specifically includes multiple usage records for each second event, where S2 is a positive integer. If the interval between the usage times of at least two usage records in the plurality of usage records of each second event does not exceed a first threshold, delete the usage record with the latest usage time among the at least two usage records of each second event.

2. The method according to claim 1, characterized in that, The L events in the deduplication result include S3 third events, wherein each third event corresponds to an associated event, multiple usage records of each third event are consecutive, and multiple usage records of each third event are adjacent to the usage records of the corresponding associated event, and S3 is a positive integer; and, after deleting the remaining usage records other than the first usage records from at least two usage records of the first event corresponding to each of the at least some time periods for each first event to obtain the deduplication result, the method further includes: If the duration between the usage time of the usage record of the associated event corresponding to each third event and the usage time corresponding to the multiple usage records of each third event does not exceed a first preset duration, delete the usage records of each third event except for the earliest usage record.

3. The method according to claim 1, characterized in that, The L events in the deduplication result include S4 fourth events and there is no associated event corresponding to each fourth event, where S4 is a positive integer; And, after deleting the remaining usage records other than the first usage record from at least two usage records corresponding to each of the at least partial time periods for each first event to obtain a deduplication result, the method further includes: If the usage duration corresponding to all usage records of each fourth event exceeds the second preset duration, all usage records of each fourth event are deleted.

4. The method according to claim 1, characterized in that, After deleting the remaining usage records other than the first usage record from at least two usage records corresponding to each of the at least partial time periods for each first event to obtain a deduplication result, the method further includes: If the deduplication result contains multiple usage records corresponding to one event and there are logical errors, delete at least one usage record with a logical error among the multiple usage records corresponding to the one event; or... If, in the deduplication result, at least a portion of the usage record of one of the two related events contains a logical error, and the usage record of the other event contains a logical error, then delete the usage record of the at least portion of the usage record of the one event and the usage record of the other event containing the logical error.

5. The method according to claim 1, characterized in that, The deduplication result includes at least one usage record of the fifth event in the two associated events; and, after deleting the remaining usage records other than the first usage record from at least two usage records of the first event corresponding to each of the at least partial time periods to obtain the deduplication result, the method further includes: If no usage record for the sixth event in the two associated events is detected, delete at least one usage record for the fifth event; or, If at least one usage record of the sixth event is detected, and the minimum interval between the usage time of at least one usage record of the fifth event and the usage time of at least one usage record of the sixth event exceeds a second threshold, the usage record of the fifth event associated with the minimum interval is deleted.

6. An electronic device, characterized in that, The electronic device includes one or more processors and a memory; the memory is coupled to the one or more processors and is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 5.

7. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the one or more processors being used to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 5.

Images