Data deduplication method, electronic equipment, storage medium and chip

Abstract

The invention discloses a data deduplication method, electronic equipment, a storage medium and a chip. The method comprises the steps that data to be subjected to duplicate removal are obtained, the data to be subjected to duplicate removal comprises M use records, the M use records comprise use records of L events, the L events comprise S1 first events, and all the use records of each first event are located in at least one time period, each time period of at least partial time periods in the at least one time period corresponds to at least two use records of the first event; for each first event, in at least two use records of the first event corresponding to each time period of the at least partial time periods, deleting the remaining use records except the first use record to obtain a duplicate removal result, the first use record is the use record with the earliest use time in the at least two use records of the first event corresponding to each time period of the at least partial time periods. According to the method, the accuracy of a data deduplication result can be improved, so that the data quality after deduplication is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and more specifically, to a data deduplication method, electronic device, storage medium, and chip. Background Art

[0002] With the continuous development of computer technology and network technology, the amount of data storage has also increased exponentially. These massive amounts of data (for example, usage records generated by the application of electronic devices) can be stored in a database.

[0003] Currently, with the surge in the amount of data stored in databases, if the amount of data in the database is not reduced, the database will store redundant and dirty data, resulting in poor data quality. Therefore, how to accurately deduplicate data is an urgent problem that needs to be solved. Summary of the Invention

[0004] The present application provides a data deduplication method, electronic device, storage medium and chip, which can improve the accuracy of data deduplication results, thereby improving the quality of deduplicated data.

[0005] In a first aspect, the present application provides a data deduplication method, the method comprising:

[0006] Obtaining data to be deduplicated, where the data to be deduplicated includes M usage records, the M usage records include usage records of L events, the L events include S1 first events, all usage records of each first event are in at least one time period, and each time period of at least some time periods within the at least one time period corresponds to at least two usage records of the first event, M is an integer greater than 1, and L and S1 are positive integers;

[0007] For each first event, in the at least two usage records of the first event corresponding to each time period of at least part of the time period, the remaining usage records other than the first usage record are deleted to obtain a deduplication result, wherein the first usage record is the usage record with the earliest usage time among the at least two usage records of the first event corresponding to each time period of at least part of the time period, and the deduplication result includes the usage records in the M usage records except the remaining usage records corresponding to each time period of at least part of the time period for each first event.

[0008] Optionally, multiple time periods correspond to multiple time buckets, and the time period of each time bucket is the corresponding time period.

[0009] In the above technical solution, when deduplication is performed on M usage records, for each first event, in the case of at least two usage records of the first event corresponding to each time period of at least part of the time period, it is necessary to delete the remaining usage records other than the usage record with the earliest usage time (i.e., the first usage record) in the at least two usage records of the first event corresponding to each time period of at least part of the time period. That is to say, based on the data deduplication method provided by the present application, for each first event, in the case of multiple usage records of each first event in each time period, only the usage record of each first event that appears for the first time in each time period (i.e., the first usage record) is retained. Therefore, this method can avoid the problem of missed deduplication that exists when traditional technology performs deduplication on data. In summary, the data deduplication method provided by the present application can improve the accuracy of the data deduplication results, thereby improving the quality of the data after deduplication.

[0010] In one possible implementation, the L events in the deduplication result include S2 second events, wherein the deduplication result specifically includes multiple usage records of each second event, where S2 is a positive integer; and after deleting, for each first event, the remaining usage records other than the first usage record from the at least two usage records of the first event corresponding to each time period of at least a portion of the time period to obtain the deduplication result, the method further includes:

[0011] When an interval between usage times corresponding to at least two usage records in the plurality of usage records of each second event does not exceed a first threshold, the usage record with the latest usage time in the at least two usage records of each second event is deleted.

[0012] The deduplication result is the result obtained by performing the above deduplication processing on the M usage records of the deduplication data. It can be seen that the L events in the deduplication result are the same as the L events in the M usage records, but the number of usage records corresponding to the L events in the deduplication result is different from the number of M usage records of the data to be deduplicated.

[0013] The multiple usage records of each second event in the S2 events are located in multiple time periods, and the mapping relationship between the multiple usage records and the multiple time periods is not specifically limited.

[0014] The L events in the deduplication result include S2 second events, and each of the S2 second events may be the same as or different from each of the above S1 first events.

[0015] The value of the first threshold is not specifically limited and can be set according to actual needs. For example, the first threshold can be, but is not limited to, 2 seconds or 3 seconds.

[0016] In the above technical solution, after performing data deduplication on each of the S1 first events in the M usage records included in the data to be deduplicated, the deduplication results obtained include multiple usage records of each second event located in multiple time periods. Based on this, each second event can be deduplicated again for short-term repeated behavioral data, that is, among the at least two usage records of the multiple usage records of each second event, the usage record with the latest usage time is deleted from the at least two usage records of each second event. This method can remove some events of the same type that happen to be distributed at the edges of adjacent time periods, minimize the data noise of the deduplication results obtained after the second deduplication, improve the accuracy of the data deduplication results, and thus improve the quality of the data after deduplication.

[0017] In another possible implementation, the L events in the deduplication result include S3 third events, where each third event corresponds to an associated event, multiple usage records of each third event are continuous, and multiple usage records of each third event are adjacent to usage records of the corresponding associated event, and S3 is a positive integer; and, for each first event, after deleting remaining usage records other than the first usage record from at least two usage records of the first event corresponding to each time period of at least a portion of the time period to obtain a deduplication result, the method further includes:

[0018] When the duration between the usage time of the usage record of the associated event corresponding to each third event and the usage time corresponding to multiple usage records of each third event does not exceed the first preset duration, the usage records except the usage record with the earliest usage time in the multiple usage records of each third event are deleted.

[0019] The multiple usage records of each third event are located in multiple time periods, and the mapping relationship between the multiple usage records and the multiple time periods is not specifically limited.

[0020] The third event corresponds to an associated event, wherein two associated events may be two events that are relatively connected or clearly correlated, and there is no specific limitation on the two associated events. For example, the two associated events may include a "plug in" event and a "unplug" event. For example, the two associated events may include a "device connected" event and a "device disconnected" event.

[0021] The deduplication result includes that the usage record of the associated event corresponding to each third event and the multiple consecutive usage records of each third event are adjacent.

[0022] For example, the third event is a power-off event, and the associated event corresponding to the third event is a "plug-in" event. The three consecutive usage records of the "plug-out" event and the usage record of the "plug-in" event can be described as: plug-in, unplug, unplug, unplug; or unplug, unplug, unplug, plug-in.

[0023] In the above technical solution, after performing data deduplication based on the time feature (i.e., the time period above) for each of the S1 first events in the M usage records included in the data to be deduplicated, the deduplication result obtained includes each third event corresponding to an associated event. Thereafter, deduplication is performed based on the correlation between events, that is, if the duration between the usage time of the usage record of the associated event corresponding to each third event and the usage time corresponding to the multiple usage records of each third event is shorter, then the usage records except for the usage record with the earliest usage time in the multiple usage records of each third event are deleted. This method deduplicates data from two dimensions: time feature and correlation between events. This method has high versatility and robustness, and can improve the accuracy of data deduplication results, thereby improving the quality of data after deduplication.

[0024] In another possible implementation, the L events in the deduplication result include S4 fourth events and there is no associated event corresponding to each fourth event, where S4 is a positive integer; and, for each first event, after deleting remaining usage records other than the first usage record from at least two usage records of the first event corresponding to each time period of at least a portion of the time period to obtain a deduplication result, the method further includes:

[0025] When the usage duration corresponding to all usage records of each fourth event exceeds the second preset duration, all usage records of each fourth event are deleted.

[0026] In the above technical solution, after performing data deduplication based on the time feature (i.e., the time period above) for each of the S1 first events in the M usage records included in the data to be deduplicated, the deduplication result obtained includes each fourth event corresponding to the associated event, and the deduplication result does not include the associated event corresponding to each fourth event. Thereafter, deduplication is performed based on the correlation between events, that is, if the usage time corresponding to all usage records of each fourth event is longer, then all usage records of each fourth event are deleted. In other words, this method performs deduplication on data from the two dimensions of time feature and correlation between events. This method has high versatility and robustness, and can improve the accuracy of data deduplication results, thereby improving the quality of data after deduplication.

[0027] In another possible implementation, after deleting the remaining usage records other than the first usage record from the at least two usage records of the first event corresponding to each time period of at least a portion of the time period for each first event to obtain a deduplication result, the method further includes:

[0028] In the case that an event result of one usage record and an event result of another usage record in two usage records included in the deduplication result contain logical errors, the usage record containing the logical errors in the two usage records is deleted.

[0029] Two usage records correspond to two different events, or two usage records correspond to the same event.

[0030] In the above technical solution, after performing data deduplication based on the time feature (i.e., the time period above) for each of the S1 first events in the M usage records included in the data to be deduplicated, when there are logical errors in the two usage records included in the deduplication results, the usage records with logical errors are deleted. This method performs deduplication on the data from the two dimensions of time features and event logic respectively. This method has high versatility and robustness, and can improve the accuracy of the data deduplication results, thereby improving the quality of the data after deduplication.

[0031] In another possible implementation, after deleting the remaining usage records other than the first usage record from the at least two usage records of the first event corresponding to each time period of at least a portion of the time period for each first event to obtain a deduplication result, the method further includes:

[0032] If there are logical errors in multiple usage records corresponding to an event included in the deduplication result, at least one usage record with the logical error among the multiple usage records corresponding to the event is deleted; or

[0033] If there are logical errors in at least part of the usage records of one event and the usage records of the other event in the two associated events included in the deduplication result, the usage records with the logical errors in at least part of the usage records of one event and the usage records of the other event are deleted.

[0034] In the above technical solution, after data deduplication is performed based on the time feature (i.e., the time period above) for each of the S1 first events in the M usage records included in the data to be deduplicated, when the deduplication results include multiple usage records corresponding to one event or two events with logical errors, the usage records with logical errors will be deleted. This method performs deduplication on data from two dimensions, namely, time features and the logic of related events. This method has high versatility and robustness, and can improve the accuracy of the data deduplication results, thereby improving the quality of the data after deduplication.

[0035] In another possible implementation, the deduplication result includes at least one usage record of a fifth event in the two associated events; and after, for each first event, deleting remaining usage records other than the first usage record from the at least two usage records of the first event corresponding to each time period of at least a portion of the time period to obtain the deduplication result, the method further includes:

[0036] In the case where at least one usage record of the sixth event in the two associated events is not detected, deleting at least one usage record of the fifth event; or,

[0037] When at least one usage record of the sixth event is detected and the minimum interval between the usage time of at least one usage record of the fifth event and the usage time of at least one usage record of the sixth event exceeds a second threshold, the usage record of the fifth event associated with the minimum interval is deleted.

[0038] In the above technical solution, if two related events (i.e., the fifth event and the sixth event) both occur within a relatively short period of time, the two related events can be considered to be real events, and the usage records of the real events need to be retained during data deduplication. If only one of the two related events occurs within a relatively short period of time, or the interval between the occurrence times of the two related events is relatively long, the two related events are considered not to be real events, and the usage records of the events that did not actually occur need to be deleted during data deduplication.

[0039] In a second aspect, the present application provides a data deduplication method, the method comprising:

[0040] Obtaining data to be deduplicated, wherein M usage records in the data to be deduplicated include S events, each event corresponds to an associated event, multiple usage records of each event are continuous, and multiple usage records of each event are adjacent to usage records of the corresponding associated event, M is an integer greater than 2, and S is a positive integer;

[0041] When the duration between the usage time of the usage record of the associated event corresponding to each event and the usage time corresponding to multiple usage records of each event does not exceed the preset duration, the usage records except the usage record with the earliest usage time in the multiple usage records of each event are deleted.

[0042] In a third aspect, the present application provides a data deduplication method, the method comprising:

[0043] Obtaining data to be deduplicated, wherein M usage records in the data to be deduplicated include S events and there is no associated event corresponding to each event, each event corresponds to an associated event, M is an integer greater than 1, and S is a positive integer;

[0044] When the usage duration corresponding to all usage records of each event exceeds a preset duration, all usage records of each event are deleted.

[0045] In a fourth aspect, the present application provides a data deduplication method, the method comprising:

[0046] Obtaining data to be deduplicated, wherein the data to be deduplicated includes M usage records, where M is an integer greater than 1;

[0047] In the case where multiple usage records corresponding to one event included in the M usage records have logical errors, deleting at least one usage record having the logical error among the multiple usage records corresponding to one event; or

[0048] When there are logical errors in at least a portion of the usage record of one event and the usage record of the other event in two associated events included in M ​​usage records, the usage record with the logical errors in at least a portion of the usage record of one event and the usage record of the other event are deleted.

[0049] In a fifth aspect, the present application provides a data deduplication device, which includes a processing unit, and the processing unit is used to execute any one of the methods in the first to fourth aspects.

[0050] In a sixth aspect, the present application provides an electronic device comprising a unit for executing any one of the methods of aspects 1 to 4. The device may be a terminal device or a chip within the terminal device. The device may include an input unit and a processing unit.

[0051] When the device is a terminal device, the processing unit may be a processor, and the input unit may be a communication interface; the terminal device may also include a memory for storing computer program code, and when the processor executes the computer program code stored in the memory, the terminal device executes any one of the data deduplication methods in the first to fourth aspects.

[0052] When the device is a chip in a terminal device, the processing unit can be a processing unit inside the chip, and the input unit can be an output interface, a pin or a circuit, etc.; the chip can also include a memory, which can be a memory inside the chip (for example, a register, a cache, etc.), or a memory located outside the chip (for example, a read-only memory, a random access memory, etc.); the memory is used to store computer program code, and when the processor executes the computer program code stored in the memory, the chip executes any one of the data deduplication methods in the first to fourth aspects.

[0053] In a possible implementation, the memory is configured to store computer program code; and the processor is configured to execute the computer program code stored in the memory, and when the computer program code stored in the memory is executed, the processor is configured to execute any one of the data deduplication method in the first aspect to the fourth aspect.

[0054] In a seventh aspect, the present application provides a computer readable storage medium, which stores computer program code, and when the computer program code is run by a data deduplication device, the data deduplication device executes any one of the data deduplication method in the first aspect to the fourth aspect.

[0055] In an eighth aspect, the present application provides a computer program product, which comprises computer program code, and when the computer program code is run by a data deduplication device, the data deduplication device executes any one of the data deduplication method in the first aspect to the fourth aspect.

[0056] It can be understood that the beneficial effects of the above-mentioned second aspect to the eighth aspect can be referred to the related description in the first aspect, and will not be repeated here.

[0057] It should be understood that the description of technical features, technical solutions, beneficial effects or similar language in the present application does not imply that all features and advantages can be realized in any single embodiment. On the contrary, it can be understood that the description of a feature or a beneficial effect means that the specific technical feature, technical solution or beneficial effect is included in at least one embodiment. Therefore, the description of technical features, technical solutions or beneficial effects in the specification does not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions and beneficial effects described in the embodiments can be combined in any appropriate manner. Those skilled in the art will understand that the embodiments can be implemented without one or more specific technical features, technical solutions or beneficial effects of a specific embodiment. In other embodiments, additional technical features and beneficial effects can be identified in specific embodiments that do not embody all embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 is a schematic diagram of a hardware system of an electronic device 100 provided by an embodiment of the present application.

[0059] Figure 2 is a software structure block diagram of the electronic device 100 of the embodiment of the present application.

[0060] Figure 3 is a schematic diagram of a data deduplication method provided by an embodiment of the present application.

[0061] Figure 4 is a schematic diagram of data to be deduplicated provided by an embodiment of the present application.

[0062] Figure 5 This embodiment of the present application provides the above Figure 4 Schematic diagram of the result obtained after deduplication of the data to be deduplicated.

[0063] Figure 6 This embodiment of the present application provides the above Figure 5 Schematic diagram of the results obtained after performing deduplication again.

[0064] Figure 7 This is a schematic diagram of a data deduplication method provided in an embodiment of the present application.

[0065] Figure 8 This is a schematic diagram of a deduplication result provided in an embodiment of the present application.

[0066] Figure 9 This is a schematic diagram of a deduplication result provided in an embodiment of the present application.

[0067] Figure 10 This is a schematic diagram of a deduplication result provided in an embodiment of the present application.

[0068] Figure 11 This is a schematic diagram of a deduplication result provided in an embodiment of the present application.

[0069] Figure 12 This is a schematic diagram of a data deduplication device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0071] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; “and / or” in this article is merely a way to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can mean: A exists alone; A and B exist at the same time; B exists alone.

[0072] In the description of the embodiments of this application, the terms "first," "second," "third," etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly specifying the number of the technical features indicated. Thus, features defined as "first," "second," "third," etc. may explicitly or implicitly include one or more features. In the description of this embodiment, unless otherwise specified, "multiple" means two or more.

[0073] In the description of the embodiments of the present application, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their collections. The terms "including," "comprising," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0074] Below, the technical solution provided by this application is described in detail.

[0075] The data deduplication method provided in the embodiment of the present application can be applied to electronic devices. Below, the hardware structure and software structure of the electronic device are described in detail with reference to the accompanying drawings.

[0076] Figure 1 Schematic diagram of the hardware system of the electronic device 100 provided in an embodiment of the present application.

[0077] There is no specific limitation on the type of electronic device 100 and it can be selected according to the actual scenario. For example, the electronic device 100 can be a mobile phone, a smart screen, a tablet computer, a wearable electronic device, an in-vehicle electronic device, an augmented reality (AR) device, a virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a projector, an in-vehicle device, etc. The embodiments of the present application do not impose any limitation on the specific type of the electronic device 100.

[0078] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0079] It should be noted that Figure 1 The structure shown does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include Figure 1 More or fewer components than those shown, or the electronic device 100 may include Figure 1 Combinations of some of the components shown, or alternatively, the electronic device 100 may include Figure 1 Subassemblies of some of the components shown. Figure 1 The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0080] The processor 110 may include one or more processing units. For example, the processor 110 may include at least one of the following processing units: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and a neural-network processing unit (NPU). The different processing units may be independent devices or integrated devices.

[0081] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0082] A memory may also be provided in the processor 110 for storing instructions and data. For example, instructions for executing the data deduplication method provided in an embodiment of the present application may be stored in the processor 110. For example, data to be deduplicated or the deduplication results obtained by executing the data deduplication method provided in an embodiment of the present application may be stored in the processor 110. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to use the instruction or data again, it may be called directly from the memory. Repeated access is avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved. In some embodiments, the processor 110 may include one or more interfaces. For example, the processor 110 may include at least one of the following interfaces: an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and a USB interface.

[0083] Figure 1 The connection relationship between the modules shown is only for illustrative purposes and does not limit the connection relationship between the modules of the electronic device 100. Optionally, the modules of the electronic device 100 may also adopt a combination of the multiple connection modes in the above embodiments.

[0084] The hardware system of electronic device 100 is described in detail above. The following describes the software system of electronic device 100. The software system can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. In the embodiment of the present application, the layered architecture is used as an example to exemplify the software system of electronic device 100.

[0085] For example, Figure 2 Schematic diagram of the software system of the electronic device 100 provided in the embodiment of the present application. Figure 2The software system adopts a layered architecture. The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers: from top to bottom, the application layer 210, the application framework layer 220, the Android runtime and core library layer 230, the hardware abstraction layer (HAL) 240, and the kernel layer 250.

[0086] The application layer 210 may include a series of application packages. For example, the application package may include voice assistant, clipping, camera, gallery, chat, call, map, navigation, calendar, Bluetooth, music, video and other applications.

[0087] Each of the above applications may include more specific functional modules. For example, a gallery may include a business module and a notification module. For example, a camera may include a photo module.

[0088] The aforementioned respective application programs may be used to generate application data. For example, the gallery is used to generate photo data.

[0089] The application framework layer 220 provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer 220 includes some predefined functions.

[0090] like Figure 2 As shown, the application framework layer 220 may include a window manager, a notification manager, an activity manager, an input manager, a view system, a content provider, a resource manager, and the like.

[0091] The window manager provides window management services (WMS). WMS can be used for window management, window animation management, surface management, and as a transfer station for the input system.

[0092] Content providers are used to store and retrieve data and make it accessible to applications. This data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0093] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images. For example, a display interface can be, but is not limited to, Figure 4 or Figure 6 The user interface shown.

[0094] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0095] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0096] The activity manager can provide activity management services (AMS), which can be used to start, switch, and schedule system components (such as activities, services, content providers, and broadcast receivers) as well as manage and schedule application processes.

[0097] The input manager provides input management services (IMS), which can be used to manage system input, such as touch screen input, key input, and sensor input. The IMS retrieves events from input device nodes and, through interaction with the WMS, distributes the events to the appropriate window.

[0098] The Android Runtime consists of the core library and the virtual machine. The Android Runtime is responsible for scheduling and management of the Android system.

[0099] The core library consists of two parts: one part is the function that the programming language (for example, Java) needs to call, and the other part is the Android core library.

[0100] The application layer 210 and the application framework layer 220 run in a virtual machine. The virtual machine executes the programming files (e.g., Java files) of the application layer 210 and the application framework layer 220 as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0101] The core library layer 230 may include multiple functional modules, such as a surface manager, a media framework, libc, SQLite, OpenGL ES, and Webkit.

[0102] The surface manager is used to manage the display subsystem and provide the fusion of two-dimensional (2D) and three-dimensional (3D) layers for multiple applications.

[0103] The media framework supports playback and recording of a variety of common audio and video formats, as well as static image files.

[0104] libc (C library) is the standard library for the C language. It's a low-level library in the system and is implemented through Linux system calls. For example, libc can be used to connect to and disconnect from the camera service, set camera shooting parameters, start and stop previewing, and take pictures.

[0105] The hardware abstraction layer (HAL) 240 is an interface layer located between the operating system kernel and the upper-level software, and its purpose is to abstract the hardware. The hardware abstraction layer is an abstract interface driven by the device kernel, which is used to implement the application programming interface that provides access to the underlying device to the higher-level Java API framework. HAL contains multiple library modules, such as camera HAL (such as aperture, TOF sensor, lens or focus motor, etc.), Vendor warehouse, display, Bluetooth, audio, etc. Each library module implements an interface for a specific type of hardware component. It can be understood that the camera HAL can provide an interface for the camera FWK to access hardware components such as the camera. The Vendor warehouse can provide an interface for the media FWK to access hardware components such as the encoder. When the system framework layer API requires access to the hardware of the portable device, the Android operating system will load the library module for the hardware component.

[0106] The kernel layer 250 is the foundation of the Android operating system. The final functions of the Android operating system are all completed through the kernel layer. The kernel layer may include display drivers, camera drivers, audio drivers, and sensor drivers.

[0107] It should be noted that the application provides Figure 2 The software structure diagram of the electronic device shown is only an example and does not limit the specific module division in different layers of the Android operating system. For details, please refer to the introduction of the Android operating system software structure in conventional technology. In addition, the data deduplication method provided in this application can also be implemented based on other operating systems (for example, IOS or Hongmeng, etc.), and this application will not give examples one by one.

[0108] Next, combine Figures 3 to 7 The data deduplication method provided in the embodiments of the present application is introduced in detail.

[0109] Figure 3Schematic diagram of a data deduplication method provided in an embodiment of the present application. The data deduplication method provided in an embodiment of the present application can be executed by an electronic device. It is understood that the electronic device can be implemented as software, or a combination of software and hardware. For example, the electronic device in the embodiment of the present application can be, but is not limited to, Figure 1 The electronic device 100 is shown. Figure 3 As shown, the data deduplication method provided in the embodiment of the present application includes steps S310 to S340. Below, steps S310 to S340 are described in detail.

[0110] S310, the electronic device obtains user service data 1, wherein the user service data 1 includes multiple usage records 1 corresponding to multiple applications, and each usage record 1 includes at least an identification of the application, an event corresponding to the application, and time information of using the application.

[0111] There is a one-to-one correspondence between the multiple applications and the multiple usage records 1 , and each usage record 1 records information about the user using the corresponding application.

[0112] Each usage record 1 includes at least an identifier of the application, an event corresponding to the application, and time information of using the application.

[0113] The application identifier is used to identify the application and is not specifically limited to the application identifier. For example, the application identifier can be the application package name or the application name.

[0114] The events corresponding to an application are used to indicate the purpose of the application. For example, taking a chat application as an example, the event corresponding to the chat application is chat. For example, taking a weather application as an example, the event corresponding to the weather application is check the weather.

[0115] The time information of using an application is used to indicate the time when an event corresponding to the application occurs. For example, the time when an event corresponding to the application occurs can be a time period or a moment, etc., and is not specifically limited to this. In one example, the time information of using an application can include the time when the application is used (i.e., the time when the event corresponding to the application occurs).

[0116] It should be noted that each usage record 1 described above may also include other information, which is not specifically limited. For example, the other information may be a user's identity document (ID), where the user's ID may refer to the user's account number used to log in to the application.

[0117] For example, the user service data 1 in the above step S310 includes multiple usage records 1 corresponding to multiple applications, which can be as follows: Figure 4 The table shows a plurality of usage records 1. See Figure 3Each use record 1 corresponding to each application includes the id of the user, the package name of the application, the event corresponding to the application, and the time of using the application. For example, the use record 1 recorded in the first row of the table shown in the table is taken as an example, the use record 1 includes the id of the user is "123", the package name of the application is "package name A", the event corresponding to the application is "chat", and the time of using the application is "2023-06-02 21:41:30.560". Figure 3

[0118] In the embodiments of the present application, the acquisition method of the electronic device for acquiring the user service data 1 is not specifically limited. For example, the electronic device acquires the user service data 1, which can include the following steps: in response to the electronic device running each application in the plurality of applications, the electronic device records each use record 1 corresponding to each application in the storage space allocated by the electronic device to each application; and the electronic device acquires the user service data 1 including a plurality of use records 1 corresponding to a plurality of applications by accessing a plurality of storage spaces allocated to the plurality of applications.

[0119] S320, the electronic device obtains a plurality of time buckets according to the plurality of use records 1, wherein the plurality of time buckets correspond to a plurality of continuous time periods, each time bucket includes at least one use record 1 in the plurality of use records 1, and the at least one use record 1 includes the time indicated by the use time information within the time period corresponding to each time bucket.

[0120] The plurality of continuous time periods means that there is no time interval between any two adjacent time periods in the plurality of time periods. For example, taking time period 1 as "20:00:00 to 20:05:59", time period 2 as "20:06:00 to 20:07:59", and time period 3 as "20:08:00 to 20:09:59" as an example, time period 1, time period 2 and time period 3 are three continuous time periods, and time period 1 and time period 3 are two discontinuous time periods.

[0121] The length of each time period in the plurality of continuous time periods is not specifically limited and can be set according to actual conditions. For example, each time period can be but is not limited to 5 minutes, 8 minutes, or 10 minutes, etc.

[0122] For example, continuing the plurality of use records 1 corresponding to the plurality of applications in the S320 step described above, the content recorded in the table shown in the table, and each time period is 10 minutes, the plurality of use records 1 corresponding to the plurality of applications can be divided as Figure 4 Figure 4 ​​There are 6 time buckets shown, among which the time period corresponding to time bucket 1 is "21:41:00-21:50:00", the time period corresponding to time bucket 2 is "21:51:00-22:00:00", the time period corresponding to time bucket 3 is "22:01:00-22:10:00", the time period corresponding to time bucket 4 is "22:11:00-22:20:00", the time period corresponding to time bucket 5 is "22:21:00-22:30:00", and the time period corresponding to time bucket 6 is "22:31:00-22:40:00".

[0123] For example, taking time bucket 1 as an example, time bucket 1 includes Figure 4 The first four rows of the table show four usage records 1 , and the time indicated by the usage time information included in each usage record is within the time period corresponding to time bucket 1 .

[0124] In the embodiment of the present application, there is no specific limitation on the electronic device executing the above step S320. The following describes the process of the electronic device executing the above step S320 by way of example. It should be understood that the process described below is merely illustrative and does not constitute any limitation on the process of the electronic device obtaining multiple time buckets based on multiple usage records 1.

[0125] In one example, an electronic device obtains multiple time buckets based on multiple usage records 1, which can include the following steps: the electronic device obtains a start time, an end time and a preset time interval 1, wherein the start time is the usage time of the earliest used application among the multiple applications corresponding to the multiple usage records 1, and the start time is the usage time of the latest used application among the multiple applications corresponding to the multiple usage records 1; the electronic device obtains multiple consecutive time periods based on the start time, the end time and the preset time interval 1, wherein the time length of the preset time interval 1 is the same as the time length of the time period corresponding to each time bucket; the electronic device obtains multiple time buckets based on the multiple consecutive time periods and the time of using the application recorded in each usage record 1 in the multiple usage records 1.

[0126] S330, the electronic device performs deduplication processing on at least one usage record 1 included in each time bucket according to data deduplication strategy 0, and obtains an initial deduplication result, wherein the initial deduplication result includes multiple deduplicated time buckets corresponding to the multiple time buckets.

[0127] The multiple time buckets correspond to the multiple deduplicated time buckets one by one, wherein each deduplicated time bucket is a result obtained by the electronic device performing deduplication processing on the corresponding time bucket according to data deduplication strategy 0.

[0128] Data deduplication strategy 0 includes retaining only the first usage record 1 included in each time bucket when the identifiers of multiple applications corresponding to multiple usage records 1 included in each time bucket are the same, and the multiple events corresponding to the multiple usage records 1 are the same. The time of using the application recorded in the first usage record 1 is earlier than the time of using the application recorded in other usage records 1 in the multiple usage records 1 included in each time bucket.

[0129] For example, continue to use the multiple time buckets in the above step S330 as Figure 4 Taking the 6 time buckets shown in the figure as an example, the electronic device follows the data deduplication strategy 0 described above, Figure 4 After deduplication processing is performed on each of the 6 time buckets shown, the following can be obtained: Figure 5 The six time buckets after deduplication are shown.

[0130] Specifically, Figure 4 The time bucket i and Figure 6 The time bucket i after deduplication is shown as follows, i = 1, 2, 3, 4, 5, 6. Figure 4 The time buckets 1 and 2 are shown. Figure 6 Taking the deduplicated time bucket 1 as an example, the time bucket 1 includes 4 usage records 1, wherein the package name of the application corresponding to the second usage record 1 and the third usage record 1 in the 4 usage records 1 is the same (i.e., package name B), and the events corresponding to the second usage record 1 and the third usage record 1 included in the time bucket 1 are the same (i.e., community). Therefore, the deduplicated time bucket 1 corresponding to the time bucket 1 should include the following: Figure 5 The three usage records 1 shown are Figure 4 The first usage record 1, the second usage record 1, and the fourth usage record 1 in time bucket 1 are shown.

[0131] In the above-mentioned step S330, the electronic device deduplicates each time bucket obtained in the above-mentioned step S320 according to the data deduplication strategy 0, that is, when each time bucket includes multiple repeated usage records, only the first usage record in each time bucket is retained. In this way, the problem of missed deduplication caused by chain data processing (the same event occurs continuously and will be interrupted by other events) can be solved.

[0132] S340 , the electronic device performs deduplication processing on the usage records 1 included in the multiple deduplication time buckets according to the data deduplication strategy 1 , and obtains the target deduplication result 1 .

[0133] Data deduplication strategy 1 includes deleting the usage record corresponding to the latest usage application time of the two usage records 1 corresponding to two adjacent deduplicated time buckets if the interval between the two usage application times corresponding to the two usage records 1 is less than the preset time interval 2 when the application identifiers of the two usage records 1 corresponding to the two usage records 1 are the same and the events corresponding to the two usage records 1 are the same.

[0134] The two time periods corresponding to two adjacent deduplicated time buckets are two consecutive time periods. Figure 5 Taking the multiple deduplicated time buckets shown as an example, deduplicated time bucket 1 and deduplicated time bucket 2 are adjacent deduplicated time buckets. Deduplicated time bucket 2 and deduplicated time bucket 3 are adjacent deduplicated time buckets. Deduplicated time bucket 2 and deduplicated time bucket 4 are not adjacent deduplicated time buckets.

[0135] The length of the preset time interval 2 is less than the length of each time bucket after deduplication, and the preset time interval 2 can be set according to actual conditions. For example, the preset time interval 2 can be, but is not limited to, 1 minute, 2 minutes, or 3 minutes.

[0136] It should be noted that the time length corresponding to each deduplicated time bucket, the time length corresponding to each time bucket, and the time length of the preset time interval 1 mentioned above are all the same.

[0137] In an example, one of the two usage records 1 corresponding to the two adjacent deduplicated time buckets described in the current text is the first usage record 1 in one of the two adjacent deduplicated time buckets, the other usage record 1 is the last usage record 1 in the other deduplicated time bucket, and the time period corresponding to the other deduplicated time bucket is the time period before the time period corresponding to the one deduplicated time bucket.

[0138] For example, the following is a multiple time bucket after deduplication in the above step S340. Figure 5 Taking the six deduplication time buckets shown as an example, the target deduplication result 1 obtained after the electronic device executes the above S340 is described as an example.

[0139] See also Figure 5The rectangular box 601 shown includes the usage record 1A recorded in the time bucket 2 after deduplication, and the usage record 1B recorded in the time bucket 3 after deduplication. It can be seen that the package name (i.e., package name A) of the application corresponding to these two usage records 1 (i.e., usage record 1A and usage record 1B) is the same, and the events (i.e., chat) corresponding to these two usage records 1 are also the same. Based on this, when the interval between the time of using the two applications corresponding to these two usage records 1 is less than 5 minutes (i.e., an example of the preset time interval 2), based on the data deduplication strategy 1 described above, the usage record 1B recorded in the time bucket 3 after deduplication needs to be deleted.

[0140] See also Figure 5 The rectangular box 602 shown includes the usage record 1C recorded in the time bucket 4 after deduplication, and the usage record 1D recorded in the time bucket 5 after deduplication. It can be seen that the package name of the application corresponding to these two usage records 1 (i.e., usage record 1C and usage record 1D) is the same (i.e., package name H), and the events corresponding to these two usage records 1 (i.e., local life) are also the same. Therefore, based on the data deduplication strategy 1 described above, it is necessary to delete the usage record 1D recorded in the time bucket 5 after deduplication. Based on this, since the interval between the time of using the two applications corresponding to these two usage records 1 exceeds 5 minutes (i.e., an example of the preset time interval 2), based on the data deduplication strategy 1 described above, there is no need to delete the usage record 1D recorded in the time bucket 3 after deduplication.

[0141] Based on this, electronic devices are deduplicated according to data deduplication strategy 1. Figure 5 The usage record 1 recorded in the 6 deduplicated time buckets shown is deduplicated to obtain the target deduplication result 1 as shown in FIG. Figure 6 The contents recorded in the table are shown.

[0142] In the above-mentioned step S340, the electronic device performs a short-term (i.e., preset time interval 2) repeated behavioral data deduplication on the initial deduplication result (i.e., multiple deduplicated time buckets) obtained by executing the above-mentioned step 330, and removes some repeated usage records 1 that happen to be on the edge of two adjacent deduplicated time buckets. In this way, data noise can be minimized.

[0143] It should be understood that the above Figure 3 The data deduplication method shown is for illustration only and does not constitute any limitation on the data deduplication method provided in this application.

[0144] Figure 7Schematic diagram of a data deduplication method provided in an embodiment of the present application. The data deduplication method provided in an embodiment of the present application can be executed by an electronic device. It is understood that the electronic device can be implemented as software, or a combination of software and hardware. For example, the electronic device in the embodiment of the present application can be, but is not limited to, Figure 1 The electronic device 100 is shown. Figure 7 As shown, the data deduplication method provided in the embodiment of the present application includes S710 and S720. Below, steps S710 and S720 are described in detail.

[0145] S710, the electronic device obtains data to be deduplicated, wherein the data to be deduplicated includes M usage records, the M usage records include usage records of L events, the L events include S1 first events, all usage records of each first event are located in at least one time period, and each time period of at least part of the at least one time period corresponds to at least two usage records of the first event, M is an integer greater than 1, and L and S1 are positive integers.

[0146] Each of the M usage records corresponds to an event, meaning each usage record records a corresponding event. There are no specific restrictions on the type of event recorded in each usage record. For example, each usage record can record, but is not limited to, any of the following events: chatting, messaging, community, charging, power outage, connecting to Bluetooth, disconnecting from Bluetooth, watching videos, listening to music, making payments, or working.

[0147] In the embodiment of the present application, as described above, each usage record is used to record a corresponding event. There is no specific limitation on the content of each usage record in the M usage records, and it can be determined according to actual conditions.

[0148] In one example, each usage record may include the usage time of the event (ie, the time when the event occurred) and the event.

[0149] In another example, each usage record may include a user identifier that triggers the occurrence of an event (eg, the user's ID), the usage time of the event (ie, the occurrence time of the event), and the event.

[0150] In another example, each usage record may include the user identifier (such as a user account) that triggers the event, the application identifier associated with the event, the usage time of the event (i.e., the time when the event occurs), and the event. The application indicated by the application identifier associated with the event is used to generate the corresponding event, wherein the application identifier associated with the event can be the application name (also known as the application package name). For example, taking the event of chatting as an example, the application identifier associated with the event can be the application identifier of the chat application. For example, taking the event of listening to music as an example, the application identifier associated with the event can be the application identifier of the music application.

[0151] Optionally, the usage record of each of the above examples may also include an event result of the event. For example, taking charging as an example, the event result of the event may be the amount of electricity.

[0152] The data to be deduplicated includes M usage records, and the total number of events of all types included in the M usage records is L events, where M is an integer greater than 1 and L is a positive integer. There is no specific limit on the number of events (i.e., L) included in the M usage records. For example, the value of L can be, but is not limited to, 1, 2, 5, 7, or 10.

[0153] For example, the above M (ie, M=4) usage records are used as Figure 4 The time bucket 1 shown includes 4 usage records as an example. Figure 4 As shown, the event of the first usage record in the four usage records is "chat", the events of the second usage record and the third usage record are both "community", and the event of the fourth usage record is "note intention". Therefore, the four usage records include 3 (i.e., L=3) events (i.e., "chat", "community" and "note intention").

[0154] In an embodiment of the present application, the L events included in the M usage records include S1 first events, wherein all usage records of each first event are located in at least one time period, and each time period of at least part of the time period within at least one time period corresponds to at least two usage records of the first event, and S1 is a positive integer.

[0155] All usage records of each first event are located in at least one time period, that is, all usage records of each first event can be located in one time period or multiple time periods, which is not specifically limited in the embodiments of the present application.

[0156] Each of at least a portion of the at least one time period corresponds to at least two usage records of the first event. In one example, when the at least one time period is one time period, the at least a portion of the at least one time period refers to the one time period. In another example, when the at least one time period is multiple time periods, the at least a portion of the at least one time period refers to one or more time periods among the multiple time periods.

[0157] Optionally, each time period corresponds to a time bucket, that is, the time period of each time bucket is the corresponding time period. When there are multiple time periods, the multiple time periods correspond to the multiple time buckets one-to-one.

[0158] M is an integer greater than 1, L and S1 are positive integers, and S1 is a positive integer less than or equal to L. The values ​​of M, L, and S1 can be set according to actual circumstances and are not specifically limited. The following example describes the M usage records, L events, and S1 first events described above.

[0159] In one example, when S1 is a positive integer equal to 1, the L events included in the M usage records include only one first event, all usage records of the first event are located in at least one time period, and each time period of at least part of the at least one time period corresponds to at least two usage records of the first event.

[0160] For example, take M (ie, M=4) usage records as Figure 4 Take the 4 usage records included in the time bucket 1 as an example. Time bucket 1 corresponds to a time period (i.e., 21:41:00-21:50:00). In the time period corresponding to time bucket 1, the 4 usage records include L events, namely, "chat", "community", and "note intention" (i.e., L=3). In the time period corresponding to time bucket 1, the 4 usage records include 2 usage records of "community". Since the number of usage records of other events (i.e., "chat" or "note intention") included in the time period corresponding to time bucket 1 is 1, when M usage records are Figure 4 When the time bucket 1 shown includes 4 usage records, the M usage records only include one first event "community" (ie, S1=1).

[0161] In another example, when S1 is a positive integer greater than 1, the L events included in the M usage records include two or more first events, all usage records for each first event are within at least one time period, and each time period of at least a portion of the at least one time period corresponds to at least two usage records for the first event. In other words, when two usage records for an event are included in a time period, the event can be referred to as a first event.

[0162] For example, M (ie, M=11) usage records include Figure 4 Taking the example of time bucket 1 including 4 usage records and time bucket 2 including 7 usage records, based on the description of the first event mentioned above, it can be seen that the L events included in these 11 usage records are "chat", "community", "note intention", and "office software". Since the time period corresponding to time bucket 1 includes 2 usage records of community, and the time period corresponding to time bucket 2 includes 2 usage records of community, community is a first event; since the time period corresponding to time bucket 2 includes 2 usage records of office software, office software is a first event; since the time period corresponding to time bucket 1 and the time period corresponding to time bucket 2 each include only 1 usage record of chat, chat is not a first event; since the time period corresponding to time bucket 2 includes only 1 usage record of browser, browser is not a first event. In summary, these 11 usage records include 2 first events (i.e., S1 = 2), namely, "community" and "office software".

[0163] It should be noted that the M usage records, L events and S1 first events mentioned above are only for illustration and do not constitute any limitation on the M usage records, L events and S1 first events, nor do they constitute any limitation on the specific values ​​of M, L and S1.

[0164] S720, the electronic device deletes, for each first event, the remaining usage records other than the first usage record in the at least two usage records of the first event corresponding to each time period of at least part of the time period, to obtain a deduplication result, wherein the first usage record is the usage record with the earliest usage time in the at least two usage records of the first event corresponding to each time period of at least part of the time period, and the deduplication result includes the usage records in the M usage records except the remaining usage records corresponding to each time period of at least part of the time period for each first event.

[0165] In the above step S720, for each first event, when the electronic device includes multiple usage records of each first event in each time period, only retains the usage record of each first event that appears for the first time in each time period (ie, the first usage record).

[0166] The first usage record is the usage record with the earliest usage time among the at least two usage records of the first event corresponding to each time period of at least a portion of the time period.

[0167] For example, take the above M (i.e. M is equal to 4) usage records as Figure 4 Taking the four usage records included in time bucket 1 as an example, when the first event is "community", the first usage record is the second usage record included in time bucket 2, that is, "123, package name B, community, 2023-06-02 21:43:31.693"; the remaining usage records other than the first usage record are the third usage record included in time bucket 1, that is, "123, package name B, community, 2023-06-02 21:48:24.231".

[0168] The following describes M usage records and deduplication results as an example.

[0169] For example, take the above M (ie M=4) usage records as Figure 4 Taking the four usage records included in the time bucket 1 as an example, after the electronic device executes the above step S720, the deduplication result can be seen in Figure 5 The deduplicated time bucket 1 includes three events (i.e., “chat,” “community,” and “note intention”) and three usage records corresponding to them.

[0170] For example, M (ie, M is equal to 11) usage records include Figure 4Taking the example of 4 usage records included in time bucket 1 and 7 usage records included in time bucket 2, after the electronic device executes the above step S720, the deduplication results obtained include Figure 5 The deduplicated time bucket 1 shows 3 usage records corresponding to 3 events (i.e., "chat", "community" and "note intention"), and the deduplicated time bucket 2 shows 5 usage records corresponding to 5 events (i.e., "office software", "note intention", "browser", "community" and "chat").

[0171] For example, M usage records include Figure 4 As an example, after the electronic device executes the above step S720, the deduplication results obtained include: Figure 5 The illustrated time buckets 1 to 6 after deduplication processing include all usage records.

[0172] For example, an example of the deduplication method described in steps S710 and S720 can be found in the above Figure 3 The deduplication method provided in steps S310 to S330 of the provided method, specifically, the user business data 1 in the above step S310 is an example of the above data to be deduplicated, the multiple usage records 1 in the above step S310 are an example of the above M usage records, the multiple usage buckets in the above step S320 are a specific example of the above at least one time period, the data deduplication strategy 0 in the above step S330 is an example of the deduplication strategy described in the above step S720, the initial deduplication result in the above S330 is an example of the above deduplication result, and the content not described in detail here can be found in the relevant descriptions in steps S310 to S330 above.

[0173] Based on the above-mentioned time-based deduplication method, the electronic device performs deduplication processing on the M usage records included in the deduplication data, which can avoid the problem of missed deduplication that exists when traditional technologies perform data deduplication based on time intervals on chain data (i.e., data obtained when the same event occurs continuously and is interrupted by other events). In summary, the data deduplication method provided by this application can improve the accuracy of the data deduplication results, thereby improving the quality of the deduplicated data.

[0174] In actual applications, in some scenarios, some specific events have obvious correlations. For example, electronic device disconnection events and electronic device connection events. Another example is electronic device charging events and electronic device unplugging events. Based on this, if there is only a single event (such as charging) in the data, and the corresponding event (such as unplugging) that is correlated with the single event is missing, or there are continuous single events (such as charging) in the data, there may be data omission or data miscollection, and the erroneous data needs to be deduplicated. In other scenarios, some events recorded in the data have obvious logical errors with subsequent events (for example, after the electronic device is unplugged, the power of the electronic device increases instead), then the data with logical errors needs to be deduplicated. Therefore, in an embodiment of the present application, after the electronic device executes the data deduplication method based on time features described in steps S710 and S720 above to obtain a deduplication result, the electronic device can also perform further deduplication on the deduplication result based on event correlation, that is, the deduplication method shown in implementation methods one to five below. Below, implementation methods one to five are introduced respectively.

[0175] Implementation method 1:

[0176] In implementation method one, the L events in the deduplication result include S2 second events, wherein the deduplication result specifically includes multiple usage records of each second event, and S2 is a positive integer; and, for each first event, in the at least two usage records of the first event corresponding to each time period of at least part of the time period, the remaining usage records other than the first usage record are deleted to obtain the deduplication result, and the electronic device further performs the following steps: when the interval between the usage times corresponding to at least two usage records in the multiple usage records of each second event does not exceed the first threshold, the usage record with the latest usage time in the at least two usage records of each second event is deleted.

[0177] The deduplication result is the result obtained by performing the above deduplication processing on the M usage records of the deduplication data, that is, the L events in the deduplication result are the same as the L events in the M usage records, but the number of usage records corresponding to the L events in the deduplication result is different from the number of M usage records of the data to be deduplicated.

[0178] The multiple usage records of each second event in the S2 events are located in multiple time periods, and the mapping relationship between the multiple usage records and the multiple time periods is not specifically limited.

[0179] For example, multiple usage records correspond to multiple time periods in a one-to-one manner, i.e., each of the multiple time periods corresponds to one of the multiple usage records, and each usage record is a usage record for the corresponding time period, i.e., the usage time corresponding to each usage record falls within the corresponding time period. For example, multiple usage records correspond to multiple time periods in a one-to-one manner, i.e., each of the multiple time periods corresponds to at least one of the multiple usage records, and at least one of the multiple time periods corresponds to a portion of the multiple usage records.

[0180] The L events in the deduplication result include S2 second events, and each of the S2 second events may be the same as or different from each of the above S1 first events.

[0181] The value of the first threshold is not specifically limited and can be set according to actual needs. For example, the first threshold can be, but is not limited to, 2 seconds or 3 seconds.

[0182] In this application, "exceed" means "greater than", and "not exceed" means "less than" or "equal to". The meaning of "exceed" or "not exceed" appearing below is the same as here and will not be repeated below.

[0183] For example, an example of the deduplication method described in the above implementation method 1 can be found in the above Figure 3 The data deduplication strategy 1 described in step S340 of the provided method, wherein the usage record 1 included in the multiple deduplicated time buckets in the above step S340 is an example of the deduplication result in the above implementation method one, and the preset time interval 2 in the above step S340 is an example of the first threshold in the above implementation method one. For matters not described in detail here, please refer to the relevant description in the above step S340.

[0184] In the above-mentioned first implementation method, after performing data deduplication on each of the S1 first events in the M usage records included in the data to be deduplicated, the deduplication results obtained include multiple usage records of each second event located in multiple time periods. Based on this, each second event can be deduplicated again for short-term repeated behavioral data, that is, among the at least two usage records of the multiple usage records of each second event, the usage record with the latest usage time is deleted from the at least two usage records of each second event. This method can remove some events of the same type that happen to be distributed at the edges of adjacent time periods, minimize the data noise of the deduplication results obtained after the second deduplication, improve the accuracy of the data deduplication results, and thus improve the quality of the data after deduplication.

[0185] Implementation method 2:

[0186] In implementation method two, the L events in the deduplication result include S3 third events, wherein each third event corresponds to an associated event, the multiple usage records of each third event are continuous, the multiple usage records of each third event are adjacent to the usage records of the corresponding associated event, and S3 is a positive integer; and, for each first event, in the at least two usage records of the first event corresponding to each time period of at least part of the time period, the remaining usage records other than the first usage record are deleted to obtain the deduplication result, and the electronic device also performs the following steps: when the duration between the usage time of the usage record of the associated event corresponding to each third event and the usage time corresponding to the multiple usage records of each third event does not exceed the first preset duration, the usage records other than the usage record with the earliest usage time are deleted from the multiple usage records of each third event.

[0187] The multiple usage records of each third event are located in multiple time periods, and the mapping relationship between the multiple usage records and the multiple time periods is not specifically limited. For example, the multiple usage records of each third event correspond to the multiple time periods in a one-to-one manner.

[0188] The multiple usage records of each third event being continuous means that the multiple usage records corresponding to each third event are recorded continuously in all usage records included in the deduplication result.

[0189] For example, the usage records of L events included in the above deduplication result can be as follows Figure 8 For example, the three usage records in the table shown in (1) are Figure 8 The two usage records of the "unplug" event shown in (1) are continuous.

[0190] The third event corresponds to an associated event, wherein two associated events may be two events that are relatively connected or clearly correlated, and there is no specific limitation on the two associated events. For example, the two associated events may include a "plug in" event and a "unplug" event. For example, the two associated events may include a "device connected" event and a "device disconnected" event.

[0191] The deduplication result includes usage records of the associated events corresponding to each third event and the multiple consecutive usage records of each third event that are adjacent. In other words, the usage records of the associated events corresponding to each third event and the multiple consecutive usage records of each third event are consecutive usage records. The usage time of the usage record of the associated event corresponding to each third event can be earlier or later than the usage time of the earliest usage record in the multiple consecutive usage records of each third event.

[0192] For example, the usage records of L events included in the above deduplication result can be as follows Figure 8 For example, the three usage records in the table shown in (1) are Figure 8 The two usage records shown in (1) of the plug-in event and the unplug-out event are adjacent.

[0193] For example, the usage records of the L events included in the above deduplication result can be as follows: Figure 8 The three usage records recorded in the table shown in (1) correspond to two (i.e., L=2) events (i.e., "plug in" and "unplug"). Based on this, the electronic device performs deduplication processing on the three usage records according to the second implementation method above, that is, it is necessary to delete the usage records of the other two consecutive unplugging events except the first one (i.e., the usage records of the second unplugging event). The result after deletion is as follows: Figure 8 (2) in the table shows two usage records.

[0194] Each third event in the above-mentioned S3 third events, each second event in the above-mentioned S2 second events, and each first event in the above-mentioned S1 first events may be the same or different, and this is not specifically limited in the embodiments of the present application.

[0195] For example, each third event of the S3 third events included in the L events in the above deduplication result, each second event of the S2 second events included in the L events in the above deduplication result, and each first event of the S1 first events included in the L events in the above deduplication result are the same event.

[0196] For example, each third event of the S3 third events included in the L events in the above deduplication result, each second event of the S2 second events included in the L events in the above deduplication result, and each first event of the S1 first events included in the L events in the above deduplication result are three different events.

[0197] In the above technical solution, after performing data deduplication based on the time feature (i.e., the time period above) for each of the S1 first events in the M usage records included in the data to be deduplicated, the deduplication result obtained includes each third event corresponding to an associated event. Thereafter, deduplication is performed based on the correlation between events, that is, if the duration between the usage time of the usage record of the associated event corresponding to each third event and the usage time corresponding to the multiple usage records of each third event is shorter, then the usage records except for the usage record with the earliest usage time in the multiple usage records of each third event are deleted. This method deduplicates data from two dimensions: time feature and correlation between events. This method has high versatility and robustness, and can improve the accuracy of data deduplication results, thereby improving the quality of data after deduplication.

[0198] Implementation method three:

[0199] In implementation method three, the L events in the deduplication result include S4 fourth events and there is no associated event corresponding to each fourth event, S4 is a positive integer; and, for each first event, in at least two usage records of the first event corresponding to each time period of at least part of the time period, the remaining usage records other than the first usage record are deleted to obtain the deduplication result. The electronic device also performs the following steps: when the usage time corresponding to all usage records of each fourth event exceeds the second preset time, all usage records of each fourth event are deleted.

[0200] The second preset duration is a shorter time period, and the length of the second preset duration is not specifically limited and can be set according to actual needs. For example, the length of the second preset duration can be, but is not limited to, 20 minutes, 30 minutes, 1 hour, or 2 hours.

[0201] The second preset time length is shorter than the first preset time length. For example, the first preset time length may be 24 hours, and the second preset time length may be 2 hours.

[0202] For example, the usage records of the L events included in the above deduplication result can be as follows: Figure 9 The 6 usage records recorded in the table (1) are related events in the 3rd and 4th usage records. The event in the 6th usage record (i.e., "unplug") should have a corresponding related event (i.e., "plug in"), but Figure 9 There is no related event in the 6 usage records shown in (1) of FIG. 1 . Therefore, the data deduplication method described in the third implementation method above needs to delete the 6th usage record. The result after deletion is as follows: Figure 9 (2) in FIG. 5 shows the five usage records recorded in the table.

[0203] Each fourth event in the above-mentioned S4 fourth events, each third event in the above-mentioned S3 third events, each second event in the above-mentioned S2 second events, and each first event in the above-mentioned S1 first events may be the same or different, and no specific limitation is made to this in the embodiments of the present application.

[0204] In the technical solution, after performing data deduplication on each of the S1 first events in the M usage records included in the to-be-deduplicated data based on the time feature (i.e., the time period), each fourth event included in the deduplication result corresponds to an associated event, and the deduplication result does not include the associated event corresponding to each fourth event. Then, deduplication is performed based on the correlation between events, that is, if the usage duration corresponding to all usage records of each fourth event is large, all usage records of each fourth event are deleted. That is, the method performs deduplication on data from two dimensions of the time feature and the correlation between events. The method has high universality and robustness, can improve the accuracy of the deduplication result, and thus improves the quality of the deduplicated data.

[0205] Implementation four

[0206] In the implementation four, after obtaining the deduplication result by deleting, for each first event, the remaining usage records except the first usage record in the at least two usage records of the first event corresponding to each time period of the at least partial time period, the electronic device further performs the following steps: in a case where the multiple usage records corresponding to one event included in the deduplication result have logical errors, deleting at least one usage record having a logical error in the multiple usage records corresponding to the one event; or in a case where at least partial usage records of one event and usage records of another event included in the deduplication result have logical errors, deleting the usage records having logical errors in the at least partial usage records of the one event and the usage records of the another event.

[0207] The data deduplication method described in the implementation four can delete the usage record having a logical error in the multiple usage records (e.g., the usage record of one event or the multiple usage records of two associated events) included in the deduplication result.

[0208] The one usage record and the another usage record having a logical error can be that the event result of the event recorded in the one usage record and the event result of the event of the another usage record do not conform to the conventional logic.

[0209] The usage time of the one usage record having a logical error is earlier or later than the usage time of the another usage record. For example, when the usage time of the one usage record is earlier than the usage time of the another usage record, the event of the one usage record can be referred to as a preceding event, and the event of the another usage record can be referred to as a subsequent event.

[0210] In one example, in a case where the deduplication result includes a plurality of usage records of one event and a logical error exists in the plurality of usage records, at least one usage record with a logical error is deleted from the plurality of usage records of one event.

[0211] In the above implementation, in the plurality of usage records of one event, when the number of usage records with a logical error is one, the one usage record is deleted; and when the number of usage records with a logical error is a plurality, the plurality of usage records are deleted.

[0212] For example, the usage records of L events included in the deduplication result can be 4 usage records shown in (1) of Figure 10 , the first usage record includes a usage time 1 of "2023-08-25 21:26:11", the event is "power off", and the result of the event is "94%"; the second usage record includes a usage time 2 of "2023-08-25 21:57:36", the event is "power off", and the result of the event is "100%". The event of the first usage record can be the preceding event described above, and the event of the second usage record can be the subsequent event described above. According to normal logic, when the electronic device is in the power-off state, the power of the electronic device remains unchanged or decreases (for example, the user uses the electronic device to cause the power to decrease). However, based on the power corresponding to the first usage record shown in (1) of Figure 10 and the power corresponding to the second usage record, the power of the electronic device increases from 90% to 95% when the electronic device is in the power-off state, which is contrary to the normal logic and does not conform to the actual situation, so the second usage record needs to be deleted, or the first usage record and the second usage record are deleted at the same time. Figure 10 The deduplication result obtained by performing deduplication processing on the 4 usage records shown in (1) of Figure 10 may refer to the 3 usage records shown in (2) of

[0213] In another example, in a case where the deduplication result includes two associated events, and a logical error exists in at least part of the usage records of one event and the usage records of the other event, the usage record with a logical error is deleted from at least part of the usage records of one event and the usage records of the other event.

[0214] In the above implementation, the number of usage records with a logical error in at least part of the usage records of one event and the usage records of the other event can be one or a plurality, which is not specifically limited.

[0215] In the above implementation, the deduplication result may include one or more usage records of one of the two associated events, and at least part of the usage records of the one event may be one or more usage records of the one event.

[0216] For example, the two associated events included in the deduplication result may be a "charging" event and a "unplugging" event. The usage record 1 of the two associated events included in the deduplication result is "2024-03-07 10:43:53, charging, 90%"; the usage record 2 is "2024-03-07 11:43:53, unplugging, 70%". Based on the deduplication result, it can be seen that after the device is charged first and then unplugged, the power of the device drops from 90% to 70%. According to normal logic, when the device is charged first and then unplugged, the power of the device will not drop. Therefore, according to the deduplication method described in the above implementation method, when performing data deduplication on the deduplication result, it is necessary to delete the usage record 2 with the logical error.

[0217] In the above technical solution, after data deduplication is performed based on the time feature (i.e., the time period above) for each of the S1 first events in the M usage records included in the data to be deduplicated, when the deduplication results include multiple usage records corresponding to one event or two events and there are logical errors, the usage records with logical errors will be deleted. This method performs deduplication on data from two dimensions, namely, time features and the logic of related events. This method has high versatility and robustness, and can improve the accuracy of the data deduplication results, thereby improving the quality of the data after deduplication.

[0218] Implementation method five:

[0219] In actual applications, when an electronic device executes a target event, it will execute multiple events (for example, music application opening event and audio event) associated with the target event (for example, music playback event) within a relatively short period of time. At the same time, the electronic device will also record information on the execution of multiple events associated with the target event. In this case, accurate judgment cannot be made based on only a single event (for example, music application opening event or audio event) among the multiple events associated with a target event. That is to say, in this scenario, it is necessary to combine multiple events associated with a target event. Only when the multiple events meet specific conditions can the multiple usage records corresponding to the multiple events be considered valid. At this time, there is no need to deduplicate the usage records including the multiple events. When the multiple events do not meet specific conditions, the multiple usage records corresponding to the multiple events are considered invalid. At this time, it is necessary to delete the usage records including the multiple events.

[0220] The above-mentioned specific condition can be that the interval between the multiple occurrence times corresponding to multiple events associated with a certain target event is less than a preset threshold. That is, in the embodiment of the present application, only when multiple events associated with a certain target event occur simultaneously within a short time interval, it is determined that the multiple events are real events. At this time, the multiple events are not deleted. When only one event is triggered among the multiple events associated with a certain target event within a short time interval, the multiple events need to be deleted. There is no specific limitation on the short time interval. For example, the short time interval can be, but is not limited to, 1 minute.

[0221] Based on this, in the above-mentioned implementation method five, the deduplication result includes at least one usage record of the fifth event in the two associated events; and, for each first event, in the at least two usage records of the first event corresponding to each time period of at least part of the time period, the remaining usage records other than the first usage record are deleted to obtain the deduplication result. The electronic device can also perform the following steps: when at least one usage record of the sixth event in the two associated events is not detected, delete at least one usage record of the fifth event, wherein the two associated events trigger the electronic device to execute the target event; or, when at least one usage record of the sixth event is detected and the minimum interval between the usage time of at least one usage record of the fifth event and the usage time of at least one usage record of the sixth event exceeds a second threshold, delete the usage record of the fifth event associated with the minimum interval; when at least one usage record of the sixth event is detected and the minimum interval between the usage time of at least one usage record of the fifth event and the usage time of at least one usage record of the sixth event does not exceed the second threshold, do not delete the usage record of the fifth event associated with the minimum interval.

[0222] The minimum interval is a difference in usage time between the usage record of the fifth event and at least one usage record of the sixth event associated with the minimum interval.

[0223] There is a minimum interval between the usage time of at least one usage record of the one event and the usage time of at least one usage record of the other event.

[0224] For example, at least one usage record of another event includes usage record A, and at least one usage record of an event is usage record B, where the usage time of usage record A is 10:00 and the usage time of usage record B is 10:02. Then the minimum interval between the usage time of a usage record of the event and the usage time of at least one usage record of the other event is 2 minutes.

[0225] For example, at least one usage record of another event includes usage record 1 and usage record 2, and at least one usage record of an event is usage record 3, wherein usage time 1 of usage record 1 is 10:00, usage time 1 of usage record 2 is 10:05, and usage time 1 of usage record 3 is 9:55. Then the minimum interval between the usage time of a usage record of the one event and the usage time of at least one usage record of the other event is 5 minutes (that is, the interval between usage time 1 of usage record 1 and usage time 1 of usage record 3 is 9:55).

[0226] There is no specific limitation on the implementation method of whether the electronic device in the above steps detects a usage record of the other event in the two associated events. For example, the electronic device detects the deduplication result, and when it is determined that the deduplication result does not include a usage record of the other event in the two associated events, determines that a usage record of the other event in the two associated events is not detected. For example, the electronic device detects the data stored in itself (i.e., including the deduplication result and the data excluding the deduplication result), and when it is determined that the data stored in itself does not include a usage record of the other event in the two associated events, determines that a usage record of the other event in the two associated events is not detected.

[0227] There is no specific limitation on the value of the second threshold, which can be set according to actual needs. For example, the second threshold can be, but is not limited to, 5 seconds or 1 minute.

[0228] For example, the above deduplication result is Figure 11 The two usage records of the music application opening event (ie, music intention) shown in (1) are taken as an example, and the second threshold is 1 minute. The electronic device also stores Figure 11 In this case, the deduplication method based on the above implementation method 5 is used to record the two usage records of the audio event shown in (2). Figure 11 When deduplication is performed on the two usage records shown in (1), due to Figure 11 The usage time of record 1 shown in (1) Figure 11 The interval of the usage time of record #1 shown in (2) is the smallest, and the interval is 20 seconds, so it can be considered that record 1 is valid data and does not need to be deleted; Figure 11 The usage time of record 2 shown in (1) Figure 11 The interval of the usage time of record #2 shown in (2) is the smallest, and the interval is 30 minutes, so it can be considered that record 2 is invalid data and needs to be deleted. In summary, based on the above implementation method 5, Figure 11 The usage records shown in (1) are deduplicated, and the results are as follows Figure 111 usage record shown in (3) in Figure 11 The record 1 shown in (1) is shown in (1).

[0229] In the above technical solution, after performing data deduplication based on the time feature (i.e., the time period mentioned above) for each of the S1 first events in the M usage records included in the data to be deduplicated, deduplication is performed based on the correlation between events, that is, if the deduplication result only includes the usage record of one of the two related events, or the interval between the usage times of the two related events is long, it is necessary to delete the usage record of one of the two related events, or the usage times of the two related events. In other words, this method performs deduplication on data from the two dimensions of time feature and correlation between events. This method has high versatility and robustness, and can improve the accuracy of the data deduplication results, thereby improving the quality of the data after deduplication.

[0230] It should be noted that the above description is based on the example that any one of the above implementation methods 1 to 5 is performed by the electronic device after executing the above steps S710 and S720. Optionally, in other implementation methods, the electronic device can also directly execute any one of the above implementation methods 1 to 5 for deduplicated data (for example, the M usage records above).

[0231] It should be understood that the above Figure 7 The data deduplication method shown is for illustrative purposes only and does not limit the data deduplication method provided herein. For example, an electronic device may only perform S710 and S720 above; an electronic device may only perform one of Implementations 1 to 5 above; or an electronic device may perform multiple of Implementations 1 to 5 above.

[0232] Combined with the above Figures 3 to 11 , describes in detail the application scenario and data deduplication method of the data deduplication method of the embodiment of the present application, and the following will be combined with Figure 12 , describing the device embodiments of the present application in detail. It should be understood that the data deduplication device in the embodiments of the present application can execute the various data deduplication methods in the aforementioned embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the aforementioned method embodiments.

[0233] Figure 12 Schematic diagram of a data deduplication device provided in an embodiment of the present application. Figure 12 The data deduplication apparatus 1200 shown includes a processing unit 1210 .

[0234] In one example, the processing unit 1210 is configured to:

[0235] Obtaining data to be deduplicated, wherein the data to be deduplicated includes M usage records, the M usage records include usage records of L events, the L events include S1 first events, all usage records of each first event are in at least one time period, each time period of at least some time periods within the at least one time period corresponds to at least two usage records of the first event, M is an integer greater than 1, and L and S1 are positive integers;

[0236] For each of the first events, in the at least two usage records of the first event corresponding to each time period of the at least partial time period, the remaining usage records other than the first usage record are deleted to obtain a deduplication result, wherein the first usage record is the usage record with the earliest usage time among the at least two usage records of the first event corresponding to each time period of the at least partial time period, and the deduplication result includes the usage records in the M usage records except the remaining usage records corresponding to each time period of the at least partial time period for each of the first events.

[0237] In one possible implementation, the L events in the deduplication result include S2 second events, wherein the deduplication result specifically includes multiple usage records of each second event, and S2 is a positive integer; and the processing unit 1210 is further configured to: after deleting the remaining usage records other than the first usage record from the at least two usage records of the first event corresponding to each time period of the at least partial time period for each first event to obtain the deduplication result, perform the following steps:

[0238] When an interval between usage times corresponding to at least two usage records in the plurality of usage records of each second event does not exceed a first threshold, the usage record with the latest usage time in the at least two usage records of each second event is deleted.

[0239] In another possible implementation, the L events in the deduplication result include S3 third events, wherein each of the third events corresponds to an associated event, the multiple usage records of each of the third events are continuous, the multiple usage records of each of the third events are adjacent to the usage records of the corresponding associated event, and S3 is a positive integer; and the processing unit 1210 is further used to: after deleting the remaining usage records other than the first usage record from the at least two usage records of the first event corresponding to each time period of the at least partial time period for each of the first events to obtain a deduplication result, perform the following steps:

[0240] When the duration between the usage time of the usage record of the associated event corresponding to each of the third events and the usage time corresponding to multiple usage records of each of the third events does not exceed a first preset duration, delete the usage records except the usage record with the earliest usage time in the multiple usage records of each of the third events.

[0241] In another possible implementation, the L events in the deduplication result include S4 fourth events and there is no associated event corresponding to each of the fourth events, where S4 is a positive integer; and the processing unit 1210 is further configured to: after deleting the remaining usage records other than the first usage record from the at least two usage records of the first event corresponding to each time period of the at least partial time period for each of the first events to obtain a deduplication result, perform the following steps:

[0242] In a case where the usage duration corresponding to all usage records of each of the fourth events exceeds a second preset duration, all usage records of each of the fourth events are deleted.

[0243] In another possible implementation, the processing unit 1210 is further configured to: after deleting the remaining usage records other than the first usage record from the at least two usage records of the first event corresponding to each time period of the at least partial time period for each first event to obtain a deduplication result, perform the following steps:

[0244] If there are logical errors in the multiple usage records corresponding to an event included in the deduplication result, deleting at least one usage record containing the logical error among the multiple usage records corresponding to the event; or

[0245] If there are logical errors in at least part of the usage records of one event and the usage records of another event in the two associated events included in the deduplication result, delete the usage records with logical errors in at least part of the usage records of one event and the usage records of the other event.

[0246] In another possible implementation, the deduplication result includes at least one usage record of a fifth event in the two associated events; and the processing unit 1210 is further configured to:

[0247] After deleting the remaining usage records other than the first usage record from the at least two usage records of the first event corresponding to each time period of the at least partial time period for each of the first events to obtain a deduplication result, performing the following steps:

[0248] In the case where at least one usage record of the sixth event in the two associated events is not detected, deleting at least one usage record of the fifth event; or,

[0249] When at least one usage record of the sixth event is detected and the minimum interval between the usage time of at least one usage record of the fifth event and the usage time of at least one usage record of the sixth event exceeds a second threshold, the usage record of the fifth event associated with the minimum interval is deleted.

[0250] In another example, the processing unit 1210 is configured to:

[0251] Obtaining data to be deduplicated, wherein M usage records in the data to be deduplicated include S events, each event corresponds to an associated event, multiple usage records of each event are continuous, and multiple usage records of each event are adjacent to corresponding usage records of the associated event, M is an integer greater than 2, and S is a positive integer;

[0252] When the duration between the usage time of the usage record of the associated event corresponding to each of the events and the usage time corresponding to the multiple usage records of each of the events does not exceed the preset duration, the usage records except the usage record with the earliest usage time in the multiple usage records of each of the events are deleted.

[0253] In yet another example, the processing unit 1210 is configured to:

[0254] Obtaining data to be deduplicated, wherein M usage records in the data to be deduplicated include S events and there is no associated event corresponding to each of the events, each of the events corresponds to an associated event, M is an integer greater than 1, and S is a positive integer;

[0255] When the usage duration corresponding to all usage records of each of the events exceeds a preset duration, all usage records of each of the events are deleted.

[0256] In yet another example, the processing unit 1210 is configured to:

[0257] Acquire data to be deduplicated, wherein the data to be deduplicated includes M usage records, where M is an integer greater than 1;

[0258] In the case where a plurality of usage records corresponding to one event included in the M usage records have logical errors, deleting at least one usage record having the logical error among the plurality of usage records corresponding to the one event; or

[0259] In the event that there are logical errors in at least part of the usage record of one event and the usage record of another event in the two associated events included in the M usage records, the usage record with the logical errors in at least part of the usage record of one event and the usage record of the other event are deleted.

[0260] In yet another example, the processing unit 1210 is configured to:

[0261] Acquire data to be deduplicated, wherein the data to be deduplicated includes M usage records, the M usage records include at least one usage record of a fifth event in two associated events, and M is a positive integer;

[0262] In the case where at least one usage record of the sixth event in the two associated events is not detected, deleting at least one usage record of the fifth event; or,

[0263] When at least one usage record of the sixth event is detected and the minimum interval between the usage time of at least one usage record of the fifth event and the usage time of at least one usage record of the sixth event exceeds a second threshold, the usage record of the fifth event associated with the minimum interval is deleted.

[0264] It should be noted that the data deduplication device 1200 is implemented in the form of a functional unit. The term "unit" here can be implemented in the form of software and / or hardware, and is not specifically limited to this.

[0265] For example, a "unit" may be a software program, a hardware circuit, or a combination of the two that implements the aforementioned functionality. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (e.g., a shared processor, a dedicated processor, or a group processor) and memory for executing one or more software or firmware programs, combined logic circuits, and / or other suitable components that support the described functionality.

[0266] Therefore, the units of each example described in the embodiments of this application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0267] The present application also provides a computer program product, which, when executed by a processor, implements the data deduplication method described in any method embodiment of the present application.

[0268] The computer program product may be stored in a memory, for example, a program, which is converted into an executable target file that can be executed by a processor after undergoing processes such as preprocessing, compilation, assembly, and linking.

[0269] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer, implements the data deduplication method described in any method embodiment of the present application. The computer program can be a high-level language program or an executable target program.

[0270] In this application, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0271] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0272] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0273] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0274] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for example, the division of the units is merely a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection of some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0275] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0276] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0277] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A data deduplication method, characterized in that: The method comprises: Obtaining data to be deduplicated, wherein the data to be deduplicated includes M usage records, the M usage records include usage records of L events, the L events include S1 first events, all usage records of each first event are in at least one time period, each time period of at least some time periods within the at least one time period corresponds to at least two usage records of the first event, M is an integer greater than 1, and L and S1 are positive integers; For each of the first events, in the at least two usage records of the first event corresponding to each time period of the at least partial time period, the remaining usage records other than the first usage record are deleted to obtain a deduplication result, wherein the first usage record is the usage record with the earliest usage time among the at least two usage records of the first event corresponding to each time period of the at least partial time period, and the deduplication result includes the usage records in the M usage records except the remaining usage records corresponding to each time period of the at least partial time period for each of the first events.

2. The method according to claim 1, characterized in that The L events in the deduplication result include S2 second events, wherein the deduplication result specifically includes multiple usage records of each second event, where S2 is a positive integer; and after, for each first event, deleting remaining usage records other than the first usage record from the at least two usage records of the first event corresponding to each time period of the at least partial time period to obtain a deduplication result, the method further includes: When an interval between usage times corresponding to at least two usage records in the plurality of usage records of each second event does not exceed a first threshold, the usage record with the latest usage time in the at least two usage records of each second event is deleted.

3. The method according to claim 1 or 2, characterized in that The L events in the deduplication result include S3 third events, wherein each of the third events corresponds to an associated event, multiple usage records of each of the third events are continuous, and multiple usage records of each of the third events are adjacent to usage records of the corresponding associated event, and S3 is a positive integer; and after, for each of the first events, deleting remaining usage records other than the first usage record from the at least two usage records of the first event corresponding to each time period of the at least partial time period to obtain a deduplication result, the method further includes: When the duration between the usage time of the usage record of the associated event corresponding to each of the third events and the usage time corresponding to multiple usage records of each of the third events does not exceed a first preset duration, delete the usage records except the usage record with the earliest usage time in the multiple usage records of each of the third events.

4. The method according to claim 1 or 2, characterized in that The L events in the deduplication result include S4 fourth events and there is no associated event corresponding to each of the fourth events, where S4 is a positive integer; Furthermore, after deleting the remaining usage records other than the first usage record from the at least two usage records of the first event corresponding to each time period of the at least partial time period for each of the first events to obtain a deduplication result, the method further includes: In a case where the usage duration corresponding to all usage records of each of the fourth events exceeds a second preset duration, all usage records of each of the fourth events are deleted.

5. The method according to claim 1 or 2, characterized in that After deleting, for each of the first events, the remaining usage records other than the first usage record from the at least two usage records of the first event corresponding to each time period of the at least partial time period to obtain a deduplication result, the method further includes: If there are logical errors in the multiple usage records corresponding to an event included in the deduplication result, deleting at least one usage record with the logical error among the multiple usage records corresponding to the event; or If there are logical errors in at least part of the usage records of one event and the usage records of another event in the two associated events included in the deduplication result, delete the usage records with logical errors in at least part of the usage records of one event and the usage records of the other event.

6. The method according to claim 1 or 2, characterized in that The deduplication result includes at least one usage record of a fifth event in the two associated events; and after, for each of the first events, deleting remaining usage records other than the first usage record from the at least two usage records of the first event corresponding to each time period of the at least partial time period to obtain a deduplication result, the method further includes: In the case where at least one usage record of the sixth event in the two associated events is not detected, deleting at least one usage record of the fifth event; or, When at least one usage record of the sixth event is detected and the minimum interval between the usage time of at least one usage record of the fifth event and the usage time of at least one usage record of the sixth event exceeds a second threshold, the usage record of the fifth event associated with the minimum interval is deleted.

7. A data deduplication method, characterized in that: The method comprises: Obtaining data to be deduplicated, wherein M usage records in the data to be deduplicated include S events, each event corresponds to an associated event, multiple usage records of each event are continuous, and multiple usage records of each event are adjacent to corresponding usage records of the associated event, M is an integer greater than 2, and S is a positive integer; When the duration between the usage time of the usage record of the associated event corresponding to each of the events and the usage time corresponding to the multiple usage records of each of the events does not exceed the preset duration, the usage records except the usage record with the earliest usage time in the multiple usage records of each of the events are deleted.

8. A data deduplication method, characterized in that: The method comprises: Obtaining data to be deduplicated, wherein M usage records in the data to be deduplicated include S events and there is no associated event corresponding to each of the events, each of the events corresponds to an associated event, M is an integer greater than 1, and S is a positive integer; When the usage duration corresponding to all usage records of each of the events exceeds a preset duration, all usage records of each of the events are deleted.

9. A data deduplication method, characterized in that: The method comprises: Acquire data to be deduplicated, wherein the data to be deduplicated includes M usage records, where M is an integer greater than 1; In the case where a plurality of usage records corresponding to one event included in the M usage records have logical errors, deleting at least one usage record having the logical error among the plurality of usage records corresponding to the one event; or In the event that there are logical errors in at least part of the usage record of one event and the usage record of another event in the two associated events included in the M usage records, the usage record with the logical errors in at least part of the usage record of one event and the usage record of the other event are deleted.

10. An electronic device, characterized in that: The electronic device includes one or more processors and a memory; the memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method as described in any one of claims 1 to 6, 7, 8 or 9.

11. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the one or more processors are used to call computer instructions so that the electronic device executes the method as described in any one of claims 1 to 6, 7, 8 or 9.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium includes instructions, which, when executed on an electronic device, cause the electronic device to perform the method according to any one of claims 1 to 6, 7, 8 or 9.

Images