Data processing method, device, electronic device, program product and storage medium

Abstract

The present application provides a data processing method, device, electronic device, computer program product and computer-readable storage medium; applied to the field of maps, the method includes: obtaining a first set of behavior objects in a first time period, and obtaining behavior objects in a target time period, wherein the target time period is adjacent to the first time period and is located after the first time period; performing descending sorting processing on the behavior objects in the target time period based on object identifiers to obtain a first descending result; obtaining a first target behavior object whose object identifier is greater than all behavior objects in the first behavior object set from the first descending result; merging the first target behavior object sorted according to the first descending result into the first behavior object set to obtain a second behavior object set for a second time period. Through the present application, data processing efficiency can be improved.

Description

Technical Field

[0001] The present application relates to vehicle networking and big data computing technology, and in particular to a data processing method, device, electronic device, computer program product and computer-readable storage medium. Background Art

[0002] With the development of computer technology, the types and number of applications are increasing. Objects that use applications are called behavior objects. The access and browsing behavior of behavior objects will generate corresponding object behavior data. Computers can also obtain all object behavior data up to different time nodes, and process the obtained object behavior data in full to obtain a collection of behavior objects accumulated to different time nodes. The collection of behavior objects accumulated to different time nodes can be used to represent the access situation of applications in real time.

[0003] However, as time goes by, the amount of object behavior data will become larger and larger. When determining the set of behavior objects accumulated to different time nodes through the above method, the amount of object behavior data obtained by the computer will be very large, resulting in a long time for processing the full data, and full data processing needs to be performed for each different time node, which limits the data processing efficiency. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, device, electronic device, computer program product, and computer-readable storage medium, which can improve data processing efficiency and improve data processing resource utilization.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application provides a data processing method, including:

[0007] Acquire a first behavior object set in a first time period, and acquire behavior objects in a target time period, wherein the target time period is adjacent to the first time period and is located after the first time period;

[0008] Sorting the behavior objects within the target time period in descending order based on the object identifiers to obtain a first descending order result;

[0009] Acquire, from the first descending result, a first target behavior object whose object identifier is greater than that of all behavior objects in the first behavior object set;

[0010] The first target behavior objects sorted according to the first descending result are merged into the first behavior object set to obtain a second behavior object set for a second time period, wherein the second time period includes the first time period and the target time period.

[0011] The present application provides a data processing device, including:

[0012] an acquisition module, configured to acquire a first set of behavior objects in a first time period, and acquire behavior objects in a target time period, wherein the target time period is adjacent to the first time period and is located after the first time period;

[0013] A sorting module, used to sort the behavior objects within the target time period in descending order based on the object identifiers to obtain a first descending result;

[0014] A comparison module, configured to obtain, from the first descending result, a first target behavior object whose object identifier is greater than that of all behavior objects in the first behavior object set;

[0015] A merging module is used to merge the first target behavior objects sorted according to the first descending result into the first behavior object set to obtain a second behavior object set in a second time period, wherein the second time period includes the first time period and the target time period.

[0016] In the above scheme, when the first time period is an initial time period starting from an initial moment, the acquisition module is also used to: obtain the object behavior data at the initial moment of the first time period; perform object deduplication processing on the object behavior data at the initial moment to obtain the initial behavior object set at the initial moment; and determine the first behavior object set of the first time period based on the initial behavior object set at the initial moment.

[0017] In the above scheme, the acquisition module is also used to: perform descending sorting processing on the behavior objects in the initial behavior object set based on the object identifier to obtain the first maximum object identifier of the initial behavior object set; perform descending sorting processing on the behavior objects in the initial behavior object set based on the object identifier to obtain a second descending result; obtain the second target behavior object whose object identifier is greater than the first maximum object identifier from the second descending result; merge the second target behavior object into the initial behavior object set to obtain the first behavior object set of the first time period.

[0018] In the above scheme, the acquisition module is also used to: insert the second target behavior object into the last position of the descending sort result of the behavior objects in the initial behavior object set in the order represented by the second descending result, so as to obtain the first behavior object set.

[0019] In the above scheme, the acquisition module is also used to: perform the following processing on each behavior object in the second descending result in the order represented by the second descending result: compare the object identifier of the behavior object in the second descending result with the first maximum object identifier; when the comparison result represents that the object identifier of the behavior object in the second descending result is greater than the first maximum object identifier, determine the behavior object in the second descending result as the second target behavior object; when the comparison result represents that the object identifier of the behavior object in the second descending result is not greater than the first maximum object identifier, determine that the comparison processing will be stopped.

[0020] In the above scheme, the acquisition module is also used to: acquire the object behavior data within the target time period; perform object deduplication processing on the object behavior data within the target time period to obtain the behavior objects within the target time period.

[0021] In the above scheme, the comparison module is further used to: obtain the second largest object identifier of the first behavior object set; and obtain the first target behavior object whose object identifier is greater than the second largest object identifier from the first descending result.

[0022] In the above scheme, the comparison module is also used to: perform the following processing on each behavior object in the first descending result in the order represented by the first descending result: compare the object identifier of the behavior object in the first descending result with the second maximum object identifier; when the comparison result represents that the object identifier of the behavior object in the first descending result is greater than the second maximum object identifier, determine the behavior object in the first descending result as the first target behavior object; when the comparison result represents that the object identifier of the behavior object in the first descending result is not greater than the second maximum object identifier, determine that the comparison processing will be stopped.

[0023] In the above scheme, the device also includes: a cumulative statistical module, which is used to: obtain the cumulative visits and the cumulative number of objects in the first time period, and obtain the cumulative visits in the target time period; add the number of the first target behavior objects to the cumulative number of objects in the first time period to obtain the cumulative number of objects in the second time period; add the cumulative visits of the target time period to the cumulative visits of the first time period to obtain the cumulative visits of the second time period.

[0024] In the above scheme, when the first time period is an initial time period starting from an initial moment, the cumulative statistics module is also used to: obtain object behavior data at the initial moment of the first time period; perform object deduplication processing on the object behavior data at the initial moment to obtain an initial behavior object set at the initial moment; perform mapping and reduction processing on the behavior objects in the initial behavior object set to obtain an initial cumulative number of visits and an initial cumulative number of objects at the initial moment; determine the cumulative number of visits and the cumulative number of objects for the first time period based on the initial cumulative number of visits and the initial cumulative number of objects.

[0025] In the above scheme, the cumulative statistics module is also used to: perform descending sorting processing on the behavior objects in the initial behavior object set based on the object identifier to obtain the first maximum object identifier of the initial behavior object set; perform descending sorting processing on the behavior objects in the initial behavior object set based on the object identifier to obtain a second descending result; obtain the second target behavior object whose object identifier is greater than the first maximum object identifier from the second descending result; add the number of the second target behavior objects to the initial cumulative number of objects to obtain the cumulative number of objects in the first time period; add the cumulative number of visits in the first time period to the initial cumulative number of visits to obtain the cumulative number of visits in the first time period.

[0026] An embodiment of the present application provides an electronic device, including:

[0027] A memory for storing executable instructions;

[0028] The processor is used to implement the data processing method provided in the embodiment of the present application when executing the executable instructions stored in the memory.

[0029] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for implementing the data processing method provided in the embodiment of the present application when executed by a processor.

[0030] The embodiments of the present application have the following beneficial effects:

[0031] Through the embodiment of the present application, a first behavior object set of a first time period is obtained, and behavior objects within a target time period are obtained; the behavior objects within the target time period are sorted in descending order based on the object identifier to obtain a first descending result, and the first target behavior object whose object identifier is greater than that of all behavior objects in the first behavior object set is obtained from the first descending result, and the first target behavior object sorted according to the first descending result is merged into the first behavior object set to obtain a second behavior object set for a second time period, wherein the second time period includes the first time period and the target time period. When obtaining the second behavior object set for the second time period, it is no longer necessary to perform big data statistical processing on the entire data of the second time period, which effectively improves computing efficiency and saves computing resource usage. Since it is only necessary to sort the behavior objects within the target time period in descending order, the computing resource usage caused by the need to sort the entire behavior objects by the end of each time period is effectively reduced, thereby effectively improving computing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flowchart of mapping and reduction processing of object behavior data provided in the related art;

[0033] Figure 2 It is a flowchart of data statistics of object behavior data provided in the related art;

[0034] Figure 3 It is a schematic diagram of the architecture of the data processing system provided in the embodiment of the present application;

[0035] Figure 4 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0036] Figures 5A-5C It is a flowchart of a data processing method provided in an embodiment of the present application;

[0037] Figure 6 It is a flowchart of data statistics of object behavior data provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0039] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0040] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0042] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0043] 1) De-duplication statistics: For repeated access records of the same behavior object in the sequence, only one statistics is performed.

[0044] 2) Aggregation operation: Aggregation functions are used to perform calculations on a set of values ​​and return a single value. Aggregation functions ignore null values.

[0045] 3) Full table: stores all the latest status data every day.

[0046] 4) Incremental table: Incremental data is the new data after the last export. The incremental table records the data added each time, rather than the total amount of data. The incremental table only reports the change amount. If there is no change amount, there is no need to report the incremental table.

[0047] 5) Data skew: In a data set processed in parallel, the data in one part is significantly larger than that in other parts, making the processing speed of this part the bottleneck of the entire data set processing.

[0048] 6) Unique Visitor (UV): refers to the sum of the number of people who visit a product within a statistical period. The number of visitors needs to be deduplicated.

[0049] 7) Total Unique Visitors (TUV): refers to the sum of the number of people who visit a product during all statistical periods. The total unique visitor number needs to be deduplicated.

[0050] 8) Page View (PV): refers to the sum of the number of page views within a statistical period. The acquisition of page views does not require deduplication operations.

[0051] 9) Total Page View (TPV): refers to the sum of the number of page views during all statistical periods. The acquisition of total page views does not require deduplication.

[0052] 10) Access object behavior data set (U): refers to the object behavior data set composed of the number of people who visited a product within a statistical period after deduplication.

[0053] 11) Total Visit User (TU): refers to the object behavior dataset composed of the number of people who visited a product in all statistical periods after deduplication.

[0054] 12) Sorting interpolation: A data processing method that first sorts the user set or data sequence from large to small (or small to large), and then selects a certain sorting position for interpolation.

[0055] 13) Intelligent Vehicle Infrastructure Cooperative System (IVICS), referred to as IVICS, is a development direction of Intelligent Transportation System (ITS). IVICS uses advanced wireless communication and new generation Internet technologies to implement all-round dynamic real-time information interaction between vehicles and roads, and conducts active vehicle safety control and road cooperative management based on the collection and integration of dynamic traffic information in all time and space, fully realizing the effective coordination of people, vehicles and roads, ensuring traffic safety, and improving traffic efficiency, thus forming a safe, efficient and environmentally friendly road traffic system.

[0056] See also Figure 1 , Figure 1It is a flowchart of the mapping and reduction processing of the object behavior data provided in the related technology. The distributed computing in the related technology adopts a distributed iterative method. First, an initial moment is selected as the 0 point, and a distributed technology computing method is used to aggregate and calculate (for example, sum operation) all historical data before the initial moment as the initial value TUV0 of the aggregate calculation (for example, sum operation). For example, one month after the function is launched is taken as the initial moment, and the Mapredu ce platform in the related technology is used to count all the deduplicated users of this month as the initial value TUV0, and the sum of the page views of all users in this month is counted as the initial value TPV0. All historical object behavior data before the initial moment are taken as the full object behavior data at the initial moment, and are deduplicated and sorted from small to large according to the object identifier to obtain the initial full user set TU0.

[0057] See also Figure 2 , Figure 2 The flowchart of the data statistics of the object behavior data provided in the related art is as follows. For the data calculation at time T1, the object behavior data at time T1 is deduplicated according to the object identifier, and the object behavior data U1 at time T1 is matched with the initial user set TU0 to obtain the newly added user set. And add new user sets Add to the initial full user set TU0 to get the full user set TU1 at time T1. At the same time, calculate the number of new users UV1 at time T1 new The cumulative number of users at time T1 is TUV1 = TUV0 + UV1 new , calculate the number of new users PV1 at time T1 new The cumulative number of users at time T1 is TPV1 = TPV0 + PV1 new , and so on, we get T n The total user set TU at the moment n (n≥1) and the cumulative number of users TUV n =TUV n-1 +UV t new , cumulative number of users Among them, TUV n-1 Indicates T n-1 The number of users at a given moment, TPV n-1 Indicates T n-1 The cumulative number of visits at the moment.

[0058] In the related art, when the distributed computing solution calculates TUV and TPV in each statistical period, it is necessary to use the user table U in the current statistical period and the full table TU in the previous statistical period to perform a Left Join operation. However, when the user table U in the current statistical period is large, the table operation of Left Join will still cause data skew, thereby limiting the data processing efficiency. In addition, in the related art, when calculating the newly added UV and newly added PV in each statistical period, it is still necessary to use MapReduce to perform the calculation. When counting the newly added user UV and newly added PV in each statistical period, it is necessary to use the Left Join operation based on the MapReduce architecture to match the newly added user set NU. Therefore, MapReduce still needs to be called in each statistical period, which consumes a lot of data processing resources.

[0059] The embodiments of the present application provide a data processing method, device, electronic device, computer program product, and computer-readable storage medium, which can improve data processing efficiency and improve data processing resource utilization. The exemplary application of the electronic device provided by the embodiments of the present application is described below. The electronic device provided by the embodiments of the present application can be implemented as various types of user terminals such as laptops, tablet computers, desktop computers, set-top boxes, mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), and can also be implemented as servers. Below, an exemplary application when the device is implemented as a server will be described.

[0060] See also Figure 3 , Figure 3 It is a schematic diagram of the architecture of the data processing system provided in an embodiment of the present application. To support an electronic map application, the terminal 400 is connected to the server 200 via the network 300. The network 300 may be a wide area network or a local area network, or a combination of the two, and a wireless link is used to realize data transmission.

[0061] In an application scenario, the function of the electronic map is implemented based on the server 200 and the terminal 400. When the user uses the terminal 400 to access the electronic map, in response to the terminal 400 receiving the behavior operation of the object, the terminal 400 obtains the behavior object data of the first time period and the behavior object data within the target time period, and sends them to the server 200, so that the server 200 implements the data processing method provided in the embodiment of the present application on the received data, obtains the behavior object set of the second time period, the second time period includes the first time period and the target time period, and stores the behavior object set of the second time period in the database 500.

[0062] In other embodiments, when the data processing method provided by the embodiments of the present application is implemented by a terminal alone, in the application scenario described above, in the process of a user using a terminal to access an electronic map, in response to the terminal 400 receiving a behavior operation of an object, the terminal obtains the behavior object data of the first time period and the behavior object data within the target time period, and implements the data processing method provided by the embodiments of the present application on the obtained data to obtain a behavior object set of the second time period, the second time period including the first time period and the target time period, and stores the behavior object set of the second time period in a database.

[0063] In some embodiments, the server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal 400 may include but is not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, etc., and the terminal 400 may be provided with a client, such as an instant messaging client, a learning client, a game client, a map client, a road toll system client, etc., but is not limited thereto. The terminal and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0064] Next, the structure of the electronic device for implementing the data processing method provided by the embodiment of the present application is described. As mentioned above, the electronic device provided by the embodiment of the present application may be Figure 3 Server 200 in . Figure 4 , Figure 4 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, Figure 4 The server 200 shown includes: at least one processor 210, a memory 250, and at least one network interface 220. The various components in the server 200 are coupled together via a bus system 240. It is understood that the bus system 240 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 240 is not described in detail. Figure 4 Various buses are labeled as bus system 240 .

[0065] The processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0066] The memory 250 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 250 may optionally include one or more storage devices that are physically remote from the processor 210.

[0067] The memory 250 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 250 described in the embodiments of the present application is intended to include any suitable type of memory.

[0068] In some embodiments, memory 250 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0069] The operating system 251 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic businesses and processing hardware-based tasks; a network communication module 252 is used to reach other computing devices via one or more (wired or wireless) network interfaces 220. Exemplary network interfaces 220 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB), etc.

[0070] In some embodiments, the data processing device provided in the embodiments of the present application can be implemented in software. Figure 4 A data processing device 255 stored in a memory 250 is shown, which may be software in the form of a program or plug-in, including the following software modules: an acquisition module 2551, a sorting module 2552, a comparison module 2553, a merging module 2554, and a cumulative statistics module 2555. These modules are logical, and therefore may be arbitrarily combined or further split according to the functions implemented. The functions of each module will be described below.

[0071] In some embodiments, the terminal or server can implement the data processing method provided in the embodiments of the present application by running a computer program. For example, the computer program can be a native program or software module in the operating system; it can be a native application (APP, Application), that is, a program that needs to be installed in the operating system to run, such as a map APP; it can also be a small program, that is, a program that can be run only by downloading it to a browser environment; it can also be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module or plug-in.

[0072] Next, the data processing method provided in the embodiment of the present application is described. In actual implementation, the data processing method provided in the embodiment of the present application can be Figure 3 The terminal 400, the vehicle terminal equipped with an electronic map or the server 200 shown in the figure can be implemented separately, or the terminal 400, the vehicle terminal equipped with an electronic map and the server 200 can be implemented in coordination.

[0073] The following description is made by taking the data processing method provided by the embodiment of the present application as an example in which the terminal 400 alone implements the data processing method. Figure 5A , Figure 5A is a flow chart of the data processing method provided in the embodiment of the present application, which will be combined with Figure 5A The steps shown are explained.

[0074] In step 101, a first behavior object set in a first time period is obtained, and behavior objects in a target time period are obtained.

[0075] As an example, the target time period is adjacent to the first time period and is located after the first time period.

[0076] In some embodiments, when the first time period is an initial time period starting from an initial moment, obtaining the first behavior object set of the first time period in step 101 can be achieved by the following technical solution: obtaining the object behavior data at the initial moment of the first time period; performing object deduplication processing on the object behavior data at the initial moment to obtain the initial behavior object set at the initial moment; and determining the first behavior object set of the first time period based on the initial behavior object set at the initial moment.

[0077] As an example, the object behavior data at time T0 is input as the initial input. Time T0 is the initial time of the entire data processing flow. For example, 00:00 on the 30th day after the function is launched is taken as the initial time. Distributed technology is used to calculate and aggregate all historical data before the initial time. The Map reduce platform is used to count all deduplicated user numbers for this month through mapping and reduction processing as the initial cumulative object number TUV0. The sum of the page views of all users for this month is counted as the initial cumulative visit volume TPV0. All object behavior data before the initial time is taken as the full object behavior data at the initial time, and are sorted from small to large according to the object identifier to obtain the full initial behavior object set TU0.

[0078] In some embodiments, the above-mentioned initial behavior object set based on the initial moment determines the first behavior object set for the first time period, which can be achieved through the following technical scheme: performing descending sorting processing on the behavior objects in the initial behavior object set based on the object identifier to obtain the first maximum object identifier of the initial behavior object set; performing descending sorting processing on the behavior objects in the initial behavior object set based on the object identifier to obtain a second descending result; obtaining a second target behavior object whose object identifier is greater than the first maximum object identifier from the second descending result; merging the second target behavior object into the initial behavior object set to obtain the first behavior object set for the first time period.

[0079] In some embodiments, the above-mentioned merging of the second target behavior object into the initial behavior object set to obtain the first behavior object set in the first time period can be achieved through the following technical scheme: sorting the behavior objects in the initial behavior object set in descending order; inserting the second target behavior object into the end of the descending sort result of the behavior objects in the initial behavior object set in the order represented by the second descending result, to obtain the first behavior object set.

[0080] As an example, when merging the second target behavior object into the initial behavior object set, the second target behavior object is obtained from the second descending result. Therefore, if there are multiple second target behavior objects, the order of the multiple second target behavior objects can be represented by the second descending result. The second target behavior object is directly inserted into the end of the descending sort result of the behavior objects in the initial behavior object set in the order represented by the first descending result, which is equivalent to sequentially splicing the two sorting results, so that the behavior objects in the first behavior object set have their own sequential attributes based on object identifiers.

[0081] In some embodiments, the above-mentioned obtaining of the second target behavior object whose object identifier is greater than the first maximum object identifier from the second descending result can be achieved through the following technical solution: performing the following processing on each behavior object in the second descending result in the order represented by the second descending result: comparing the object identifier of the behavior object in the second descending result with the first maximum object identifier; when the comparison result represents that the object identifier of the behavior object in the second descending result is greater than the first maximum object identifier, determining the behavior object in the second descending result as the second target behavior object; when the comparison result represents that the object identifier of the behavior object in the second descending result is not greater than the first maximum object identifier, determining that the comparison processing will be stopped.

[0082] As an example, the behavior objects in the initial behavior object set TU0 are sorted in descending order based on the object ID, and the object ID with the first largest object ID in the initial behavior object set is obtained. The initial behavior object and the first largest object identifier As the comparison parameter, input the behavior object U1 in the first time period. Since the first time period is the initial time period starting from the initial moment, the first time period can be regarded as the first single cycle. U1 is sorted in descending order from large to small based on the object ID, and the object ID u1 of each behavior object in U1 is compared with the first largest object ID in descending order. For comparison, , the behavior object corresponding to u1 is used as the second target behavior object, and the second target behavior object constitutes the set of newly added behavior objects in the first single cycle. When the comparison result indicates that the object identifier of the behavior object in the second descending result is not greater than the object identifier of the initial behavior object, it is determined that the comparison process will be stopped, that is, the second target behavior object will no longer be obtained, and the second target behavior object is inserted into the last position of the descending sort result of the behavior objects in the initial behavior object set TU0 in the order represented by the second descending result to obtain the first behavior object set. Therefore, the behavior objects in the first behavior object set here are also arranged in descending order according to the size of the object identifier. Here, the Union operation is used to merge the behavior object corresponding to u1 with TU0. Thus, a first behavior object set TU1 is obtained. The first behavior object set TU1 is a set of all users up to a first moment, and the first moment is the end moment of the first time period.

[0083] In some embodiments, the above-mentioned acquisition of behavior objects within the target time period can be achieved through the following technical solutions: acquiring object behavior data within the target time period; performing object deduplication processing on the object behavior data within the target time period to obtain the behavior objects within the target time period.

[0084] As an example, when the first time period is from January 1, 2021 to January 2, 2021, the target time period is from January 2, 2021 to January 3, 2021, and the second time period is from January 1, 2021 to January 3, 2021. The object behavior data from January 2, 2021 to January 3, 2021 is obtained, for example, including 5,000 access records, and the object behavior data within the target time period (5,000 access records) is deduplicated. Since these 5,000 access records are obtained based on the behavior of 1,000 behavior objects, some behavior objects are accessed multiple times, and therefore object deduplication is required to obtain the behavior objects within the target time period.

[0085] In step 102, the behavior objects within the target time period are sorted in descending order based on the object identifiers to obtain a first descending order result.

[0086] As an example, the target time period is the time period between two time nodes. For example, when performing data statistics, data is updated and stored according to the same time interval, with the target time period as T n -T n-1 To illustrate, the first time period is T n-1 , represents the time period from the initial moment to the first moment (the last time data was updated and stored), and the second time period is T n , representing the time period from the initial moment to the second moment (this data update and storage), and the target time period is the time period between the moment when the data was last updated and stored and the moment when the data is currently updated and stored.

[0087] In step 103, a first target behavior object whose object identifier is greater than that of all behavior objects in the first behavior object set is obtained from the first descending result.

[0088] In step 104, the first target behavior objects sorted according to the first descending result are merged into the first behavior object set to obtain a second behavior object set in the second time period.

[0089] As an example, the second time period includes the first time period and the target time period.

[0090] In some embodiments, the first target behavior object sorted according to the first descending result is merged into the first behavior object set in the above step 104, which can be achieved through the following technical solution: the first target behavior object is inserted into the end of the descending sort result of the behavior objects in the first behavior object set in the order represented by the first descending result.

[0091] As an example, when merging the first target behavior object into the first behavior object set, the first target behavior object is obtained from the first descending result. Therefore, if the number of first target behavior objects is multiple, the order of the multiple first target behavior objects can be represented by the first descending result, and the behavior objects in the first behavior object set have their own sequence attributes based on object identifiers. Therefore, the behavior objects in the second behavior object set in the second time period also have their own sequence attributes based on object identifiers. The first target behavior object is directly inserted into the end of the descending sort result of the behavior objects in the first behavior object set in the order represented by the first descending result, which is equivalent to sequentially splicing the two sorting results.

[0092] As an example, based on the first time period T n-1 The first behavior of the object set generates the second time period T n The process of generating the second behavior object set can be traced back to the process of generating the behavior object set of the first time period (T1) based on the data of the initial moment, and generating the behavior object set of the second time period (T2) based on the behavior object set of the first time period (T1). When generating the behavior object set of the first time period (the initial time period with the initial moment as the starting point) based on the data of the initial moment, it is necessary to perform sorting processing. Therefore, the behavior objects in the behavior object set of the first time period (T1) also have a sequential attribute based on the object identifier. When generating the behavior object set of the second time period (T2) based on the behavior object set of the first time period (T1), it is necessary to introduce the first descending result of the behavior objects of the target time period. Therefore, the behavior objects in the generated behavior object set of the second time period also have a sequential attribute based on the object identifier. By analogy, each time a new target time period is introduced, it is only necessary to sort the behavior objects within the target time period, and there is no need to sort the behavior objects of the first time period.

[0093] See also Figure 5B , Figure 5B is a flow chart of the data processing method provided in the embodiment of the present application. In step 103, the first target behavior object whose object identifier is greater than all the behavior objects in the first behavior object set is obtained from the first descending result. Figure 5B Steps 1031 to 1032 in are implemented.

[0094] In step 1031, the second largest object identifier of the first line object set is obtained.

[0095] As an example, when the first time period corresponding to the first behavior object set is the initial time period, the behavior objects in the first behavior object set need to be sorted in descending order to obtain the second largest object identifier of the first behavior object set; if the first time period corresponding to the first behavior object set also includes the time period after the initial time period, and the first behavior object set has its own sequence attribute based on the object identifier, the second largest object identifier of the first behavior object set can be directly obtained; if the first behavior object set does not have its own sequence attribute based on the object identifier, the behavior objects in the first behavior object set need to be sorted in descending order based on the object identifier to obtain the second largest object identifier of the first behavior object set.

[0096] In step 1032, a first target behavior object whose object ID is greater than the second largest object ID is obtained from the first descending result.

[0097] In some embodiments, obtaining the first target behavior object whose object identifier is greater than the second maximum object identifier from the first descending result in step 1032 can be achieved by the following technical solution: performing the following processing on each behavior object in the first descending result in the order represented by the first descending result: comparing the object identifier of the behavior object in the first descending result with the second maximum object identifier; when the comparison result represents that the object identifier of the behavior object in the first descending result is greater than the second maximum object identifier, determining the behavior object in the first descending result as the first target behavior object; when the comparison result represents that the object identifier of the behavior object in the first descending result is not greater than the second maximum object identifier, determining that the comparison processing will be stopped.

[0098] As an example, based on the first time period T n-1 The first row of the object set TU is sorted in descending order n-1 , get the second largest object ID in the first row of the object set The behavior object and the second largest object identifier As the comparison parameter, input the behavior object set U of the target time period n , for U n The behavior objects in the object are sorted in descending order based on the object identifier, and U n The object identifier u of each behavior object in n Sort in descending order with the second largest object ID For comparison, When u n The corresponding behavior object is used as the first target behavior object, and the first target behavior object constitutes the set of newly added behavior objects in the target time period. When the comparison result indicates that the object ID of the behavior object in the first descending result is not greater than the second largest object ID, it is determined that the comparison process will be stopped, that is, the first target behavior object will no longer be obtained, and the first target behavior object will be inserted into the first behavior object set TU in the order represented by the first descending result. n The last one of the descending sorting results of the behavior objects in the second behavior object set is obtained. Therefore, the behavior objects in the second behavior object set are also sorted in descending order according to the size of the object identifier. Here, the Union operation is used to sort u n Corresponding behavior objects and TU n-1 merge, Thus, the second behavior object set TU is obtained n , the second line is the object set TU n It is the total set of users from the initial moment to the nth moment.

[0099] In some embodiments, see Figure 5C , Figure 5C is a flow chart of the data processing method provided in an embodiment of the present application. After obtaining the first target behavior object whose object identifier is greater than all behavior objects in the first behavior object set from the first descending result in step 103, the following can be executed: Figure 5C Steps 105 to 107 in .

[0100] In step 105, the cumulative number of visits and the cumulative number of objects in the first time period are obtained, and the cumulative number of visits in the target time period is obtained.

[0101] In some embodiments, when the first time period is an initial time period starting from an initial moment, obtaining the cumulative visits and the cumulative number of objects in the first time period in step 105 can be achieved by the following technical solutions: obtaining the object behavior data at the initial moment of the first time period; performing object deduplication processing on the object behavior data at the initial moment to obtain an initial behavior object set at the initial moment; performing mapping and reduction processing on the behavior objects in the initial behavior object set to obtain the initial cumulative visits and the initial cumulative number of objects at the initial moment; determining the cumulative visits and the cumulative number of objects in the first time period based on the initial cumulative visits and the initial cumulative number of objects.

[0102] In some embodiments, the above-mentioned determination of the cumulative visits and the cumulative number of objects in the first time period based on the initial cumulative visits and the initial cumulative number of objects can be achieved through the following technical solutions: performing descending sorting processing on the behavior objects in the initial behavior object set based on the object identifier to obtain the first maximum object identifier of the initial behavior object set; performing descending sorting processing on the behavior objects in the initial behavior object set based on the object identifier to obtain a second descending result; obtaining a second target behavior object whose object identifier is greater than the first maximum object identifier from the second descending result; adding the number of the second target behavior objects to the initial cumulative number of objects to obtain the cumulative number of objects in the first time period; adding the cumulative visits of the first time period to the initial cumulative visits to obtain the cumulative visits of the first time period.

[0103] As an example, enter the number of behavior objects UV1 in the first time period new , input the cumulative number of objects TUV0 at the initial time T0, the cumulative number of objects TUV0 at the initial time T0 is the cumulative number of behavior objects in the time period from the initial time to the first time, calculate the number of second target behavior objects UV1 from the initial time to the first time (first time period) new , the number of the second target behavior objects is the number of new users from the initial moment to the first moment (the first time period), and the cumulative number of users up to the first moment is TUV1 = TUV0 + UV1 new , ending at time 1, represents the period from the initial time to time 1. Enter the cumulative number of visits PV1 in the first time period new , input the cumulative number of visits TPV0 at the initial time T0, the cumulative number of objects TUV0 at the initial time T0 is the cumulative number of visits to the behavior object in the time period from the first access behavior to the initial time, calculate the cumulative number of visits PV1 from the initial time to the first time (the first time period) new The cumulative visits in the first time period are the number of new user visits from the initial moment to the first moment (the first time period). According to the iterative formula n=n+1, the cumulative visits up to the first moment are obtained as TPV1=TPV0+PV1 new , up to the first moment, represents from the initial moment to the first moment.

[0104] In step 106, the number of first target behavior objects is added to the cumulative number of objects in the first time period to obtain the cumulative number of objects in the second time period.

[0105] As an example, enter the number of behavior objects in the target time period. Enter the first time period T n-1 Cumulative number of objects TUV n-1 , the first time period T n-1 Cumulative number of objects TUVn-1 is the cumulative number of behavior objects in the time period from the initial moment to the n-1th moment, and calculates the number of first target behavior objects from the n-1th moment to the nth moment (target time period) The number of the first target behavior objects is the number of new users from the n-1th moment to the nth moment (target time period). According to the iterative formula n=n+1, the cumulative number of users up to the nth moment is obtained. The period from the initial moment to the nth moment is represented as the second time period, and the period from the initial moment to the nth moment is represented as the first time period.

[0106] In step 107, the cumulative number of visits in the target time period is added to the cumulative number of visits in the first time period to obtain the cumulative number of visits in the second time period.

[0107] As an example, enter the cumulative number of visits during the target time period. Enter the first time period T n-1 Total number of visits TPV n-1 , the first time period T n-1 Cumulative number of objects TUV n-1 is the cumulative number of visits to the behavior object in the time period from the initial moment to the n-1th moment, and calculates the cumulative number of visits from the n-1th moment to the nth moment (target time period) The cumulative number of visits in the target time period is the number of new user visits from the n-1th moment to the nth moment (the target time period). According to the iterative formula n=n+1, the cumulative number of visits up to the nth moment is obtained. The period from the initial moment to the nth moment is represented as the second time period, and the period from the initial moment to the nth moment is represented as the first time period.

[0108] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0109] In some embodiments, the function of the electronic map is implemented based on a server and a terminal. When a user uses a terminal to access the electronic map, in response to the terminal receiving a behavior operation of an object, the terminal obtains the behavior object data of a first time period and the behavior object data within a target time period, and sends them to the server, so that the server implements the data processing method provided in the embodiment of the present application on the received data, obtains a behavior object set of a second time period, the second time period includes the first time period and the target time period, and stores the behavior object set of the second time period in a database.

[0110] The data processing method provided in the embodiment of the present application is applied to the field of big data computing optimization for Internet of Vehicles products, obtains log data of Internet of Vehicles products and corresponding modules of Internet of Vehicles products, and extracts relevant object behavior data (detailed data of each user's click behavior data in Internet of Vehicles products, browsing pictures, browsing pages, recharge operations, etc.) and stores them in a database (MySQL, Oracle, HDFS, HBASE, Hive, etc.).

[0111] The overall process of the data processing method provided in the embodiment of the present application can be mainly divided into the following six stages: (1) initial data input stage, (2) acquisition of initial user set, initial browsing times and initial visiting user number, (3) acquisition of new user set in T1 single cycle, (4) calculation of cumulative visitor number TUV and cumulative visit volume TPV at T1, (5) n Periodic new user set acquisition phase, (6)T n The calculation stage of the cumulative number of visitors TUV and the cumulative number of visits TPV at any moment.

[0112] See also Figure 6 , Figure 6 This is a flow chart of data statistics of object behavior data provided by the embodiment of the present application. Figure 6 The six-stage data processing process is described in detail.

[0113] In the first stage, the object behavior data at time T0 is input as the initial input, and time T0 is the initial time of the entire data processing flow.

[0114] In the second stage, the initial object behavior data is input, the object behavior data is processed by deduplication, and the object identifier Userid is sorted in descending order from large to small to obtain the initial full user set TU0, and the initial cumulative object number TUV0 and the initial cumulative object number TPV0 are calculated in a MapReduce distributed manner.

[0115] In the third stage, the initial full user set TU0 sorted in descending order in the second stage is input, and the object identifier with the highest sorting is selected. As a parameter, input the user set U1 of the single cycle of T1, sort U1 in descending order based on object identity, and compare the object identity u1 of each behavior object in U1 with For comparison, if Then add u1 in descending order Using Union operation Merged with TU0, Thus, the total user set TU1 up to time T1 is obtained, and the formula n=n+1 is used to iterate Number of users UV1 new Sum PV1 new .

[0116] In the fourth stage, the cumulative user UV and cumulative PV calculation stage at time T1. Input the number of users UV1 in the T1 period of Step 3 new Sum PV1 new , input TUV0 and TPV0 of Step 2, and then calculate the number of new users UV1 at time T1 according to the iterative formula new The cumulative number of users at time T1 is TUV1 = TUV0 + UV1 new , calculate the number of new users PV1 at time T1 new , we get the cumulative number of users at time T1: TPV1 = TPV0 + PV1 new .

[0117] In the fifth stage, T n The periodic new user set acquisition phase. Input the full user set TU that has been sorted in descending order in the n-1 period n-1 , and select the first object ID As a parameter, enter T n The single-period user set U of the period n , for U n Sort in descending order from large to small, and put U n The object identifier u n and For comparison, if Then put u n Add in descending order At the same time, the iterative formula n=n+1 is used, and the iteration Number of users and times Using Union operation With TU n-1 Combined to get T n All user sets at the moment TU n ,Right now

[0118] In the sixth stage, enter the number of users in the single cycle (target time period) of the second time period in the fifth stage and times Enter the first time period T n-1 Cumulative number of objects TUV n-1 And the cumulative number of visits TPV n-1 , the first time period T n-1 Cumulative number of objects TUV n-1is the cumulative number of behavior objects in the time period from the initial moment to the n-1th moment, and calculates the number of new users from the n-1th moment to the nth moment (target time period) According to the iterative formula n=n+1, the cumulative number of users as of the nth moment is obtained Calculate the number of new users from the n-1th moment to the nth moment (target time period) Get the cumulative number of users as of the nth moment

[0119] In the data processing method provided in the embodiment of the present application, the full user table up to the last period and the user table of the current statistical period are first sorted in descending order from large to small according to the object identifier, and the object identifier ranked first in the full table is selected as an input parameter, and compared with the object identifier sorted in descending order in the current statistical period. If the object identifier of the current statistical period is greater than the input object identifier, this part of the object identifier is selected to form a new object behavior data set for the current period. In the data processing method provided in the embodiment of the present application, only the MapReduce method is needed to perform statistical calculations during the initial statistical period calculation, and MapReduce processing does not need to be performed in the remaining periods, which effectively improves the computing efficiency and saves computing resources.

[0120] In the data processing method provided in the embodiment of the present application, when acquiring a newly added user set, it is not necessary to use the user table of the current statistical period to match the full user table with a Left Join operation, thereby effectively improving the computing efficiency.

[0121] In the data processing method provided in the embodiment of the present application, when iterative calculation is used, T is calculated n The data at the moment is based on the cut-off T n-1 Cumulative number of objects TUV until the end of the period n-1 , cumulative number of visits TPV n-1 , plus the number of new users and new page views in the current statistical period that is greater than the number of users identified by the input object. There is no need to use Left Join, Distinct, Group By and other operations, which can effectively reduce computing resources and improve computing efficiency.

[0122] In the data processing method provided in the embodiment of the present application, when updating the full user table, after the newly added users in the current statistical period are arranged in descending order based on the object identifier, a Union operation is used to merge the full user table sorted in descending order based on the object identifier up to the last statistical period, thereby obtaining a full user table sorted in descending order up to the current statistical period. The full user table only needs to perform a full user sorting once in the initial period (up to time T0), and only needs to sort the newly added users in descending order in each subsequent statistical period, thereby effectively reducing the computing resource occupation caused by the need to sort all users up to each statistical period, and effectively improving the computing efficiency.

[0123] The data processing method provided in the embodiment of the present application is a solution for optimizing data calculation based on big data statistical technology. In addition to the Internet of Vehicles, it can also be used in big data statistical analysis of all Internet.

[0124] It is understandable that in the embodiments of the present application, related data such as user information is involved. When the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0125] The following further describes an exemplary structure of the data processing device 255 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 4 As shown, the software modules stored in the data processing device 255 of the memory 250 may include: an acquisition module 2551, used to acquire a first set of behavior objects in a first time period, and acquire behavior objects in a target time period, wherein the target time period is adjacent to the first time period and is located after the first time period; a sorting module 2552, used to perform descending sorting processing on the behavior objects in the target time period based on the object identifier to obtain a first descending result; a comparison module 2553, used to acquire a first target behavior object whose object identifier is greater than all behavior objects in the first behavior object set from the first descending result; a merging module 2554, used to merge the first target behavior object sorted according to the first descending result into the first behavior object set to obtain a second set of behavior objects in a second time period, wherein the second time period includes the first time period and the target time period.

[0126] In some embodiments, when the first time period is an initial time period starting from an initial moment, the acquisition module 2551 is also used to: obtain the object behavior data at the initial moment of the first time period; perform object deduplication processing on the object behavior data at the initial moment to obtain an initial behavior object set at the initial moment; and determine the first behavior object set for the first time period based on the initial behavior object set at the initial moment.

[0127] In some embodiments, the acquisition module 2551 is also used to: perform descending sorting processing on the behavior objects in the initial behavior object set based on the object identifier to obtain the first maximum object identifier of the initial behavior object set; perform descending sorting processing on the behavior objects in the initial behavior object set based on the object identifier to obtain a second descending result; obtain a second target behavior object whose object identifier is greater than the first maximum object identifier from the second descending result; merge the second target behavior object into the initial behavior object set to obtain the first behavior object set for the first time period.

[0128] In some embodiments, the acquisition module 2551 is further used to: insert the second target behavior object into the last position of the descending sort result of the behavior objects in the initial behavior object set in the order represented by the second descending result, to obtain the first behavior object set.

[0129] In some embodiments, the acquisition module 2551 is also used to: perform the following processing on each behavior object in the second descending result in the order represented by the second descending result: compare the object identifier of the behavior object in the second descending result with the first maximum object identifier; when the comparison result represents that the object identifier of the behavior object in the second descending result is greater than the first maximum object identifier, determine the behavior object in the second descending result as the second target behavior object; when the comparison result represents that the object identifier of the behavior object in the second descending result is not greater than the first maximum object identifier, determine that the comparison processing will be stopped.

[0130] In some embodiments, the acquisition module 2551 is further used to: acquire object behavior data within a target time period; perform object deduplication processing on the object behavior data within the target time period to obtain the behavior objects within the target time period.

[0131] In some embodiments, the comparison module 2553 is further used to: obtain the second largest object identifier of the first behavior object set; and obtain the first target behavior object whose object identifier is greater than the second largest object identifier from the first descending result.

[0132] In some embodiments, the comparison module 2553 is also used to: perform the following processing for each behavior object in the first descending result in the order represented by the first descending result: compare the object identifier of the behavior object in the first descending result with the second largest object identifier; when the comparison result represents that the object identifier of the behavior object in the first descending result is greater than the second largest object identifier, determine the behavior object in the first descending result as the first target behavior object; when the comparison result represents that the object identifier of the behavior object in the first descending result is not greater than the second largest object identifier, determine that the comparison processing will be stopped.

[0133] In some embodiments, the device also includes: a cumulative statistics module 2555, which is used to: obtain the cumulative visits and the cumulative number of objects in the first time period, and obtain the cumulative visits in the target time period; add the number of first target behavior objects to the cumulative number of objects in the first time period to obtain the cumulative number of objects in the second time period; add the cumulative visits of the target time period to the cumulative visits of the first time period to obtain the cumulative visits of the second time period.

[0134] In some embodiments, when the first time period is an initial time period starting from an initial moment, the cumulative statistics module 2555 is further used to: obtain object behavior data at the initial moment of the first time period; perform object deduplication processing on the object behavior data at the initial moment to obtain an initial behavior object set at the initial moment; perform mapping and reduction processing on the behavior objects in the initial behavior object set to obtain an initial cumulative number of visits and an initial cumulative number of objects at the initial moment; determine the cumulative number of visits and the cumulative number of objects for the first time period based on the initial cumulative number of visits and the initial cumulative number of objects.

[0135] In some embodiments, the cumulative statistics module 2555 is also used to: perform descending sorting processing on the behavior objects in the initial behavior object set based on the object identifier to obtain the first maximum object identifier of the initial behavior object set; perform descending sorting processing on the behavior objects in the initial behavior object set based on the object identifier to obtain a second descending result; obtain the second target behavior object whose object identifier is greater than the first maximum object identifier from the second descending result; add the number of the second target behavior objects to the initial cumulative number of objects to obtain the cumulative number of objects in the first time period; add the cumulative number of visits in the first time period to the initial cumulative number of visits to obtain the cumulative number of visits in the first time period.

[0136] The embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the data processing method described above in the embodiment of the present application.

[0137] The present application embodiment provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the data processing method provided by the present application embodiment, for example, Figures 5A-5C The data processing method is shown.

[0138] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EP ROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0139] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0140] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0141] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0142] To summarize, through the embodiment of the present application, a first behavior object set of a first time period is obtained, and behavior objects within a target time period are obtained, and the behavior objects within the target time period are sorted in descending order based on the object identifier to obtain a first descending result, and a first target behavior object whose object identifier is greater than that of all behavior objects in the first behavior object set is obtained from the first descending result, and the first target behavior object is merged into the first behavior object set to obtain a second behavior object set for a second time period, wherein the second time period includes the first time period and the target time period. When obtaining the second behavior object set for the second time period, it is no longer necessary to perform big data statistical processing on the entire data of the second time period, thereby effectively improving computing efficiency and saving computing resource usage. Since only the behavior objects within the target time period need to be sorted in descending order, the computing resource usage caused by the need to sort the entire behavior objects by the end of each time period is effectively reduced, thereby effectively improving computing efficiency.

[0143] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A data processing method, characterized in that: The method comprises: Acquire a first set of behavior objects in a first time period, and acquire behavior objects in a target time period, wherein the target time period is adjacent to the first time period and is located after the first time period, and the behavior objects are objects using an application; Sorting the behavior objects within the target time period in descending order based on the object identifiers to obtain a first descending order result; Obtain the second largest object identifier of the first behavior object set; Acquire a first target behavior object whose object identifier is greater than the second largest object identifier from the first descending result; The first target behavior object is merged into the first behavior object set to obtain a second behavior object set for a second time period, wherein the second time period includes the first time period and the target time period.

2. The method according to claim 1, characterized in that When the first time period is an initial time period starting from an initial moment, obtaining a first behavior object set for the first time period includes: Acquire the object behavior data at the initial moment of the first time period; Performing object deduplication processing on the object behavior data at the initial moment to obtain an initial behavior object set at the initial moment; Based on the initial behavior object set at the initial moment, a first behavior object set in the first time period is determined.

3. The method according to claim 2, characterized in that The determining, based on the initial behavior object set at the initial moment, a first behavior object set in the first time period includes: Sorting the behavior objects in the initial behavior object set at the initial moment in descending order based on the object identifier to obtain a first maximum object identifier of the initial behavior object set; Sorting the behavior objects in the first time period in descending order based on the object identifiers to obtain a second descending order result; Acquire a second target behavior object whose object identifier is greater than the first maximum object identifier from the second descending result; The second target behavior object is merged into the initial behavior object set to obtain a first behavior object set in the first time period.

4. The method according to claim 3, characterized in that The step of merging the second target behavior object into the initial behavior object set to obtain a first behavior object set for the first time period includes: The second target behavior object is inserted into the last position of the descending sort result of the behavior objects in the initial behavior object set in the order represented by the second descending result, so as to obtain the first behavior object set.

5. The method according to claim 3, characterized in that: The step of obtaining the second target behavior object whose object identifier is greater than the first maximum object identifier from the second descending result includes: The following processing is performed on each behavior object in the second descending result in sequence according to the order represented by the second descending result: Compare the object identifier of the behavior object in the second descending result with the first maximum object identifier; When the comparison result indicates that the object identifier of the behavior object in the second descending result is greater than the first maximum object identifier, determining the behavior object in the second descending result as the second target behavior object; The method further comprises: When the comparison result indicates that the object identifier of the behavior object in the second descending result is not greater than the first maximum object identifier, it is determined that the comparison process will be stopped.

6. The method according to claim 1, characterized in that The obtaining of the behavior objects within the target time period includes: Acquiring the object behavior data within the target time period; Object deduplication processing is performed on the object behavior data within the target time period to obtain the behavior objects within the target time period.

7. The method according to claim 1, characterized in that The step of obtaining the first target behavior object whose object identifier is greater than the second largest object identifier from the first descending result includes: The following processing is performed on each behavior object in the first descending result in sequence according to the order represented by the first descending result: Compare the object identifier of the behavior object in the first descending result with the second largest object identifier; When the comparison result indicates that the object identifier of the behavior object in the first descending result is greater than the second largest object identifier, determining the behavior object in the first descending result as the first target behavior object; The method further comprises: When the comparison result indicates that the object identifier of the behavior object in the first descending result is not greater than the second maximum object identifier, it is determined that the comparison process will be stopped.

8. The method according to claim 1, characterized in that The method further comprises: Obtaining the cumulative number of visits and the cumulative number of objects in the first time period, and obtaining the cumulative number of visits in the target time period; Adding the number of the first target behavior objects to the cumulative number of objects in the first time period to obtain the cumulative number of objects in the second time period; The cumulative number of visits in the target time period is added to the cumulative number of visits in the first time period to obtain the cumulative number of visits in the second time period.

9. The method according to claim 8, characterized in that When the first time period is an initial time period starting from an initial time, obtaining the cumulative number of visits and the cumulative number of objects in the first time period includes: Acquire the object behavior data at the initial moment of the first time period; Performing object deduplication processing on the object behavior data at the initial moment to obtain an initial behavior object set at the initial moment; Performing mapping and reduction processing on the behavior objects in the initial behavior object set to obtain the initial cumulative access volume and the initial cumulative number of objects at the initial moment; Based on the initial cumulative page views and the initial cumulative number of objects, the cumulative page views and the cumulative number of objects in the first time period are determined.

10. The method according to claim 9, characterized in that The determining the cumulative number of visits and the cumulative number of objects in the first time period based on the initial cumulative number of visits and the initial cumulative number of objects includes: Sorting the behavior objects in the initial behavior object set at the initial moment in descending order based on the object identifier to obtain a first maximum object identifier of the initial behavior object set; Sorting the behavior objects in the first time period in descending order based on the object identifiers to obtain a second descending order result; Acquire a second target behavior object whose object identifier is greater than the first maximum object identifier from the second descending result; Adding the number of the second target behavior objects to the initial cumulative number of objects to obtain the cumulative number of objects in the first time period; The cumulative number of visits in the first time period is added to the initial cumulative number of visits to obtain the cumulative number of visits in the first time period.

11. A data processing device, characterized in that: The device comprises: an acquisition module, configured to acquire a first set of behavior objects in a first time period, and acquire behavior objects in a target time period, wherein the target time period is adjacent to the first time period and is located after the first time period, and the behavior objects are objects using an application program; A sorting module, used to sort the behavior objects within the target time period in descending order based on the object identifiers to obtain a first descending result; A comparison module, configured to obtain a second largest object identifier of the first behavior object set; and obtain a first target behavior object whose object identifier is greater than the second largest object identifier from the first descending result; A merging module is used to merge the first target behavior object into the first behavior object set to obtain a second behavior object set in a second time period, wherein the second time period includes the first time period and the target time period.

12. An electronic device, characterized in that: The electronic device comprises: A memory for storing executable instructions; A processor, configured to implement the data processing method according to any one of claims 1 to 10 when executing the executable instructions stored in the memory.

13. A computer-readable storage medium storing executable instructions, characterized in that: When the executable instructions are executed by a processor, the data processing method according to any one of claims 1 to 10 is implemented.

14. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the data processing method according to any one of claims 1 to 10 is implemented.

Images