Data processing method and device, electronic equipment and computer readable storage medium

Abstract

A data processing method, a data processing device, an electronic device and a computer readable storage medium. The data processing method comprises: obtaining at least one sample data in a statistical group, each sample data comprising a statistical index and an index value of the statistical index; creating a first statistical array corresponding to the statistical index, the first statistical array comprising a plurality of first elements, the plurality of first elements being respectively used to statistically process different index values; and traversing the index values of the at least one sample data, and statistically processing the at least one sample data by using the first statistical array to obtain a data statistical result, the plurality of first elements in the first statistical array being respectively the statistical sub-results of each index value. The method uses an array to statistically process a plurality of index values by one data processing, thereby realizing high parallel computing, improving statistical efficiency, and having high reusability.

Description

Technical Field

[0001] Embodiments of this disclosure relate to a data processing method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] With the rapid development of science, technology, and the economy, society has entered the era of big data. Statistical analysis of big data allows us to extract valuable information, which can then be used to improve and optimize various sectors of society. For example, statistical analysis of traffic data can reveal shortcomings in urban transportation, enabling improvements to be made accordingly. Summary of the Invention

[0003] At least one embodiment of this disclosure provides a data processing method, including: acquiring at least one sample data in a statistical group, each sample data including a statistical indicator and an indicator value of the statistical indicator; creating a first statistical array corresponding to the statistical indicator, the first statistical array including a plurality of first elements, the plurality of first elements being used to perform statistics on different indicator values; and traversing the indicator values ​​of at least one sample data, and using the first statistical array to perform statistics on at least one sample data to obtain data statistical results, wherein the plurality of first elements in the first statistical array are statistical sub-results for each indicator value.

[0004] For example, in a data processing method provided in one embodiment of this disclosure, obtaining at least one sample data within the statistical group includes: obtaining multiple sample data corresponding to each statistical group within multiple statistical groups; traversing the indicator values ​​of the at least one sample data and using the first statistical array to perform statistics on the at least one sample data to obtain the data statistical result, which includes: for each statistical group, traversing the indicator values ​​of the at least one sample data and using the first statistical array to perform statistics on the at least one sample data to obtain the data statistical result.

[0005] For example, in a data processing method provided in one embodiment of this disclosure, at least one sample data obtained within a statistical unit is considered as a statistical group. The method further includes: determining at least one indicator value to be statistically analyzed from the different indicator values; obtaining multiple statistical intervals, wherein each statistical interval includes at least one statistical unit; establishing a second statistical array for the multiple statistical intervals, wherein the second statistical array includes multiple second elements, the multiple second elements corresponding one-to-one with the multiple statistical intervals; filtering interval statistical results belonging to the multiple statistical intervals from the data statistical results within each statistical group; and using the multiple second elements to perform statistics on the indicator value to be statistically analyzed in the interval statistical results to obtain the indicator value statistical results of the indicator value to be statistically analyzed in each statistical interval.

[0006] For example, in a data processing method provided in one embodiment of this disclosure, the plurality of second elements are used to perform statistics on the target indicator values ​​in the interval statistical results to obtain the indicator value statistical results of the target indicator values ​​in each statistical interval. This includes: for each interval statistical result, determining the statistical interval to which the statistical group corresponding to the interval statistical result belongs; extracting the statistical sub-results of the target indicator values ​​from the interval statistical results; and accumulating the statistical sub-results onto the second element corresponding to the statistical interval to which the statistical group corresponding to the interval statistical result belongs, to obtain the indicator value statistical results of the target indicator values ​​in each statistical interval.

[0007] For example, in a data processing method provided in one embodiment of this disclosure, the method involves iterating through the indicator values ​​of at least one sample data in the statistical group and using the first statistical array to perform statistics on the at least one sample data in the statistical group to obtain the data statistical results within the statistical group. This includes: iterating through the indicator values ​​of at least one sample data in the statistical group and using multiple elements in the first statistical array to count each indicator value in the statistical group to obtain the data statistical results within the statistical group.

[0008] For example, in a data processing method provided in one embodiment of this disclosure, multiple statistical intervals include a first statistical interval and a second statistical interval, the range of the second statistical interval is greater than the range of the first statistical interval, and the first statistical interval is within the second statistical interval.

[0009] For example, in a data processing method provided in one embodiment of this disclosure, the method further includes: receiving initial data from a data source; and establishing the at least one sample data based on the initial data.

[0010] For example, in a data processing method provided in an embodiment of this disclosure, the initial data includes statistical attribute information. Establishing the at least one sample data based on the initial data includes: determining whether a storage file exists for storing the initial data based on the statistical attribute information; in response to the existence of a storage file for storing the initial data, storing the initial data in the storage file for use as the at least one sample data; in response to the absence of a storage file for storing the initial data, determining the statistical group to which the initial data belongs based on the statistical attribute information, generating a file path based on the statistical group, and creating the storage file in the file path, wherein the storage file is used to store the initial data for use as the at least one sample data, and initial data belonging to the same statistical group are stored in the same storage file.

[0011] For example, in a data processing method provided in one embodiment of this disclosure, obtaining at least one sample data within the statistical group includes: generating a file path for a storage file corresponding to the statistical group; determining whether the file path exists; and in response to the existence of the file path, obtaining initial data within the statistical group from the storage file in the file path as the at least one sample data.

[0012] For example, in a data processing method provided in one embodiment of this disclosure, the statistical unit includes at least one terminal device with a preset time period or a preset number.

[0013] For example, in a data processing method provided in one embodiment of this disclosure, the data processing method is applied to multiple electronic devices, the at least one sample data includes multiple sample data groups, the multiple electronic devices correspond one-to-one with the multiple sample data groups, the multiple electronic devices are configured to perform statistics based on the corresponding sample data groups respectively, and add the statistical values ​​to obtain the data statistical result.

[0014] At least one embodiment of this disclosure provides a data processing apparatus, comprising: a sample acquisition unit configured to acquire at least one sample data within a statistical group, each sample data including a statistical indicator and an indicator value of the statistical indicator; an array creation unit configured to create a first statistical array corresponding to the statistical indicator, the first statistical array including a plurality of first elements, the plurality of first elements being used to perform statistics on different indicator values; and a traversal unit configured to traverse the indicator values ​​of the at least one sample data and perform statistics on the at least one sample data using the first statistical array to obtain data statistical results, wherein the plurality of first elements in the first statistical array are statistical sub-results for each indicator value.

[0015] At least one embodiment of this disclosure provides an electronic device, including a processor; a memory including one or more computer program instructions; wherein the one or more computer program instructions are stored in the memory and, when executed by the processor, implement the data processing method provided in any embodiment of this disclosure.

[0016] At least one embodiment of this disclosure provides a computer-readable storage medium that non-temporarily stores computer-readable instructions, wherein when the computer-readable instructions are executed by a processor, they implement the data processing method provided in any embodiment of this disclosure. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0018] Figure 1A A system architecture for applying the data processing method of the present disclosure embodiments is shown;

[0019] Figure 1B A flowchart of a data processing method provided by at least one embodiment of this disclosure is shown;

[0020] Figure 2 A flowchart of another data processing method provided by at least one embodiment of this disclosure is shown;

[0021] Figure 3 At least one embodiment of the present disclosure is shown. Figure 2 Flowchart of step S80;

[0022] Figure 4 A flowchart of another data processing method provided by at least one embodiment of this disclosure is shown;

[0023] Figure 5 At least one embodiment of the present disclosure is shown. Figure 4 Flowchart of step S420;

[0024] Figure 6 A flowchart of step S10 in FIG1 is shown, provided in at least one embodiment of the present disclosure;

[0025] Figure 7A A flowchart of another data processing method provided by at least one embodiment of this disclosure is shown;

[0026] Figure 7B A schematic diagram of another data processing method provided by at least one embodiment of the present disclosure is shown;

[0027] Figure 8 A schematic block diagram of a data processing apparatus provided in at least one embodiment of the present disclosure is shown;

[0028] Figure 9 A schematic block diagram of an electronic device provided for some embodiments of this disclosure;

[0029] Figure 10 A schematic block diagram of another electronic device provided for some embodiments of this disclosure; and

[0030] Figure 11 A schematic diagram of a computer-readable storage medium provided in at least one embodiment of the present disclosure is shown. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0032] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an,” “a,” or “the,” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “including,” “comprising,” or “containing,” and similar terms mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms “connected,” “linked,” or similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “upper,” “lower,” “left,” and “right,” etc., are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0033] In big data statistics, it is often necessary to perform statistical analysis on multiple items in a sample data set. In related techniques, it is usually necessary to iterate through the sample data for each item in multiple items to obtain the statistical results for each item. This statistical analysis method is time-consuming and labor-intensive. It is important to understand that the term "item" here can represent different meanings in different statistical contexts. For example, "item" can refer to different statistical indicators, or it can refer to the value of the same statistical indicator. For instance, "item" could include both the statistical indicator "total quantity" and the statistical indicator "equipment runtime." As another example, in the case of the statistical indicator "gender," "item" could refer to the gender indicator values ​​1 and 2, where "1" represents, for example, "male," and "2" represents, for example, "female."

[0034] For example, the transportation omni-media management platform is an integrated hardware and software digital platform for public transportation venue operators or media operators, providing services such as customer group analysis, information management and precise targeting, centralized management of media equipment, and real-time environmental monitoring. The front-end visual dashboard of the transportation omni-media management platform presents a rich variety of content, including information on institutional users, traffic flow, media advertising content, and hardware equipment. Each theme contains numerous statistical indicators and report components. For instance, the customer group analysis page of the front-end visual dashboard includes time-based statistics on various types of passenger flow (passing, reaching, and following), statistics on passenger flow by gender and age group, and ranking statistics on the number of followers for various advertisements. The media content page of the front-end visual dashboard includes the number of playback plans, the number of normal playback plans, the number of failed playback plans, popular advertisements, and popular materials. The hardware equipment page of the front-end visual dashboard includes information such as the runtime of various servers and playback devices. For example, the above statistical indicators need to be calculated according to the organizational dimension (such as lines, stations, and equipment in the subway scenario) and the time dimension (such as the day, 7 days, and 30 days), and most pages need to be updated regularly (such as every hour).

[0035] If the traffic multimedia management platform uses the statistical methods described in the aforementioned technologies to perform statistical analysis on the various statistical indicators and their values, it will not only require a long statistical time, but may also cause performance problems for the equipment used for statistical analysis due to the throughput of large amounts of data. It should be understood that although this disclosure uses a traffic multimedia management platform as an example to illustrate its implementation, this does not mean that this disclosure is only applicable to traffic application scenarios such as traffic multimedia management platforms. The data processing methods of this disclosure can be applied to any application scenario that involves statistical analysis of data.

[0036] At least one embodiment of this disclosure provides a data processing method, a data processing apparatus, an electronic device, and a computer-readable storage medium. The data processing method includes: acquiring at least one sample data within a statistical group, each sample data including a statistical indicator and an indicator value; creating a first statistical array corresponding to the statistical indicator, the first statistical array including a plurality of first elements, the plurality of first elements being used to perform statistics on different indicator values; and traversing the indicator values ​​of at least one sample data, and using the first statistical array to perform statistics on the at least one sample data to obtain a data statistical result, wherein the plurality of first elements in the first statistical array are respective statistical sub-results for each indicator value.

[0037] This data processing method utilizes arrays to perform statistical analysis on multiple indicator values ​​in a single data processing step, achieving high parallelism and thus improving computational efficiency. Furthermore, the method is highly reusable because it only requires updating the statistical indicators, allowing for further analysis of the updated indicators. In some embodiments, this data processing method can be executed multiple times, with each execution serving as a stage. Based on the statistical results obtained in the previous stage, subsequent stages are performed, enabling further statistical analysis of the results. This reduces data redundancy, achieves data regularization, and effectively avoids performance issues on computing devices caused by sudden large data throughput.

[0038] Figure 1A A system architecture for applying the data processing method provided in at least one embodiment of this disclosure is shown.

[0039] like Figure 1A As shown, the system architecture 100 includes a front-end visualization screen 101, a business terminal 102, and a big data back-end 103.

[0040] The Front-End Visualization Dashboard 101 is used to present a rich variety of thematic content, such as information on institutional users, traffic flow, media advertising content, and hardware equipment. For example, the customer analysis page of the Front-End Visualization Dashboard 101 includes time-based statistics on various types of customer traffic (passersby, visitors, and followers), statistics on customer traffic by gender and age group, and ranking statistics on the number of followers for various advertisements. The media content page of the Front-End Visualization Dashboard 101 includes the number of playback plans, the number of normal playback plans, the number of failed playback plans, popular advertisements, and popular creatives. The hardware equipment page of the Front-End Visualization Dashboard 101 includes information such as the runtime of various servers and playback devices.

[0041] The business unit 102 can be, for example, a server that provides support for the front-end visualization dashboard 101. The server can analyze and process data received from the front-end visualization dashboard 101, such as requests, and feed back the processing results (e.g., web pages, information, or data obtained or generated based on the request) to the front-end visualization dashboard 101. The business unit 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server, etc.

[0042] In the embodiments of this disclosure, the business terminal 102 interacts not only with the front-end visualization screen 101 but also with the big data backend 103. The business terminal 102 can serve as the interface between the front-end visualization screen 101 and the big data backend. For example, the business terminal 102 can transmit sample data from terminal devices or requests from the front-end visualization screen 101 to the big data backend 103, whereby the big data backend 103 performs statistical analysis on the sample data or responds to the requests. The business terminal 102 can also send the statistical results from the big data backend 103 to the front-end visualization screen 101 to display the results of the statistical analysis.

[0043] The big data backend 103 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server, etc. The data processing method provided in at least one embodiment of this disclosure can be executed by the big data backend 103. This system architecture can push batch data processing down to the big data backend 103 for computation, effectively reducing the computation and response time pressure on the front-end visualization screen 101 and the business terminal 102.

[0044] Understandable, Figure 1A The system architecture shown is merely an example and is not intended to limit the scope of this disclosure. Those skilled in the art can implement the data processing method of this disclosure using any system architecture. For example, the data processing method can also be executed solely by the business terminal 102 and the front-end visualization screen 101. The data processing method provided in at least one embodiment of this disclosure is mainly executed by the business terminal 102 (e.g., performing calculations). In this case, the big data backend 103 may not be included in the system architecture.

[0045] Figure 1B A flowchart of a data processing method provided by at least one embodiment of the present disclosure is shown.

[0046] like Figure 1B As shown, the method may include steps S10 to S30.

[0047] Step S10: Obtain at least one sample data within the statistical group, each sample data including a statistical indicator and the indicator value of the statistical indicator.

[0048] Step S20: Create a first statistical array corresponding to the statistical indicators. The first statistical array includes multiple first elements, which are used to perform statistics on different indicator values.

[0049] Step S30: Iterate through the index values ​​of at least one sample data and use the first statistical array to perform statistics on at least one sample data to obtain data statistical results.

[0050] For step S10, for example, at least one sample data collected within a preset time period is used as a statistical group, or at least one sample data collected by a preset terminal device (e.g., a display screen placed at a station in a subway scenario) is used as a statistical group.

[0051] For example, with a preset time period of 1 hour, at least one sample data point is acquired from 15:00 to 16:00 on March 30, 2021. For example, in a subway scenario, the sample data may include trajectory records after analyzing images captured by cameras using computer vision technology, where the cameras are cameras on displays placed in subway stations.

[0052] Table 1 below schematically shows at least one sample data collected by the statistical group between 15:00 on March 30, 2021 and 16:00 on March 30, 2021.

[0053] As shown in Table 1, Table 1 contains multiple fields, each representing a sample indicator. Each sample data can include multiple sample indicators, such as: event time, camera identifier (ID), head ID, head status, age, gender, and duration. Each sample indicator has a corresponding indicator value, which can refer to the value of the sample indicator. For example, the head status indicator values ​​include 1, 2, and 3, representing that the passenger's status was identified as passing by the screen, touching the screen, and paying attention to the screen, respectively. Here, "screen" refers, for example, to the screen of a display screen in a subway station. The age indicator values ​​are 1, 2, 3, and 4, representing the identified age range, such as 1 for 0-18 years old, 2 for 19-30 years old, 3 for 31-50 years old, and 4 for 51 years old and above. The gender indicator values ​​are 1 and 2, representing male and female, respectively. The event time indicator value can be the time when the event occurred, i.e., the time when the head ID was collected. The camera ID indicator value can refer to the number of the camera that collected the head ID. For example, in a subway scenario, the camera could be a camera placed on a display screen in the station.

[0054] For example, in Table 1, Example Data 1 indicates that at event time 1599129468132, camera04 captured a passenger marked with head ID 1 touching the screen. The passenger's age is between 0 and 18 years old, and the passenger is female.

[0055] In some embodiments of this disclosure, one or more of the multiple sample indicators can be used as statistical indicators. For example, head status can be used as a statistical indicator. Another example is gender. Yet another example is that both head status and gender are statistical indicators.

[0056] Table 1

[0057]

[0058] In some embodiments of this disclosure, the statistical indicators may be selected by those skilled in the art or by statisticians from multiple sample indicators.

[0059] For step S20, for example, based on the list of indicator values ​​for a statistical indicator, a first statistical array corresponding to the statistical indicator is created. The list of indicator values ​​contains all the indicator values ​​for that statistical indicator.

[0060] For example, the statistical indicator is the head state. Based on the list of indicator values ​​for the head state, which includes 1, 2, and 3, a first statistical array res1 corresponding to the head state is created. The first statistical array res1 includes 3 elements, namely res1[0], res1[1], and res1[2].

[0061] For example, the statistical indicators are head status and gender. Based on the list of indicator values ​​for head status, which includes 1, 2, and 3, and the list of indicator values ​​for gender, which includes 1 and 2, a first statistical array res2 corresponding to the statistical indicators head status and gender is created. The first statistical array res2 includes 5 elements, namely res2[0], res2[1], res2[2], res2[3], and res2[4].

[0062] In some embodiments of this disclosure, the first statistical array includes multiple first elements, and the number of first elements may be the same as the number of index values ​​in the index value list of the statistical index, so that the multiple first elements correspond one-to-one with the multiple index values ​​of the statistical index.

[0063] For example, in the case where the above statistical indicators are in the form of head counts, res1[0] corresponds to indicator value 1, res1[1] corresponds to indicator value 2, and res1[2] corresponds to indicator value 3. Multiple first elements res1[0], res1[1], and res1[2] are used to count indicator value 1, indicator value 2, and indicator value 3, respectively. That is, res1[0] is used to count the number of people passing by the screen, res1[1] is used to count the number of people touching the screen, and res1[2] is used to count the number of people paying attention to the screen.

[0064] For example, in the case of the above statistical indicators being head status and gender, res2[0], res2[1], and res2[2] correspond to the index values ​​1, 2, and 3 of the statistical indicator "head status," respectively, while res2[3] and res2[4] correspond to the index values ​​1 and 2 of the statistical indicator "gender," respectively. Multiple first elements res2[0], res2[1], res2[2], res2[3], and res2[4] are used to count the index values ​​1, 2, and 3 of head status, and the index values ​​1 and 2 of gender, respectively. That is, res2[0] is used to count the number of people passing by the screen, res2[1] is used to count the number of people touching the screen, res2[2] is used to count the number of people paying attention to the screen, res2[3] is used to count the number of men, and res2[4] is used to count the number of women.

[0065] For step S30, the multiple first elements in the first statistical array are the statistical sub-results for each index value.

[0066] For example, in the case where the statistical indicator is the head state, the indicator values ​​of the head state in Table 1 are iterated through, and the first elements res[0], res[1], and res[2] in the first statistical array res are used to count the number of indicator values ​​1, 2, and 3 respectively, to obtain the data statistics results. The data statistics result is, for example, res[29,42,96], that is, the statistical sub-result of indicator value 1 is 29, the statistical sub-result of indicator value 2 is 42, and the statistical sub-result of indicator value 3 is 96. For the case where the statistical indicator is the head state and gender, step S30 is similar to the case where the statistical indicator is the head state, and will not be repeated here.

[0067] For example, given the statistical results, the elements in the first statistical array can be split, and an interval statistical result table can be generated based on the elements in the first statistical array and their corresponding indicator values. Table 2 below illustrates the interval statistical result table obtained by statistically analyzing the sample data collected from 15:00 to 16:00 on March 30, 2021.

[0068] Table 2

[0069]

[0070] As shown in Table 2, the number of people who passed by the screen, the number of people who touched the screen, the number of people who followed the screen, the number of males and the number of females were obtained by using the first statistical data res2 of the above embodiment between 15:00:00 and 16:00:00 on March 30, 2021.

[0071] In some embodiments of this disclosure, for example, a statistical function y = f(col, res_list) can be defined, where col is a specified field name (i.e., the name of the statistical indicator), res_list is a list of values ​​for that field (i.e., a list of indicator values ​​for the statistical indicator), and the return value y is an array of length res_list. For example, running this statistical function can execute the above-mentioned reference. Figure 1B Steps S20 and S30 are described below. In this embodiment, those skilled in the art or statisticians only need to input the name of the statistical indicator to be statistically analyzed and a list of its values ​​to call the statistical function to perform the statistical analysis. Therefore, the data processing in this embodiment can achieve high reusability.

[0072] In some embodiments of this disclosure, step S30 may involve iterating through the index values ​​of at least one sample data within the statistical group and using multiple elements in the first statistical array to count each index value in the statistical group, so as to obtain the data statistical results of each index value within the statistical group.

[0073] For example, in the scenario where the statistical indicator is the head state, the first element of each element in the first statistical array res is first initialized to 0. Then, in response to the acquisition of sample data, the indicator value of the head state of the sample data is judged: if the indicator value of the head state of the sample data is 1, then res[0]+=1; if the indicator value of the head state of the sample data is 2, then res[1]+=1; and if the indicator value of the head state of the sample data is 3, then res[2]+=1. Thus, the statistical results of the data in the statistical group are obtained, that is, the statistical sub-results of each indicator value in the statistical group are obtained.

[0074] In some embodiments of this disclosure, the sample data can be grouped according to a certain sample metric to obtain each group data logic block. Then, within each group data logic block, statistics are performed using other sample metrics as statistical indicators. For example, the sample data can be grouped according to the camera ID, or each camera can have its own group data logic block. Then, for each branch data logic block, statistics are performed using head status and / or gender as statistical indicators.

[0075] In some embodiments of this disclosure, step S10 includes acquiring multiple sample data corresponding to each statistical group within multiple statistical groups; and step S30 includes, for each statistical group, traversing the index values ​​of at least one sample data, and using a first statistical array to perform statistics on at least one sample data to obtain data statistical results.

[0076] For example, step S10 may involve acquiring sample data collected within each of multiple preset time periods. For instance, if the preset time period is one hour, sample data is acquired sequentially for each hour, using one hour as the statistical unit. For example, if the current time is 16:32 on March 30, 2021, then sample data collected between 15:00 and 16:00 on March 30, 2021 is acquired; for times between 15:00 and 16:00 on March 30, 2021, sample data collected between 14:00 and 15:00 is acquired; and so on, acquiring sample data collected within each of the multiple preset time periods. In this embodiment, step S10 involves sequentially acquiring hourly sample data.

[0077] In this embodiment, for step S30, for the sample data of each hour, the index values ​​of at least one sample data are iterated, and the first statistical array is used to perform statistics on at least one sample data to obtain the data statistics results for that hour.

[0078] In some embodiments of this disclosure, for example, for sample data within each preset time period, statistics are performed at a certain time in the next preset time period that follows and is adjacent to the preset time period. For example, for sample data from 15:00 to 16:00 on March 30, 2021, statistics are performed at a certain time between 16:00 and 17:00 to obtain the data statistics results.

[0079] Figure 2 A flowchart of another data processing method provided by at least one embodiment of the present disclosure is shown.

[0080] like Figure 2 As shown, this data processing method includes Figure 1B Based on steps S10 to S30, steps S40 to S80 may also be included.

[0081] Step S40: Determine at least one indicator value to be statistically analyzed from the different indicator values.

[0082] For example, select one or more of the following from Table 2: number of people passing by the screen, number of people touching the screen, number of people watching the screen, number of men, and number of women, as the statistical indicator value to be analyzed. For example, the statistical indicator value is the number of people passing by the screen.

[0083] Step S50: Obtain multiple statistical intervals.

[0084] In some embodiments of this disclosure, at least one sample of data obtained within a statistical unit is considered as a statistical group, and each statistical interval includes at least one statistical unit. For example, a statistical unit may be a preset time period or a preset number of terminal devices.

[0085] In some embodiments of this disclosure, the ranges of multiple statistical intervals may increase sequentially. For example, the multiple statistical intervals include a first statistical interval and a second statistical interval, wherein the range of the second statistical interval is larger than the range of the first statistical interval, and the first statistical interval is within the second statistical interval.

[0086] For example, multiple statistical intervals include a first statistical interval, a second statistical interval, and a third statistical interval. The range of the third statistical interval is greater than the range of the second statistical interval, and the second statistical interval is within the third statistical interval. The range of the second statistical interval is greater than the range of the first statistical interval, and the first statistical interval is within the second statistical interval.

[0087] For example, the statistical unit can be a preset time period, and each statistical interval can be at least one consecutive preset time period. For example, if the current time is 16:32 on March 30, 2021, the multiple statistical intervals can be: the daily statistical interval [2021-03-30 00:00:00-2021-03-30 16:00:00), the 7-day statistical interval [2021-03-23 ​​16:00:00-2021-03-30 16:00:00), and the 30-day statistical interval [2021-02-28 16:00:00-2021-03-30 16:00:00]. That is, the multiple statistical intervals represent different time dimensions.

[0088] For example, the statistical unit is a single terminal device, and multiple statistical intervals can be, for example, a specific terminal device, all terminal devices on a site containing that terminal device, and all terminal devices on the line where that site is located.

[0089] Step S60: Establish a second statistical array for multiple statistical intervals. The second statistical array includes multiple second elements, and each of the multiple second elements corresponds one-to-one with the multiple statistical intervals.

[0090] For example, if there are N statistical intervals, where N is an integer greater than or equal to 2, then the number of second elements in the second statistical data can also be N, so that multiple second elements correspond one-to-one with multiple statistical intervals.

[0091] For example, if there are 3 statistical intervals, the second statistical array ges can include 3 elements, namely ges[0], ges[1] and ges[2]. For example, ges[0] corresponds to [2021-03-30 00:00:00-2021-03-30 16:00:00), ges[1] corresponds to [2021-03-23 ​​16:00:00-2021-03-30 16:00:00), and ges[2] corresponds to [2021-02-28 16:00:00-2021-03-30 16:00:00).

[0092] For example, if there are two statistical indicators, namely the number of people passing by the screen and the number of people touching the screen, and the number of statistical intervals is 3, then the second statistical array ges can include 6 elements, namely ges[0], ges[1], ges[2], ges[3], ges[4] and ges[5]. For example, ges[0], ges[1] and ges[2] are used to count the number of people passing by the screen in different time dimensions, and ges[3], ges[4] and ges[5] are used to count the number of people touching the screen in different time dimensions.

[0093] Step S70: Filter the interval statistical results belonging to multiple statistical intervals from the statistical results of the data in each statistical group.

[0094] For example, you can filter the statistical results from the data statistics results to select the data statistics results within the largest statistical interval. For example, in the scenario where the above multiple statistical intervals are the statistical interval for the current day, the statistical interval for 7 days, and the statistical interval for 30 days, you can filter the statistical results from the data statistics results to select the interval statistical results within the 30-day statistical interval [2021-02-28 16:00:00-2021-03-30 16:00:00].

[0095] Step S80: Use multiple second elements to perform statistics on the values ​​of the indicators to be statistically analyzed in the interval statistical results, so as to obtain the statistical results of the values ​​of the indicators to be statistically analyzed in each statistical interval.

[0096] For example, we use ges[0] to count the number of people passing by the screen in [2021-03-30 00:00:00-2021-03-30 16:00:00), ges[1] to count the number of people passing by the screen in [2021-03-23 ​​16:00:00-2021-03-30 16:00:00), and ges[2] to count the number of people passing by the screen in [2021-02-28 16:00:00-2021-03-30 16:00:00), thereby obtaining the number of people passing by the screen in [2021-03-30 00:00:00-2021-03-30 16:00:00), [2021-03-23 The number of people who passed by the screen during the period from 16:00:00 to 2021-03-30 16:00:00, and the number of people who passed by the screen during the period from 2021-02-28 16:00:00 to 2021-03-30 16:00:00.

[0097] Figure 2 The method shown can achieve data statistics across different dimensions with a single data processing step, achieving high parallelism and thus improving data computation efficiency. Furthermore, it only requires updating the value of the indicator to be statistically analyzed; the method can then be applied to the updated indicator value for statistical analysis, thus exhibiting high reusability. This two-stage approach (the first stage utilizes...) Figure 1B The method shown yields statistical results, and the second stage utilizes... Figure 2 The method shown (obtaining statistical results of index values) can reduce data redundancy, achieve data regularization, and effectively avoid performance problems of computing devices caused by instantaneous large-throughput data.

[0098] In some embodiments of this disclosure, the number of second elements included in the second statistical array and the number of first elements included in the first statistical array can be dynamically expanded according to actual needs. For example, the statistical dimension of the second statistical data can be dynamically expanded through parameter configuration according to actual needs.

[0099] Figure 3 At least one embodiment of the present disclosure is shown. Figure 2 The flowchart of step S80.

[0100] like Figure 3 As shown, the data processing method includes steps S81 to S83.

[0101] Step S81: For each interval statistical result, determine the statistical interval to which the statistical group corresponding to the interval statistical result belongs.

[0102] For example, for the interval statistical results shown in Table 2, the preset time period for the statistical group corresponding to this interval statistical result is [2021-03-30 15:00:00-2021-03-30 16:00:00). The statistical interval to which this preset time period belongs includes not only the daily statistical interval [2021-03-30 00:00:00-2021-03-30 16:00:00), but also the 7-day statistical interval [2021-03-23 ​​16:00:00-2021-03-30 16:00:00) and the 30-day statistical interval [2021-02-28 16:00:00-2021-03-30 16:00:00].

[0103] Step S82: Extract the statistical sub-results of the indicator values ​​to be statistically analyzed from the interval statistical results.

[0104] For example, if the statistical indicator to be counted is the number of passersby, the statistical sub-result of the number of passersby is extracted from the statistical results of each interval. For example, the number of passersby extracted from the interval statistical results shown in Table 2 for the statistical interval of the day [2021-03-30 15:00:00-2021-03-30 16:00:00] is 29.

[0105] Step S83: Accumulate the statistical sub-results to the second element corresponding to the statistical interval to which the statistical group to which the interval statistical result belongs, so as to obtain the statistical result of the indicator value to be statistically analyzed for each statistical interval.

[0106] For example, 29 is accumulated into the statistical intervals for the current day [2021-03-30 00:00:00-2021-03-30 16:00:00), the 7-day statistical interval [2021-03-23 ​​16:00:00-2021-03-30 16:00:00), and the 30-day statistical interval [2021-02-28 16:00:00-2021-03-30 16:00:00), thus obtaining the statistical intervals for the current day [2021-03-30 00:00:00-2021-03-30 16:00:00) and the 7-day statistical interval [2021-03-23 ​​16:00:00-2021-03-30]. The statistical results of the number of people passing by during the 16:00:00 period and the 30-day statistical interval [2021-02-28 16:00:00-2021-03-30 16:00:00].

[0107] For example, based on the three statistical intervals, the second statistical array `ges` is an array format containing three second elements. In step S83, each element of ges can be initialized to 0 first, and then the time of the statistical result of the interval can be judged: if the time of the statistical result of the interval is in [2021-03-30 00:00:00-2021-03-30 16:00:00), then res[0]+=num_pass, res[1]+=num_pass, res[2]+=num_pass; if the time of the statistical result of the interval is in [2021-03-23 ​​16:00:00-2021-03-30 16:00:00), then res[1]+=num_pass, res[2]+=num_pass; if the time of the statistical result of the interval is in [2021-02-28 16:00:00-2021-03-30 16:00:00), then res[2]+=num_pass.

[0108] Figure 4 A flowchart of another data processing method provided by at least one embodiment of the present disclosure is shown.

[0109] like Figure 4 As shown, this data processing method includes Figure 1B Based on steps S10 to S30, steps S410 to S420 may also be included. For example, steps S410 and S420 are performed before step S10.

[0110] Step S410: Receive initial data from the data source.

[0111] For example, in Figure 1A In the system architecture shown, the big data backend 103 receives initial data from the business end 102.

[0112] For example, in a subway scenario, the display screen may include a camera. The camera transmits the initial data it collects to the business terminal 102, and then the business terminal 102 transmits the initial data from the camera to the big data backend 103, so as to decentralize the data processing to the big data backend 103 and reduce the pressure on the front-end visualization screen and the business terminal in terms of computing and response time.

[0113] Step S420: Based on the initial data, establish at least one sample dataset.

[0114] For example, the big data backend 103 creates at least one sample dataset based on the initial data.

[0115] In some embodiments of this disclosure, the initial data includes statistical attribute information. The statistical attribute information serves as the information used to store the initial data. For example, if the initial data is stored according to time, the statistical attribute information could be the event time in Table 1; or, if the initial data is stored according to camera ID, the statistical attribute information could be the camera ID, etc.

[0116] For example, the storage file for storing the initial data can be determined based on the preset time period to which the event time in the initial data belongs, and the initial data within each preset time period can be stored in the same storage file. Alternatively, the storage file for storing the initial data can be determined based on the camera ID in the initial data, and the initial data collected by each camera can be stored in the same storage file.

[0117] Figure 5 At least one embodiment of the present disclosure is shown. Figure 4 The flowchart of step S420.

[0118] like Figure 5 As shown, the method may include steps S421 to S423.

[0119] Step S421: Based on the statistical attribute information, determine whether there is a storage file for storing the initial data.

[0120] For example, if the statistical attribute information is the event time, the system determines whether a storage file exists to store the initial data based on the event time of the initial data. For instance, the initial data might be stored in a storage file at the specified path structure: "... / business topic / load_date=YYYY-MM-dd / load_hour=HH". If a storage file exists at the file path corresponding to the event time, then a storage file for storing the initial data exists; otherwise, if no storage file exists at the file path corresponding to the event time, then a storage file for storing the initial data does not exist.

[0121] In some embodiments of this disclosure, initial data belonging to the same statistical group are stored in the same storage file.

[0122] For example, initial data belonging to the same preset time period are stored in the same storage file.

[0123] For example, if the statistical attribute information of the initial data is 15:32 on March 30, 2021, and the preset time period to which 15:32 on March 30, 2021 belongs is 15:00-16:00 on March 30, 2021, then the file path of the storage file storing this initial data is "topic_name / load_date=2021-03-30 / load_hour=15". In step S421, it is determined whether a storage file exists in the file path topic_name / load_date=2021-03-30 / load_hour=15.

[0124] Step S422: In response to the existence of a storage file for storing initial data, store the initial data into the storage file for use as at least one sample data.

[0125] For example, if a storage file exists in the file path topic_name / load_date=2021-03-30 / load_hour=15, then the initial data is stored in the storage file in the file path topic_name / load_date=2021-03-30 / load_hour=15, and thus this initial data is used as sample data.

[0126] Step S423: In response to the absence of a storage file for storing the initial data, determine the statistical group to which the initial data belongs based on the statistical attribute information, generate a file path based on the statistical group, and create a storage file in the file path. The storage file is used to store the initial data so that the initial data can be used as the at least one sample data.

[0127] For example, if no storage file exists in the file path topic_name / load_date=2021-03-30 / load_hour=15, then based on the statistical attribute information 15:32 on March 30, 2021, the statistical group for the initial data is determined to be the preset time period from 15:00 to 16:00 on March 30, 2021. Then, based on the preset time period of the statistical group from 15:00 to 16:00 on March 30, 2021, the file path topic_name / load_date=2021-03-30 / load_hour=15 is generated, and a storage file is created in this file path to store the initial data whose event time falls within the preset time period of 15:00 to 16:00 on March 30, 2021. This initial data is then used as sample data.

[0128] According to the above embodiments, initial data belonging to the same statistical group are stored in the same storage file. This allows the initial data to be divided when it is stored, which facilitates the subsequent retrieval of the initial data and further improves statistical efficiency.

[0129] In some embodiments of this disclosure, for example, the Kafka distributed log system can be used to store initial data from the business side to the big data platform 103 according to the above-described path structure (e.g., it can be stored in the local storage of the big data platform 103, or stored in other storage devices associated with the big data platform 103), so that the big data platform 103 can perform statistical analysis on the sample data. In this embodiment, step S10 may involve retrieving at least one sample data from a storage file that has been persisted by the Kafka distributed log system.

[0130] In some other embodiments of this disclosure, the initial data may also be stored in a database. In this embodiment, step S10 may be retrieving at least one sample data from the database.

[0131] Figure 6 A flowchart of step S10 in FIG1 is shown, provided in at least one embodiment of the present disclosure.

[0132] like Figure 6 As shown, the method may include steps S11 to S13.

[0133] Step S11: Generate the file path of the storage file corresponding to the statistical group.

[0134] For example, if the current time is 16:32 on March 30, 2021, then the sample data within the statistical group from 15:00 to 16:00 on March 30, 2021 will be obtained, based on the above. Figure 5 The storage rules described specify that the file path for storing sample data within the statistical group from 15:00 to 16:00 on March 30, 2021 is topic_name / load_date=2021-03-30 / load_hour=15.

[0135] Step S12: Determine if a file path exists.

[0136] For example, check if the file path topic_name / load_date=2021-03-30 / load_hour=15 exists.

[0137] Step S13: In response to the existence of a file path, retrieve the initial data within the statistical group from the storage file in the file path as at least one sample data.

[0138] For example, in response to the existence of the file path, the initial data within the statistical group is retrieved from the storage file in the file path as at least one sample data.

[0139] In some embodiments of this disclosure, in response to the absence of a file path, the process continues to acquire sample data for the next statistical group or performs subsequent data processing steps.

[0140] In some embodiments of this disclosure, the data processing method is applied to multiple electronic devices. These multiple electronic devices, for example, constitute the big data backend 103 shown in Figure 1; that is, multiple electronic devices form a server cluster. At least one sample data set includes multiple sample data groups, with each electronic device corresponding one-to-one with a specific sample data group. The multiple electronic devices are configured to perform statistics based on their respective sample data groups and sum the statistical values ​​to obtain the data statistical result. Multiple sample data groups may, for example, refer to sample data stored in different electronic devices.

[0141] For example, at least one sample data point within a statistical group from 15:00 to 16:00 on March 30, 2021, is partially stored on a first server located on multiple electronic devices. This partial sample data constitutes one sample data group. Additionally, at least one sample data point within the statistical group from 15:00 to 16:00 on March 30, 2021, other than the aforementioned partial sample data, is stored on a second server located on multiple electronic devices. This other sample data constitutes another sample data group. The first server performs statistical analysis on the partial sample data to obtain a first statistical result res(a), and the second server performs statistical analysis on the other sample data to obtain a second statistical result res(b). Next, the corresponding first elements in res(a) and res(b) are added together to obtain the statistical result res of the statistical group from 15:00 to 16:00 on March 30, 2021. For example, res(a)[0]+res(b)[0]=res[0], res(a)[1]+res(b)[1]=res[1] and res(a)[2]+res(b)[2]=res[2].

[0142] Figure 7A A flowchart of another data processing method provided by at least one embodiment of the present disclosure is shown.

[0143] like Figure 7A As shown, the data processing method includes steps S701 to S707. In this embodiment, the data processing method is illustrated using a preset time period of 1 hour, that is, a statistical unit of 1 hour.

[0144] Step S701: Generate the file path for the previous hour based on the current time.

[0145] For example, for sample data within each preset time period, statistics are performed at a certain time within the next preset time period that follows and is adjacent to that preset time period. For instance, for sample data from 15:00 to 16:00 on March 30, 2021, statistics are performed at a certain time between 16:00 and 17:00 to obtain the data statistics results. Therefore, if the current time is 16:32 on March 30, 2021, the file path for the previous hour is generated based on the current time; that is, the file path for the sample data from 15:00 to 16:00 on March 30, 2021 is generated. Based on the above path structure, the generated file path would be, for example, topic_name / load_date = 2021-03-30 / load_hour = 15.

[0146] This step S701 and Figure 6 The steps in step S11 are similar.

[0147] Step S702: Determine if the file path exists.

[0148] For example, check if topic_name / load_date=2021-03-30 / load_hour=15 exists.

[0149] If the file path topic_name / load_date=2021-03-30 / load_hour=15 exists, proceed to step S703. If the file path topic_name / load_date=2021-03-30 / load_hour=15 does not exist, proceed to step S705.

[0150] This step S701 and Figure 6 The steps are similar to S12.

[0151] Step S703: Aggregate and preprocess the sample data from the previous hour.

[0152] For example, perform aggregation preprocessing on sample data in the file path topic_name / load_date=2021-03-30 / load_hour=15. Aggregation preprocessing could, for example, involve performing... Figure 1B Steps S20 and S30 are used to obtain statistical results of the statistical indicators of the sample data.

[0153] The above steps S701 to S703 can yield hourly statistical results.

[0154] Step S704: Store the hourly data statistics results in the data warehouse. For example, store the data statistics results in the data warehouse according to the format of Table 2 above.

[0155] Step S705: Determine whether there are any data records within the statistical interval.

[0156] For example, if there are multiple statistical intervals, and the ranges of these intervals can increase sequentially, then step S705 could be to determine whether there are data records within the largest statistical interval. For instance, it could determine whether there are statistical results in the data warehouse that belong to the largest statistical interval.

[0157] If there are data records within the largest statistical interval, then the interval statistical results belonging to the largest statistical interval can be filtered from the data statistical results within each statistical group, and step S706 can be executed.

[0158] If there are no data records in the largest statistical interval, a message indicating no data records will be returned, and the data processing method will then end.

[0159] Step S706: Perform parallel processing of interval statistical results by hierarchical aggregation according to the time dimension.

[0160] For example, execute the above reference Figure 3 Steps S81 to S83 describe the parallel processing of the interval statistical results by hierarchical aggregation according to the time dimension to obtain the statistical results of the indicator values ​​for each time dimension. The time dimension includes, for example, the statistical interval for the current day [2021-03-30 00:00:00-2021-03-30 16:00:00), the statistical interval for 7 days [2021-03-23 ​​16:00:00-2021-03-30 16:00:00), and the statistical interval for 30 days [2021-02-28 16:00:00-2021-03-30 16:00:00].

[0161] Step S707: Insert the statistical results of the indicator values ​​into the summary data table.

[0162] For example, the statistical results of the indicators to be analyzed can be inserted into a summary data table. For instance, the summary data table can include the individual statistical results of different indicators. Summarizing the statistical results of each indicator in the summary data table facilitates comparison and analysis by statisticians.

[0163] Figure 7B A schematic diagram of another data processing method provided by at least one embodiment of the present disclosure is shown.

[0164] like Figure 7BAs shown, the data processing method includes steps S710 to S730. Step S710 is executed, for example, by a first server, and step S720 is executed, for example, by a second server.

[0165] Step S710: The first server processes the sample data in the first sample data group within the statistical group according to... Figure 1B The method shown is used for data processing. For example, the first statistical array res is used to statistically analyze the statistical values ​​of the sample data in the first sample data group. For example, if the list of statistical values ​​contains 3 values, then the first statistical array res can contain 3 first elements, namely res[0], res[1], and res[2]. The 3 first elements are used to statistically analyze the 3 values ​​to obtain the first data statistical results, that is, to obtain the statistical sub-results of each value of the first sample data group. For example, before using the first statistical array res to statistically analyze the statistical values ​​of the sample data in the first sample data group, the first statistical array res is initialized so that each first element is 0.

[0166] Step S720: The second server processes the sample data of the second sample data group within the statistical group according to... Figure 1B The method described above is used for data processing. For example, the first statistical array `res` is used to statistically analyze the indicator values ​​of the sample data in the second sample data group to obtain the second data statistical results. For example, before using the first statistical array `res` to statistically analyze the indicator values ​​of the sample data in the second sample data group, the first statistical array `res` is initialized so that each element is 0. It should be noted that the processing method of the second server is basically the same as that of the first server, the difference being that they deal with different specific sample data. For example, the second server processes the second sample data group, while the first server processes the first sample data group.

[0167] The statistical groups in steps S710 and S720 are the same, for example, both consisting of sample data obtained between 15:00 and 16:00 on March 30, 2021. Sample data from the same statistical group may be stored on different servers. Therefore, for statistical analysis of sample data from the same statistical group, it is necessary to calculate the statistical results of the sample data belonging to the same statistical group from different servers to obtain the statistical results for that same statistical group.

[0168] Step S730: Add the corresponding first element in the first data statistical result and the second data statistical result to obtain the data statistical result 700. For example, add res[0] obtained by the first server and res[0] obtained by the second server to obtain res[0] of the data statistical result 700. Similarly, add res[1] obtained by the first server and res[1] obtained by the second server to obtain res[1] of the data statistical result 700. Add res[2] obtained by the first server and res[2] obtained by the second server to obtain res[2] of the data statistical result 700.

[0169] Figure 8 A schematic block diagram of a data processing apparatus 800 provided in at least one embodiment of the present disclosure is shown.

[0170] For example, such as Figure 8 As shown, the data processing device 800 includes a sample acquisition unit 810, an array creation unit 820, and a traversal unit 830.

[0171] The sample acquisition unit 810 is configured to acquire at least one sample data within a statistical group, each sample data including a statistical indicator and the indicator value of the statistical indicator.

[0172] The sample acquisition unit 810 can, for example, perform... Figure 1B Step S10 is described.

[0173] The array creation unit 820 is configured to create a first statistical array corresponding to the statistical indicators. The first statistical array includes a plurality of first elements, which are used to perform statistics on different indicator values.

[0174] Array creation unit 820, for example, can be executed Figure 1B Step S20 is described.

[0175] The traversal unit 830 is configured to traverse the index values ​​of the at least one sample data and perform statistics on the at least one sample data using the first statistical array to obtain data statistical results, wherein the multiple first elements in the first statistical array are the respective statistical sub-results of each index value.

[0176] Traversing unit 830, for example, can be executed Figure 1B Step S30 is described.

[0177] For example, the sample acquisition unit 810, the array creation unit 820, and the traversal unit 830 can be hardware, software, firmware, or any feasible combination thereof. For example, the sample acquisition unit 810, the array creation unit 820, and the traversal unit 830 can be dedicated or general-purpose circuits, chips, or devices, or they can be a combination of a processor and memory. The embodiments of this disclosure do not limit the specific implementation of the above-mentioned units.

[0178] It should be noted that in the embodiments of this disclosure, each unit of the data processing device 800 corresponds to each step of the aforementioned data processing method. For the specific functions of the data processing device 800, please refer to the relevant description of the data processing method, which will not be repeated here. Figure 8 The components and structures of the data processing apparatus 800 shown are merely exemplary and not limiting. The data processing apparatus 800 may also include other components and structures as needed.

[0179] At least one embodiment of this disclosure also provides an electronic device including a processor and a memory, the memory including one or more computer program modules. The one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules include instructions for implementing the data processing method described above. This electronic device can utilize arrays to achieve parallel statistics of multiple indicator values ​​through a single data processing step, improving statistical efficiency and exhibiting high reusability.

[0180] Figure 9 This is a schematic block diagram of an electronic device provided for some embodiments of this disclosure. For example... Figure 9 As shown, the electronic device 900 includes a processor 910 and a memory 920. The memory 920 stores non-transitory computer-readable instructions (e.g., one or more computer program modules). The processor 910 executes the non-transitory computer-readable instructions, which, when executed by the processor 910, can perform one or more steps of the data processing method described above. The memory 920 and the processor 910 can be interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0181] For example, processor 910 may be a central processing unit (CPU), a graphics processing unit (GPU), or other form of processing unit with data processing and / or program execution capabilities. For example, the central processing unit (CPU) may be an x86 or ARM architecture. Processor 910 may be a general-purpose processor or a special-purpose processor, capable of controlling other components in electronic device 900 to perform desired functions.

[0182] For example, memory 920 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and processor 910 may run one or more computer program modules to implement various functions of electronic device 900. Various application programs and various data, as well as various data used and / or generated by the application programs, may also be stored in the computer-readable storage medium.

[0183] It should be noted that, in the embodiments of this disclosure, the specific functions and technical effects of the electronic device 900 can be referred to the description of the data processing method above, and will not be repeated here.

[0184] Figure 10 This is a schematic block diagram of another electronic device provided in some embodiments of this disclosure. The electronic device 1000 is, for example, suitable for implementing the data processing method provided in the embodiments of this disclosure. The electronic device 1000 may be a terminal device, etc. It should be noted that... Figure 10 The illustrated electronic device 1000 is merely an example and does not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0185] like Figure 10 As shown, the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 1010, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1020 or a program loaded from a storage device 1080 into a random access memory (RAM) 1030. The RAM 1030 also stores various programs and data required for the operation of the electronic device 1000. The processing device 1010, ROM 1020, and RAM 1030 are interconnected via a bus 1040. An input / output (I / O) interface 1050 is also connected to the bus 1040.

[0186] Typically, the following devices can be connected to the I / O interface 1050: input devices 1060 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1070 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1080 including, for example, magnetic tape, hard disk, etc.; and communication devices 1090. Communication device 1090 allows electronic device 1000 to communicate wirelessly or wiredly with other electronic devices to exchange data. Although Figure 10 An electronic device 1000 with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown, and the electronic device 1000 may alternatively implement or have more or fewer devices.

[0187] For example, according to embodiments of this disclosure, the above-described data processing method can be implemented as a computer software program. For instance, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program including program code for performing the above-described data processing method. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 1090, or installed from a storage device 1080, or installed from a ROM 1020. When the computer program is executed by the processing device 1010, the functions defined in the data processing method provided in embodiments of this disclosure can be implemented.

[0188] At least one embodiment of this disclosure also provides a computer-readable storage medium for storing non-transitory computer-readable instructions that, when executed by a computer, can implement the aforementioned data processing method. Using this computer-readable storage medium, parallel statistics of multiple indicator values ​​can be achieved through a single data processing operation using an array, improving statistical efficiency and providing high reusability.

[0189] Figure 11 This is a schematic diagram of a storage medium provided for some embodiments of this disclosure. For example... Figure 11 As shown, storage medium 1100 is used to store non-transitory computer-readable instructions 1110. For example, when the non-transitory computer-readable instructions 1110 are executed by a computer, one or more steps in the data processing method described above can be performed.

[0190] For example, the storage medium 1100 can be used in the aforementioned electronic device 900. For example, the storage medium 1100 can be... Figure 9 The memory 920 in the illustrated electronic device 900. For example, a description of the storage medium 1100 can be found here. Figure 9The corresponding description of the memory 920 in the illustrated electronic device 900 will not be repeated here.

[0191] The following points need to be explained:

[0192] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0193] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0194] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. The scope of protection of this disclosure should be determined by the scope of protection of the claims.

Claims

1. A data processing method, comprising: Obtain at least one sample data within the statistical group, wherein each sample data includes a statistical indicator and the indicator value of the statistical indicator; Create a first statistical array corresponding to the statistical indicators, wherein the first statistical array includes multiple first elements, each of which is used to statistically analyze different indicator values; and The system iterates through the indicator values ​​of the at least one sample data set and performs statistical analysis on the at least one sample data set using the first statistical array to obtain statistical results. The multiple first elements in the first statistical array represent the statistical sub-results for each indicator value. Obtaining at least one sample data within the statistical group includes: Obtain multiple sample data points for each statistical group within multiple statistical groups; Iterate through the indicator values ​​of the at least one sample data, and use the first statistical array to perform statistics on the at least one sample data to obtain the data statistical results, including: For each statistical group, the index values ​​of the at least one sample data are traversed, and the at least one sample data is statistically analyzed using the first statistical array to obtain the data statistical results.

2. The method according to claim 1, wherein, The method further includes: taking at least one sample of data obtained within a statistical unit as a statistical group; and considering the data as such. Determine at least one statistical indicator value from the different indicator values; Obtain multiple statistical intervals, wherein each statistical interval includes at least one of the statistical units; A second statistical array is established for the plurality of statistical intervals, wherein the second statistical array includes a plurality of second elements, and the plurality of second elements correspond one-to-one with the plurality of statistical intervals; and Filter the interval statistical results belonging to the multiple statistical intervals from the statistical results of the data within each statistical group; The statistical results of the target indicator values ​​in the interval statistical results are statistically analyzed using the multiple second elements respectively, so as to obtain the statistical results of the target indicator values ​​in each statistical interval.

3. The method according to claim 2, wherein, The statistical results of the interval statistical results are obtained by using the plurality of second elements to statistically analyze the values ​​of the indicators to be statistically analyzed in each statistical interval, including: For each interval statistical result, determine the statistical interval to which the statistical group corresponding to the interval statistical result belongs; Extract statistical sub-results from the interval statistical results to obtain the values ​​of the indicator to be statistically analyzed; and The statistical sub-results are accumulated and added to the second element corresponding to the statistical interval to which the statistical group to which the interval statistical result belongs, so as to obtain the statistical result of the indicator value to be statistically analyzed for each statistical interval.

4. The method according to any one of claims 1-3, wherein, Traverse the index values ​​of at least one sample data within the statistical group, and use the first statistical array to perform statistics on the at least one sample data in the statistical group to obtain the data statistical results within the statistical group, including: The index values ​​of at least one sample data in the statistical group are traversed, and multiple elements in the first statistical array are used to count each index value in the statistical group to obtain the data statistical results of the statistical group.

5. The method according to claim 2 or 3, wherein, The multiple statistical intervals include a first statistical interval and a second statistical interval. The range of the second statistical interval is greater than the range of the first statistical interval, and the first statistical interval is within the second statistical interval.

6. The method according to claim 1, further comprising: Receive initial data from the data source; as well as Based on the initial data, at least one sample data is established.

7. The method according to claim 6, wherein, The initial data includes statistical attribute information. Based on the initial data, the at least one sample data is established, including: Based on the statistical attribute information, determine whether there exists a storage file for storing the initial data; In response to the existence of a storage file for storing the initial data, the initial data is stored in the storage file for use as the at least one sample data; In response to the absence of a storage file for storing the initial data, the statistical group to which the initial data belongs is determined based on the statistical attribute information, a file path is generated based on the statistical group, and the storage file is created in the file path, wherein the storage file is used to store the initial data for use as the at least one sample data. Initial data belonging to the same statistical group are stored in the same storage file.

8. The method according to claim 7, wherein, Obtaining at least one sample data within the statistical group includes: Generate the file path of the storage file corresponding to the statistical group; Determine if the file path exists; and In response to the existence of the file path, the initial data within the statistical group is obtained from the storage file in the file path as the at least one sample data.

9. The method according to claim 2 or 3, wherein, The statistical unit includes at least one terminal device, which may be a preset time period or a preset number.

10. The method according to claim 1, wherein, The data processing method is applied to multiple electronic devices, and the at least one sample data includes multiple sample data groups, with each of the multiple electronic devices corresponding to one of the multiple sample data groups. The multiple electronic devices are configured to perform statistics based on their respective sample data groups, and then sum the statistical values ​​to obtain the data statistics result.

11. A data processing apparatus, comprising: The sample acquisition unit is configured to acquire at least one sample data within the statistical group, wherein each sample data includes a statistical indicator and the indicator value of the statistical indicator; An array creation unit is configured to create a first statistical array corresponding to the statistical indicators, wherein the first statistical array includes multiple first elements, each of which is used to statistically analyze different indicator values; and The traversal unit is configured to traverse the indicator values ​​of the at least one sample data and perform statistics on the at least one sample data using the first statistical array to obtain data statistical results, wherein the plurality of first elements in the first statistical array are statistical sub-results for each indicator value. Obtaining at least one sample data within the statistical group includes: Obtain multiple sample data points for each statistical group within multiple statistical groups; Iterate through the indicator values ​​of the at least one sample data, and use the first statistical array to perform statistics on the at least one sample data to obtain the data statistical results, including: For each statistical group, the index values ​​of the at least one sample data are traversed, and the at least one sample data is statistically analyzed using the first statistical array to obtain the data statistical results.

12. An electronic device, comprising: processor; Memory, which includes one or more computer program instructions; The one or more computer program instructions are stored in the memory and, when executed by the processor, implement the instructions of the data processing method according to any one of claims 1-10.

13. A computer-readable storage medium that non-temporarily stores computer-readable instructions, wherein, The data processing method according to any one of claims 1-10 is implemented when the computer-readable instructions are executed by a processor.

Images