Digital crowd generation method, system, device and storage medium

Abstract

The present application discloses a method, system, device and storage medium for generating a digital crowd. This solution can obtain a large-scale target digital crowd in a more efficient, accurate, flexible, comprehensive and low-cost manner by building a preset label database to mark the digital crowds of various industries. In addition, by utilizing the distributed computing technology of multiple workstations, it can quickly process large-scale data and improve the speed and efficiency of data processing, which is very important for processing massive crowd data and ensuring that data can be acquired and analyzed in a timely manner. At the same time, by adopting the clickhouse database as the preset label database, the columnar storage capability of the clickhouse database makes it possible to store and query massive amounts of data, supporting complex data analysis and query requirements, which provides strong support for big data analysis.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, system, device and storage medium for generating a digital crowd. Background Art

[0002] As people pay more and more attention to digital people, many companies and institutions are actively exploring how to better use big data to improve the authenticity, effectiveness and guidance of teaching scenarios, so as to highly meet the specific requirements of students in future practical work.

[0003] At present, when collecting big data from the population, traditional collection methods are mostly used to simply summarize the basic information of some people. There are defects such as incomplete data coverage, poor quality, legal risks, incomplete data, and small data volume. Especially in the digital business research and training of colleges and universities, there are problems such as high cost of big data population collection and legal risks of personal privacy protection. It is difficult to automatically, flexibly, comprehensively and low-cost obtain relevant data of more than tens of millions of real social people in all walks of life, which makes it difficult to effectively carry out corresponding practical training teaching.

[0004] Therefore, how to automatically generate large-scale digital populations in various industries has become an urgent problem that needs to be solved. Summary of the invention

[0005] The present application aims to at least solve the technical problems existing in the prior art. To this end, the first aspect of the present application proposes a method for generating a digital crowd, which is applied to a system for generating a digital crowd. The system includes a server end and multiple workstation ends. The server end and each workstation end communicate with each other using a preset efficient task distribution and data collection protocol. The method includes:

[0006] The server determines the workload of each workstation based on the preset total workload and the number of currently idle workstations, and enables a working thread for each workstation;

[0007] The server-side performs task division for the scale of the digital population to be constructed, and sends corresponding construction tasks to each workstation based on the workload and work thread of the workstation; wherein the construction task includes a preset total workload and preset related construction parameters, and the preset related construction parameters are obtained from a preset label database, which is a clickhouse database; the preset label database includes multiple industry type characteristics and digital population label groups corresponding to the industry type characteristics, the digital population label group includes multiple target digital population labels, target label feature values ​​corresponding to each target digital population label, and target label distribution probability, the digital population label group is stored in an initial grayscale bitmap of a preset size, the pixel value corresponding to each pixel point in the initial grayscale bitmap is used to represent the target label feature value, the coordinates of the pixel point are used to represent the number of the target digital population label, and the preset size is determined by the number of the target digital population labels;

[0008] Receive the build task through the workstation, generate the target digital population based on the build task, and send the target digital population to the server;

[0009] The target digital population is received through the server and saved.

[0010] In a possible implementation, the process of building a preset tag database includes:

[0011] Obtain initial population data from multiple industries;

[0012] Perform data preprocessing operations on the initial population data to generate target population data; data preprocessing operations include data cleaning, data standardization, data format conversion, data integration and data desensitization;

[0013] Process the target population data to generate multiple industry type characteristics and digital population label groups corresponding to each industry type characteristic;

[0014] Based on multiple industry type characteristics and digital population label groups corresponding to each industry type characteristic, a preset label database is constructed.

[0015] In a possible implementation, the target population data is processed to generate a plurality of industry type features and a digital population label group corresponding to each industry type feature, including:

[0016] Extract and construct labels for the target population data to obtain full labels for people in various industries;

[0017] Analyze and transform all labels of people in various industries to obtain a label pool that can be recognized by the database;

[0018] Generate multiple industry type features corresponding to the target population data based on preset feature recognition rules;

[0019] Organize the tags in the tag pool according to the characteristics of each industry type to obtain multiple digital crowd tag groups;

[0020] The characteristics of each industry type are matched with each digital population label group to obtain a digital population label group corresponding to the characteristics of each industry type.

[0021] In a possible implementation, the method further includes:

[0022] Obtaining initial attribute weights corresponding to each member of the target digital population; wherein each initial attribute weight is determined based on a pixel value corresponding to each pixel point in the initial grayscale bitmap;

[0023] For each crowd member, obtain the current operating environment information in the target business system, and determine the current behavior characteristics of the crowd member based on the current operating environment information; wherein the current operating environment information includes the number of business orders, time information, user status information, commodity market information in the business order, and commodity price fluctuation information;

[0024] The initial attribute weights are updated based on the current behavior characteristics of the crowd members to obtain new attribute weights;

[0025] The initial grayscale bitmap is updated based on the new attribute weights and pixel coordinates to obtain a new grayscale bitmap.

[0026] In a possible implementation, receiving a construction task through a workstation, generating a target digital population based on the construction task, and sending the target digital population to a server includes:

[0027] Obtaining the current control instruction sent by the server through the instruction processing module of the workstation; wherein the current control instruction includes a start instruction;

[0028] Initiate a build task based on the start-up instruction, and generate a target digital population based on the build task through a build execution module on the workstation side;

[0029] The data transmission module based on the workstation side sends the target digital population to the server side.

[0030] In a possible implementation, generating a target digital crowd based on the construction task further includes:

[0031] Based on the construction task, construct the basic attribute information corresponding to each member of the population;

[0032] For each crowd member, obtain the preset tag association relationship corresponding to the crowd member;

[0033] Use a preset binary search algorithm to find and obtain the label values ​​and probability distribution of all target associated labels corresponding to the preset label association relationship;

[0034] Based on the target associated tags and the preset attribute addition rules, the basic attribute information is subjected to attribute addition processing to generate the target attribute information; wherein the preset attribute addition rules include full addition rules or mutually exclusive addition rules, the full addition rules are used to add the label values ​​and probability distributions of all target associated tags to the crowd members, and the mutually exclusive addition rules are used to add the label values ​​and probability distributions of the target associated tags to the crowd members according to the preset addition quantity;

[0035] Based on the target attribute information, the target digital population is constructed.

[0036] In a possible implementation, sending the target digital population to the server includes:

[0037] When the number of population members of the target digital population reaches a preset number threshold, the data corresponding to the target digital population is written into the target file;

[0038] The target file is segmented according to the preset file segment size to obtain multiple segment files;

[0039] Each segment file is transmitted to the server side respectively, and each segment file is marked with a preset marking field; wherein the preset marking field is used to indicate whether the segment file has been transmitted.

[0040] The second aspect of the present application proposes a digital crowd generation system, characterized in that the system includes a server end and multiple workstation ends, and the server end and each workstation end communicate with each other using a preset efficient task distribution and data collection protocol, wherein:

[0041] The server is used to determine the workload of each workstation based on the preset total workload and the number of currently idle workstations, and enable a working thread for each workstation;

[0042] The server side is also used to divide the tasks according to the scale of the digital population to be constructed, and send corresponding construction tasks to each workstation based on the workload and work thread of the workstation; wherein the construction task includes a preset total workload and preset related construction parameters, and the preset related construction parameters are obtained from a preset label database, which is a clickhouse database; the preset label database includes multiple industry type characteristics and digital population label groups corresponding to the industry type characteristics, the digital population label group includes multiple target digital population labels, target label feature values ​​corresponding to each target digital population label and target label distribution probability, the digital population label group is stored in an initial grayscale bitmap of a preset size, the pixel value corresponding to each pixel point in the initial grayscale bitmap is used to represent the target label feature value, the coordinates of the pixel point are used to represent the number of the target digital population label, and the preset size is determined by the number of the target digital population labels;

[0043] The workstation side is used to receive the build task, generate the target digital population based on the build task, and send the target digital population to the server side;

[0044] The server side is also used to receive the target digital population and save the target digital population.

[0045] In a possible implementation, the digital crowd generation system is further used to:

[0046] Obtain initial population data from multiple industries;

[0047] Perform data preprocessing operations on the initial population data to generate target population data; data preprocessing operations include data cleaning, data standardization, data format conversion, data integration and data desensitization;

[0048] Process the target population data to generate multiple industry type characteristics and digital population label groups corresponding to each industry type characteristic;

[0049] Based on multiple industry type characteristics and digital population label groups corresponding to each industry type characteristic, a preset label database is constructed.

[0050] In a possible implementation, the digital crowd generation system is further used to:

[0051] Extract and construct labels for the target population data to obtain full labels for people in various industries;

[0052] Analyze and transform all labels of people in various industries to obtain a label pool that can be recognized by the database;

[0053] Generate multiple industry type features corresponding to the target population data based on preset feature recognition rules;

[0054] Organize the tags in the tag pool according to the characteristics of each industry type to obtain multiple digital crowd tag groups;

[0055] The characteristics of each industry type are matched with each digital population label group to obtain a digital population label group corresponding to the characteristics of each industry type.

[0056] In a possible implementation, the digital crowd generation system is further used to:

[0057] Obtaining initial attribute weights corresponding to each member of the target digital population; wherein each initial attribute weight is determined based on a pixel value corresponding to each pixel point in the initial grayscale bitmap;

[0058] For each crowd member, obtain the current operating environment information in the target business system, and determine the current behavior characteristics of the crowd member based on the current operating environment information; wherein the current operating environment information includes the number of business orders, time information, user status information, commodity market information in the business order, and commodity price fluctuation information;

[0059] The initial attribute weights are updated based on the current behavior characteristics of the crowd members to obtain new attribute weights;

[0060] The initial grayscale bitmap is updated based on the new attribute weights and pixel coordinates to obtain a new grayscale bitmap.

[0061] In a possible implementation manner, the workstation is further used for:

[0062] Obtaining the current control instruction sent by the server through the instruction processing module of the workstation; wherein the current control instruction includes a start instruction;

[0063] Initiate a build task based on the start-up instruction, and generate a target digital population based on the build task through a build execution module on the workstation side;

[0064] The data transmission module based on the workstation side sends the target digital population to the server side.

[0065] In a possible implementation manner, the workstation is further used for:

[0066] Based on the construction task, construct the basic attribute information corresponding to each member of the population;

[0067] For each crowd member, obtain the preset tag association relationship corresponding to the crowd member;

[0068] Use a preset binary search algorithm to find and obtain the label values ​​and probability distribution of all target associated labels corresponding to the preset label association relationship;

[0069] Based on the target associated tags and the preset attribute addition rules, the basic attribute information is subjected to attribute addition processing to generate the target attribute information; wherein the preset attribute addition rules include full addition rules or mutually exclusive addition rules, the full addition rules are used to add the label values ​​and probability distributions of all target associated tags to the crowd members, and the mutually exclusive addition rules are used to add the label values ​​and probability distributions of the target associated tags to the crowd members according to the preset addition quantity;

[0070] Based on the target attribute information, the target digital population is constructed.

[0071] In a possible implementation manner, the workstation is further used for:

[0072] When the number of population members of the target digital population reaches a preset number threshold, the data corresponding to the target digital population is written into the target file;

[0073] The target file is segmented according to the preset file segment size to obtain multiple segment files;

[0074] Each segment file is transmitted to the server side respectively, and each segment file is marked with a preset marking field; wherein the preset marking field is used to indicate whether the segment file has been transmitted.

[0075] The third aspect of the present application proposes an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method for generating a digital crowd as described in the first aspect.

[0076] The fourth aspect of the present application proposes a computer-readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the method for generating a digital crowd as described in the first aspect.

[0077] The embodiments of the present application have the following beneficial effects:

[0078] The method for generating a digital crowd provided by the embodiment of the present application includes: determining the workload of each workstation based on the preset total workload and the number of currently idle workstations through the server side, and enabling a work thread for each workstation, dividing the task of the scale of the digital crowd to be constructed through the server side, and sending the corresponding construction task to each workstation based on the workload and work thread of the workstation, receiving the construction task through the workstation side, generating a target digital crowd based on the construction task, and sending the target digital crowd to the server side, receiving the target digital crowd through the server side, and saving the target digital crowd. This scheme can obtain a large-scale target digital crowd more efficiently, more accurately, more flexibly, more comprehensively, and at a lower cost by constructing a preset tag database to mark the digital crowds of various industries; in addition, by utilizing the distributed computing technology of multiple workstations, large-scale data can be processed quickly, and the speed and efficiency of data processing can be improved, which is very important for processing massive crowd data, ensuring that data can be obtained and analyzed in a timely manner; at the same time, by adopting the clickhouse database as the preset tag database, the column storage capability of the clickhouse database makes it possible to store and query massive data, supporting complex data analysis and query requirements, which provides strong support for big data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 A block diagram of a computer device provided in an embodiment of the present application;

[0080] Figure 2 A flowchart of a method for generating a digital crowd provided in an embodiment of the present application;

[0081] Figure 3 A flowchart of the steps for constructing a preset tag database provided in an embodiment of the present application;

[0082] Figure 4 A flowchart of the steps for processing target population data provided in an embodiment of the present application;

[0083] Figure 5 A flowchart of the steps for updating the initial attribute weights of each group member provided in an embodiment of the present application;

[0084] Figure 6 A flowchart of the steps for constructing a target digital population provided in an embodiment of the present application;

[0085] Figure 7 Another flowchart of steps for constructing a target digital population provided in an embodiment of the present application;

[0086] Figure 8 A flowchart of the steps of sending a target digital population is provided in an embodiment of the present application. DETAILED DESCRIPTION

[0087] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0088] At present, when collecting big data crowds, traditional collection methods are often used to simply summarize some basic information of the crowds. There are defects such as incomplete data coverage, poor quality, legal risks, incomplete data, and small data volume. Especially in the digital business research and training of colleges and universities, there are problems such as high cost of big data crowd collection and legal risks of personal privacy protection. It is difficult to automatically, flexibly, comprehensively and low-cost obtain relevant data of real social crowds of more than 10 million levels in all walks of life, which makes it difficult to effectively carry out corresponding practical training. In addition, the existing big data digital crowds are used in a small number of simulated people live broadcast, simulated people hosting, and simulated people teaching. It is impossible to support the relevant practical training environment of large-scale simulated digital crowds of digital business in various colleges and universities across the country. There is a lack of simulation of different user types and the embodiment of complex business strategies. In the field of big data crowd data modeling and analysis, the existing models may be relatively single and cannot adapt well to the diversified business needs and complex data structures of digital business teaching and research in various industries. In addition, in the field of practical training in colleges and universities, it is currently difficult to efficiently, widely, flexibly, comprehensively and low-cost obtain relevant data of real social crowds of more than 10 million levels in all walks of life, which makes it difficult to effectively carry out corresponding practical training. In particular, there is a lack of support from large-scale population data from various industries in the teaching and training of thousands of majors in more than 10,000 secondary vocational schools, higher vocational schools, vocational undergraduate schools, and undergraduate schools across the country, making it difficult to carry out practical training in digital business professional skills at the secondary vocational schools, higher vocational schools, undergraduate schools, graduate schools, and doctoral schools.

[0089] Based on this, the present application proposes a method for generating a digital crowd, which includes: determining the workload of each workstation based on the preset total workload and the number of currently idle workstations through the server, and enabling a work thread for each workstation, dividing the task of the scale of the digital crowd to be constructed through the server, and sending the corresponding construction task to each workstation based on the workload and work thread of the workstation, receiving the construction task through the workstation, generating a target digital crowd based on the construction task, and sending the target digital crowd to the server, receiving the target digital crowd through the server, and saving the target digital crowd. This scheme can obtain a large-scale target digital crowd more efficiently, more accurately, more flexibly, more comprehensively and at a lower cost by constructing a preset tag database to mark the digital crowds of various industries; in addition, by utilizing the distributed computing technology of multiple workstations, large-scale data can be processed quickly, and the speed and efficiency of data processing can be improved, which is very important for processing massive crowd data, ensuring that data can be obtained and analyzed in a timely manner; at the same time, by adopting the clickhouse database as the preset tag database, the column storage capability of the clickhouse database enables the storage and query of massive data, supporting complex data analysis and query requirements, which provides strong support for big data analysis.

[0090] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "multiple" means two or more. In addition, the use of "based on" or "according to" means openness and inclusiveness, because the process, steps, calculations or other actions "based on" or "according to" one or more of the conditions or values ​​may be based on additional conditions or values ​​beyond the described values ​​in practice.

[0091] The method for generating a digital crowd provided in the present application can be applied to a computer device (electronic device), which can be a server or a terminal. The server can be a single server or a server cluster composed of multiple servers. The embodiments of the present application do not specifically limit this. The terminal can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices.

[0092] Take the computer device as a server as an example. Figure 1 A block diagram of a server is shown, such as Figure 1As shown, the server may include a processor and a memory connected via a system bus. The processor of the server is used to provide computing and control capabilities. The memory of the server includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the computer program is executed by the processor, a method for generating a digital crowd is implemented.

[0093] Those skilled in the art will understand that Figure 1 The structure shown in the figure is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the server to which the solution of the present application is applied. Optionally, the server may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0094] Figure 2 A flowchart of a method for generating a digital crowd provided in an embodiment of the present application is applied to a system for generating a digital crowd. The system includes a server and multiple workstations. The server and each workstation communicate using a preset efficient task distribution and data collection protocol. Figure 2 As shown, the method comprises the following steps:

[0095] Step 202: The server determines the workload of each workstation based on the preset total workload and the number of currently idle workstations, and enables a work thread for each workstation.

[0096] Among them, the preset total workload is also the preset total workload scale, which can be recorded as s, the current number of idle workstations can be recorded as n, and x is the workload of the last workstation. ,in, To obtain the remainder of the quotient, To discard the quotient and take the remainder, the workload of other workstations is After determining the workload of each workstation, you can enable a worker thread for each workstation.

[0097] Step 204: The server divides the tasks of the digital crowd to be constructed, and sends corresponding construction tasks to each workstation based on the workload and work thread of the workstation.

[0098] The construction task includes a preset total workload and preset related construction parameters, which are obtained from a preset label database. The preset label database includes multiple industry type features and digital population label groups corresponding to each industry type feature, and the digital population label group includes multiple target digital population labels, target label feature values ​​corresponding to each target digital population label, and target label distribution probability.

[0099] The digital crowd label group is stored in an initial grayscale bitmap of a preset size. The pixel values ​​corresponding to each pixel in the initial grayscale bitmap are used to represent the target label feature value, and the coordinates of the pixel points are used to represent the number of the target digital crowd label. The preset size is determined by the number of the target digital crowd labels.

[0100] In the initial grayscale bitmap, each pixel has only one value, which is generally between 0 and 255. This range can already meet the definition of the target label feature value. In addition, the value range of each target label feature value can be predefined as between 0 and 255. The coordinates of the pixel points in the initial grayscale image are defined as the number of the target digital population label, that is, a The bitmap has 2500 pixels, and the labels of a crowd member are usually not a huge amount of data, because even The bitmap has only 1,000,000 pixels. If the value of each pixel is 255, the size of this bitmap is about 976.56 KB. Therefore, optionally, the size to be used for storage and marking can be determined according to the number of target digital crowd labels. For example, the size selected in this application is The initial bitmap can be used.

[0101] In some optional embodiments, the preset label database is a clickhouse database. Among them, due to the huge amount of digital crowd data generated, and because the clickhouse database has the characteristics of distributed capability, simple deployment and maintenance, good query performance and support for bitmap queries, it can be selected as the database used for digital crowd storage and query. The storage of digital crowd data can define a data table with the help of the clickhouse database bitmap query capability, store the target digital crowd label in the corresponding data table, add a bitmap column to each row, and store the user number with the same target digital crowd label in a row in bitmap format, that is, to ensure that there are as many data as there are crowd sizes. At the same time, because the bitmap query has good performance for tens of millions or even billions of data queries, the clickhouse bitmap-specific function can be used to merge and filter data.

[0102] In some optional embodiments, the preset tag database can be pre-built, such as Figure 3 As shown, Figure 3 A flowchart of the steps of building a preset tag database provided in an embodiment of the present application includes:

[0103] Step 302: Obtain initial population data of multiple industries.

[0104] Step 304: perform data preprocessing on the initial population data to generate target population data.

[0105] Step 306: Process the target population data to generate multiple industry type features and digital population label groups corresponding to the industry type features.

[0106] Step 308: construct a preset tag database based on multiple industry type characteristics and digital population tag groups corresponding to each industry type characteristic.

[0107] Among them, the initial population data of multiple industries can be obtained by using database queries, web crawlers, etc., which can include user population behavior data, product population data, and population data obtained on various social media platforms.

[0108] Next, the initial population data can be subjected to data preprocessing operations to generate target population data. Optionally, the data preprocessing operations may include data cleaning, data standardization, data format conversion, data integration and data desensitization, etc. The specific implementation process can refer to the existing technology and will not be repeated here.

[0109] In this way, the target population data can be processed to generate multiple industry type characteristics and digital population label groups corresponding to each industry type characteristic. Finally, a preset label database can be constructed based on multiple industry type characteristics and digital population label groups corresponding to each industry type characteristic.

[0110] In some optional embodiments, Figure 4 As shown, Figure 4 A flowchart of the steps for processing target population data provided in an embodiment of the present application includes:

[0111] Step 402: extract and construct labels for the target population data to obtain full labels for people in various industries.

[0112] Step 404: parse and transform all labels of people in various industries to obtain a label pool that can be recognized by the database.

[0113] Step 406: Generate multiple industry type features corresponding to the target population data based on preset feature recognition rules.

[0114] Step 408: Organize the tags in the tag pool according to the characteristics of each industry type to obtain multiple digital crowd tag groups.

[0115] Step 410: Match the characteristics of each industry type with each digital population tag group to obtain a digital population tag group corresponding to each industry type characteristic.

[0116] Among them, optionally, the target population data can be extracted and constructed with labels according to pre-written rules, so as to obtain the full labels of the population in each industry. Then, the full labels of the population in each industry can be converted into a script file recognizable by the database through a pre-written data packet parsing program and imported into the corresponding structure table space, so as to standardize the labels into label pool data recognizable by the database for subsequent work.

[0117] Next, multiple industry type features corresponding to the target population data can be automatically generated based on the built-in preset feature recognition rules. At the same time, the tags in the tag pool can be organized according to the characteristics of each industry type to obtain multiple digital population tag groups. The tag management system can also be used to manage and organize multiple digital population tag groups.

[0118] Therefore, the characteristics of each industry type can be matched with each digital population label group to obtain the digital population label group corresponding to the characteristics of each industry type.

[0119] In this embodiment, by personalizing and organizing the full set of labels for people in various industries to show the characteristics and needs of different industries and classifying and associating the full set of label systems in a targeted manner, a digital human label system that adapts to the characteristics of each industry can be formed, which facilitates the subsequent generation of digital people.

[0120] In some optional embodiments, since it is necessary to be able to satisfy the effect of the association between the current behavior and the last behavior of the crowd members in the specific business system, it is necessary to design a rolling calculation mechanism for the weight of the crowd basic tag value. Figure 5 As shown, Figure 5 A flowchart of the steps for updating the initial attribute weights of each group member provided in an embodiment of the present application includes:

[0121] Step 502: Obtain the initial attribute weight corresponding to each member of the target digital crowd.

[0122] Step 504: For each crowd member, obtain the current operating environment information in the target business system, and determine the current behavior characteristics of the crowd member based on the current operating environment information.

[0123] Step 506: Update the initial attribute weights based on the current behavior characteristics of the crowd members to obtain new attribute weights.

[0124] Step 508: Update the initial grayscale bitmap based on the new attribute weights and the coordinates of the pixel points to obtain a new grayscale bitmap.

[0125] Among them, the initial attribute weight is the weight of the target digital population label value of the population member, and each initial attribute weight is determined based on the pixel value corresponding to each pixel point in the initial grayscale bitmap. After obtaining the initial attribute weight corresponding to each population member in the target digital population, the current operating environment information in the target business system can be obtained for each population member, and the current behavioral characteristics of the population member can be determined based on the current operating environment information. The current operating environment information may include but is not limited to the number of business orders, time information, user status information, commodity market information and commodity price fluctuation information in the business order. The time information may include information such as the current season and festival, and the user status information may include but is not limited to the user's economic status information and user mood information.

[0126] Optionally, you can set the user to have n attributes in total, and the current value of the i-th attribute is recorded as v i , the corresponding weight is recorded as w i , where i=1,2,3,..., n In addition, it can be assumed that there are m pieces of current operating environment information as external factors, and the value of the jth external factor at the current moment is recorded as e j , and its weight on behavior is recorded as u j , where j=1,2,3,..., m .

[0127] Next, we can assume that there are a total of k behaviors, using B l Indicates l Behavior, l =1,2,…,k, and assume that P(B l ) indicates the l The probability of a behavior occurring is determined based on the current operating environment information, that is, the current behavior feature obtained, P(B l ) is calculated as shown in formula (1). This formula normalizes the influence of different behaviors based on user attributes and current operating environment information to obtain the probability of each behavior occurring, with a numerical range between 0 and 1, and the sum of all behavior probabilities is 1.

[0128] (1)

[0129] in, For the lThe first association weight between the behavior and the i-th attribute of the user, wherein the parameter for measuring the degree of association between different behaviors and attributes can be pre-set as the first association weight, which can be obtained based on the correlation coefficient method of historical data (such as Pearson correlation coefficient, Spearman correlation coefficient, etc.) or regression analysis method and other related methods.

[0130] For the l The second association weight between the j-th behavior and the j-th current operating environment information can also be preset as a parameter for measuring the degree of association between different behaviors and each current operating environment information. The second association weight can be based on statistical analysis of the behavior data, such as frequency statistics. In the historical data, the frequency of the j-th current operating environment information appearing when the behavior occurs is counted, and then the frequency is used as The estimated value of .

[0131] In addition, you can set the basic period of behavior change to T, the last behavior After the end, the impact coefficient on this cycle is recorded as in, The value range of can be set according to the actual situation, for example , its value reflects the impact of the last behavior on the cycle, so the cycle of this behavior The calculation formula affected by it is: ,here It is a function that determines the impact on the cycle based on the specific circumstances of the last behavior. For example, it can be simply set based on a certain quantitative score of the behavior. The specific form needs to be determined based on the actual business scenario. The periodicity formula of the behavior is shown in formula (2).

[0132] (2)

[0133] Among them, A l It is l The amplitude of the behavior curve reflects the magnitude of the behavior change and can be set according to the characteristics of different behaviors and related factors; It is the initial phase, which is used to adjust the starting position of the cycle and is also set as needed.

[0134] Therefore, according to this behavior The probability of occurrence, that is, the current behavior characteristics, is used to update the i-th initial attribute weight , the update formula is shown in formula (3).

[0135] (3)

[0136] in, Represents the learning rate parameter, which is used to control the magnitude of weight update. It is generally a small positive number, such as 0.01; s(B l ) is a l A certain sign function indicates whether to update the weight positively or negatively. For example, it can be set that when the behavior achieves a certain expected and set result, a positive update is performed, otherwise a negative update is performed.

[0137] The numerical value stored in the corresponding position of the initial bitmap can be understood as the initial attribute weight of a certain population member. When calculating the weight, it should be combined with the influencing factors mentioned above. At the same time, when returning the new weight value, the pixel coordinates that affect the corresponding weight and the new weight value should be provided to update the initial attribute weight, thereby obtaining a new attribute weight. Therefore, the initial grayscale bitmap can be updated based on the new attribute weight and the coordinates of the pixel points to obtain a new grayscale bitmap.

[0138] Step 206: Receive the construction task through the workstation, generate the target digital population based on the construction task, and send the target digital population to the server.

[0139] Among them, the workstation side may include an instruction processing module, a construction execution module, and a data transmission module, such as Figure 6 As shown, Figure 6 A flowchart of steps for constructing a target digital population provided in an embodiment of the present application includes:

[0140] Step 602: Obtain the current control instruction sent by the server through the instruction processing module of the workstation.

[0141] Step 604: Start the construction task based on the start instruction, and generate the target digital population based on the construction task through the construction execution module on the workstation side.

[0142] Step 606: The data transmission module based on the workstation sends the target digital population to the server.

[0143] Among them, the construction execution module on the workstation side can carry out construction work according to the tasks assigned by the main control program, including construction parameters such as construction scale and probability distribution. The instruction processing module can receive and parse the current control instructions from the server, which may include start instructions, pause instructions, stop instructions, etc., and perform corresponding operations according to the current control instructions. The data transmission module can transmit the construction results to the server in the format specified by the protocol, and during the transmission process, data can be transmitted according to a custom data sharding algorithm.

[0144] Optionally, the server and workstation sides can adopt a client-server model. The workstation side acts as a client, actively establishing a connection with the server side and receiving instructions. The server side acts as a service side to perform unified control and data collection and processing, and at the same time provide management and control services for the browser / server (Browser / Server Architecture, referred to as B / S) architecture program to perform interface monitoring and management.

[0145] In addition, the Streamlined Task Distribution and Data Harvesting Protocol (ST&DH Protocol) can be used for communication. The ST&DH Protocol is a custom protocol for servers and workstations designed based on the TCP protocol, which is used to develop distributed build programs. The master control program allocates configuration information such as the build scale of each workstation to achieve collaborative work of each workstation, and submits the result data to the corresponding data storage device. The protocol specification is built on the TCP protocol and is located above the application layer. The TCP protocol provides a reliable, connection-oriented data transmission service, which provides the underlying guarantee for the message transmission of this protocol. The specific protocol information is shown in Table 1.

[0146] Table 1

[0147]

[0148] The above current control instructions may include:

[0149] 0x00, used by the master control program when sending default data (i.e. non-control instructions). At this time, all data except the bytes represented by the workstation are text type data. This instruction is used to provide system control terminal and workstation status information display and log recording.

[0150] 0x01, used for the master program to send configuration information to the workstation to dynamically set the configuration of the controlled program. Its main purpose is to tell the workstation how much data needs to be landed and transmitted synchronously during production, as well as the link information used for queries involving related data.

[0151] 0x02, start work instruction, is used to send instructions to start specific work of the workstation. The message header is parsed normally according to the agreement. Since the data is sent once, the value of the field used to determine whether the sending is completed (the 5th bit) is always 1.

[0152] 0x03, the master control program uses this instruction to send a request to obtain the progress report of the current corresponding workstation. The completion segment value is always 1. The workstation uses the same instruction value when reporting the current progress to identify the corresponding progress.

[0153] 0x04: This instruction is sent by the workstation to the master control program, indicating that the workstation has jobs that are ready to be submitted.

[0154] 0x05, after the main control program receives the 0x04 instruction, the main control program sends the workstation instruction based on the current state to prepare to receive the job environment to tell the workstation that it can accept the job data and please send it. After that, the workstation will fragment the local ready file and submit the job data accordingly using the 0x05 instruction. The 5th byte indicates whether the data is completed. If it is not completed, it needs to continue to wait for the remaining part. When the workstation completes the job submission, the workstation automatically ends the work, and the main control program marks this workstation as idle and waiting for the next job.

[0155] 0x06, sent by the master control program to control whether the workstation suspends the current work. After receiving the pause command, the workstation saves the current working status context information and replies to the master control program with this command to indicate that it has been suspended and waits for the next time the master control is enabled to resume.

[0156] 0x07, corresponding to instruction 0x06, is sent by the master control program to start and pause the workstation.

[0157] 0x08, the master control program cancels the specified workstation instruction.

[0158] 0x09, used by the master control program to send a work reset to the specified workstation.

[0159] 0x10, provides workstation error reporting. It is used to report to the main control program when an exception or failure occurs in the operation of the workstation. When there is an error workstation, the main control program will resend the corresponding workstation task to a new workstation and start the work. If there is no available workstation, the main control program will record the failed workstation and task information in this build work to facilitate the subsequent separate activation of this part of the work.

[0160] It should be noted that 0x06~0x09 are the control instructions of the main control program on the server side. After the workstation receives the control instructions, it should respond with the same instructions.

[0161] In some other optional embodiments, when constructing a target digital population, such as Figure 7 As shown, Figure 7 Another flowchart of steps for constructing a target digital population provided in an embodiment of the present application includes:

[0162] Step 702: construct basic attribute information corresponding to each group member based on the construction task.

[0163] Step 704: For each crowd member, obtain the preset tag association relationship corresponding to the crowd member.

[0164] Step 706: Use a preset binary search algorithm to search and obtain the tag values ​​and probability distribution of all target-related tags corresponding to the preset tag association relationship.

[0165] Step 708: Perform attribute addition processing on the basic attribute information based on the target association tag and the preset attribute addition rule to generate target attribute information.

[0166] Step 710: construct a target digital population based on the target attribute information.

[0167] Among them, the basic attribute information corresponding to each crowd member can be constructed based on the construction task, and then the attribute addition module can be called to add relevant attributes. Exemplarily, the basic attribute information corresponding to the crowd member can include information such as name, and the target association tag can include the age, gender, marital status, education level, and location of the crowd member, and these target association tags have corresponding preset tag association relationships with the crowd members.

[0168] Optionally, for each crowd member, a preset tag association relationship corresponding to the crowd member may be obtained, and then a preset binary search algorithm may be used to search and obtain tag values ​​and probability distributions of all target association tags corresponding to the preset tag association relationship.

[0169] Among them, the algorithm logic of the preset binary search algorithm can be: first, all target digital population labels are arranged from small to large according to the probability distribution size to obtain a probability distribution list. In order to reduce CPU consumption, the target digital population labels can be arranged and cached when they are first obtained, and can be used directly thereafter. Next, a random value of 0-100 can be randomly generated, and the probability distribution list and the random value can be passed as parameters to the binary search algorithm to quickly locate all target digital population labels that are just less than this random value, and the corresponding label value and probability distribution are used as the label value and probability distribution of the hit target associated label.

[0170] Thus, based on the target associated tags and the preset attribute addition rules, the basic attribute information is subjected to attribute addition processing to generate the target attribute information, and finally the target digital population can be constructed based on the target attribute information. Among them, the preset attribute addition rules may include full addition rules or mutually exclusive addition rules. The full addition rules are used to add the label values ​​and probability distributions of all target associated tags to the population members, and the mutually exclusive addition rules are used to add the label values ​​and probability distributions of the target associated tags to the population members according to the preset addition quantity. It should be noted that when the value of the preset addition quantity is not set, the preset addition quantity can be defaulted to 1.

[0171] In some optional embodiments, when the target digital population is sent to the server, Figure 8 As shown, Figure 8 A flowchart of the steps of sending a target digital population provided in an embodiment of the present application includes:

[0172] Step 802: When the number of members of the target digital population reaches a preset number threshold, data corresponding to the target digital population is written into a target file.

[0173] Step 804: Segment the target file according to a preset file segment size to obtain a plurality of segment files.

[0174] Step 806: Transmit each segment file to the server side respectively, and mark each segment file with a preset marking field.

[0175] Among them, when the number of members of the target digital population reaches the preset number threshold, the data corresponding to the target digital population is written into the target file. After the data corresponding to the target digital population is written, the data synchronization service is started to send a 0x04 instruction to the main control program, in which the data segment can be set to the file name corresponding to the target file. The main control program records the request to be transmitted into the processing queue and waits for its processing service to send a 0x05 instruction to receive the asynchronous transmission of data.

[0176] Then, the target file may be segmented according to a preset file segment size to obtain a plurality of segment files. Optionally, the preset file segment size may be pre-defined, and illustratively, the preset file segment size may be 1024 KB.

[0177] Thus, each fragment file can be transmitted to the server side respectively, and each fragment file can be marked with a preset mark field. Optionally, the 0x05 instruction can be used to use the 5th field mark bit in the protocol as the preset mark field, and the default data format is binary. The preset mark field is used to indicate whether the fragment file has been transmitted.

[0178] Step 208: Receive the target digital population through the server and save the target digital population.

[0179] Among them, the server side is deployed with a master control program, which is the core control unit of the entire distributed construction system, responsible for allocating the construction scale and other configurations of each workstation, and receiving and processing the result data submitted by the workstation. The data storage module on the server side can be responsible for submitting the received result data to the storage device in batches. Therefore, after receiving the target digital population on the server side, the target digital population can be saved.

[0180] In some optional embodiments, interface services can also be provided externally to facilitate other systems and applications to obtain and use the target digital population constructed by this application, which promotes data sharing and application and improves the openness and flexibility of the system.

[0181] In some optional embodiments, the interface provided to the outside is not completely open, and only applications authorized by the system can have access rights, so it is necessary to design an interface authorization mechanism. Optionally, the interface protection can be implemented in the form of encrypted signatures. When the application calls this interface, the request parameters need to be signed, and the interface service needs to be signed and verified. It can be released only after the verification is passed.

[0182] The present application provides a method for generating a digital crowd, the method comprising: determining the workload of each workstation based on the preset total workload and the number of currently idle workstations through the server, and enabling a work thread for each workstation, dividing the task of the scale of the digital crowd to be constructed through the server, and sending the corresponding construction task to each workstation based on the workload and work thread of the workstation, receiving the construction task through the workstation, generating a target digital crowd based on the construction task, and sending the target digital crowd to the server, receiving the target digital crowd through the server, and saving the target digital crowd. This scheme can obtain a large-scale target digital crowd more efficiently, more accurately, more flexibly, more comprehensively, and at a lower cost by constructing a preset tag database to mark the digital crowds of various industries; in addition, by utilizing distributed computing technology, it can quickly process large-scale data and improve the speed and efficiency of data processing, which is very important for processing massive crowd data, ensuring that data can be obtained and analyzed in a timely manner; at the same time, by adopting the clickhouse database as the preset tag database, the column storage capability of the clickhouse database makes it possible to store and query massive data, supporting complex data analysis and query requirements, which provides strong support for big data analysis.

[0183] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0184] The present application also provides a digital crowd generation system, which includes a server end and multiple workstation ends. The server end and each workstation end communicate with each other using a preset efficient task distribution and data collection protocol, wherein:

[0185] The server is used to determine the workload of each workstation based on the preset total workload and the number of currently idle workstations, and enable a working thread for each workstation;

[0186] The server side is also used to divide the tasks according to the scale of the digital population to be constructed, and send corresponding construction tasks to each workstation based on the workload and work thread of the workstation; wherein the construction task includes a preset total workload and preset related construction parameters, and the preset related construction parameters are obtained from a preset label database, which is a clickhouse database; the preset label database includes multiple industry type characteristics and digital population label groups corresponding to the industry type characteristics, the digital population label group includes multiple target digital population labels, target label feature values ​​corresponding to each target digital population label and target label distribution probability, the digital population label group is stored in a grayscale bitmap of a preset size, the pixel value corresponding to each pixel point in the grayscale bitmap is used to represent the target label feature value, the coordinates of the pixel point are used to represent the number of the target digital population label, and the preset size is determined by the number of the target digital population labels;

[0187] The workstation side is used to receive the build task, generate the target digital population based on the build task, and send the target digital population to the server side;

[0188] The server side is also used to receive the target digital population and save the target digital population.

[0189] The digital crowd generation system provided in the embodiment of the present application has similar implementation principles and technical effects to those of the above-mentioned method embodiment, and will not be described in detail here.

[0190] In one embodiment of the present application, a computer device is provided, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0191] The server determines the workload of each workstation based on the preset total workload and the number of currently idle workstations, and enables a working thread for each workstation;

[0192] The server-side performs task division for the scale of the digital population to be constructed, and sends corresponding construction tasks to each workstation based on the workload and work thread of the workstation; wherein the construction task includes a preset total workload and preset related construction parameters, and the preset related construction parameters are obtained from a preset label database, which is a clickhouse database; the preset label database includes multiple industry type characteristics and digital population label groups corresponding to the industry type characteristics, the digital population label group includes multiple target digital population labels, target label feature values ​​corresponding to each target digital population label, and target label distribution probability, the digital population label group is stored in an initial grayscale bitmap of a preset size, the pixel value corresponding to each pixel point in the initial grayscale bitmap is used to represent the target label feature value, the coordinates of the pixel point are used to represent the number of the target digital population label, and the preset size is determined by the number of the target digital population labels;

[0193] Receive the build task through the workstation, generate the target digital population based on the build task, and send the target digital population to the server;

[0194] The target digital population is received through the server and saved.

[0195] In one embodiment of the present application, when the processor executes the computer program, the processor further implements the following steps:

[0196] Obtain initial population data from multiple industries;

[0197] Perform data preprocessing operations on the initial population data to generate target population data; data preprocessing operations include data cleaning, data standardization, data format conversion, data integration and data desensitization;

[0198] Process the target population data to generate multiple industry type characteristics and digital population label groups corresponding to each industry type characteristic;

[0199] Based on multiple industry type characteristics and digital population label groups corresponding to each industry type characteristic, a preset label database is constructed.

[0200] In one embodiment of the present application, when the processor executes the computer program, the processor further implements the following steps:

[0201] Extract and construct labels for the target population data to obtain full labels for people in various industries;

[0202] Analyze and transform all labels of people in various industries to obtain a label pool that can be recognized by the database;

[0203] Generate multiple industry type features corresponding to the target population data based on preset feature recognition rules;

[0204] Organize the tags in the tag pool according to the characteristics of each industry type to obtain multiple digital crowd tag groups;

[0205] The characteristics of each industry type are matched with each digital population label group to obtain a digital population label group corresponding to the characteristics of each industry type.

[0206] In one embodiment of the present application, when the processor executes the computer program, the processor further implements the following steps:

[0207] Obtaining initial attribute weights corresponding to each member of the target digital population; wherein each initial attribute weight is determined based on a pixel value corresponding to each pixel point in the initial grayscale bitmap;

[0208] For each crowd member, obtain the current operating environment information in the target business system, and determine the current behavior characteristics of the crowd member based on the current operating environment information; wherein the current operating environment information includes the number of business orders, time information, user status information, commodity market information in the business order, and commodity price fluctuation information;

[0209] The initial attribute weights are updated based on the current behavior characteristics of the crowd members to obtain new attribute weights;

[0210] The initial grayscale bitmap is updated based on the new attribute weights and pixel coordinates to obtain a new grayscale bitmap.

[0211] In one embodiment of the present application, when the processor executes the computer program, the processor further implements the following steps:

[0212] Obtaining the current control instruction sent by the server through the instruction processing module of the workstation; wherein the current control instruction includes a start instruction;

[0213] Initiate a build task based on the start-up instruction, and generate a target digital population based on the build task through a build execution module on the workstation side;

[0214] The data transmission module based on the workstation side sends the target digital population to the server side.

[0215] In one embodiment of the present application, when the processor executes the computer program, the processor further implements the following steps:

[0216] Based on the construction task, construct the basic attribute information corresponding to each member of the population;

[0217] For each crowd member, obtain the preset tag association relationship corresponding to the crowd member;

[0218] Use a preset binary search algorithm to find and obtain the label values ​​and probability distribution of all target associated labels corresponding to the preset label association relationship;

[0219] Based on the target associated tags and the preset attribute addition rules, the basic attribute information is subjected to attribute addition processing to generate the target attribute information; wherein the preset attribute addition rules include full addition rules or mutually exclusive addition rules, the full addition rules are used to add the label values ​​and probability distributions of all target associated tags to the crowd members, and the mutually exclusive addition rules are used to add the label values ​​and probability distributions of the target associated tags to the crowd members according to the preset addition quantity;

[0220] Based on the target attribute information, the target digital population is constructed.

[0221] In one embodiment of the present application, when the processor executes the computer program, the processor further implements the following steps:

[0222] When the number of population members of the target digital population reaches a preset number threshold, the data corresponding to the target digital population is written into the target file;

[0223] The target file is segmented according to the preset file segment size to obtain multiple segment files;

[0224] Each segment file is transmitted to the server side respectively, and each segment file is marked with a preset marking field; wherein the preset marking field is used to indicate whether the segment file has been transmitted.

[0225] The computer device provided in the embodiment of the present application has similar implementation principles and technical effects to those of the above-mentioned method embodiment, and will not be described in detail here.

[0226] In one embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0227] The server determines the workload of each workstation based on the preset total workload and the number of currently idle workstations, and enables a working thread for each workstation;

[0228] The server-side performs task division for the scale of the digital population to be constructed, and sends corresponding construction tasks to each workstation based on the workload and work thread of the workstation; wherein the construction task includes a preset total workload and preset related construction parameters, and the preset related construction parameters are obtained from a preset label database, which is a clickhouse database; the preset label database includes multiple industry type characteristics and digital population label groups corresponding to the industry type characteristics, the digital population label group includes multiple target digital population labels, target label feature values ​​corresponding to each target digital population label, and target label distribution probability, the digital population label group is stored in an initial grayscale bitmap of a preset size, the pixel value corresponding to each pixel point in the initial grayscale bitmap is used to represent the target label feature value, the coordinates of the pixel point are used to represent the number of the target digital population label, and the preset size is determined by the number of the target digital population labels;

[0229] Receive the build task through the workstation, generate the target digital population based on the build task, and send the target digital population to the server;

[0230] The target digital population is received through the server and saved.

[0231] In one embodiment of the present application, when the computer program is executed by a processor, the following steps are further implemented:

[0232] Obtain initial population data from multiple industries;

[0233] Perform data preprocessing operations on the initial population data to generate target population data; data preprocessing operations include data cleaning, data standardization, data format conversion, data integration and data desensitization;

[0234] Process the target population data to generate multiple industry type characteristics and digital population label groups corresponding to each industry type characteristic;

[0235] Based on multiple industry type characteristics and digital population label groups corresponding to each industry type characteristic, a preset label database is constructed.

[0236] In one embodiment of the present application, when the computer program is executed by a processor, the following steps are further implemented:

[0237] Extract and construct labels for the target population data to obtain full labels for people in various industries;

[0238] Analyze and transform all labels of people in various industries to obtain a label pool that can be recognized by the database;

[0239] Generate multiple industry type features corresponding to the target population data based on preset feature recognition rules;

[0240] Organize the tags in the tag pool according to the characteristics of each industry type to obtain multiple digital crowd tag groups;

[0241] The characteristics of each industry type are matched with each digital population label group to obtain a digital population label group corresponding to the characteristics of each industry type.

[0242] In one embodiment of the present application, when the computer program is executed by a processor, the following steps are further implemented:

[0243] Obtaining initial attribute weights corresponding to each member of the target digital population; wherein each initial attribute weight is determined based on a pixel value corresponding to each pixel point in the initial grayscale bitmap;

[0244] For each crowd member, obtain the current operating environment information in the target business system, and determine the current behavior characteristics of the crowd member based on the current operating environment information; wherein the current operating environment information includes the number of business orders, time information, user status information, commodity market information in the business order, and commodity price fluctuation information;

[0245] The initial attribute weights are updated based on the current behavior characteristics of the crowd members to obtain new attribute weights;

[0246] The initial grayscale bitmap is updated based on the new attribute weights and pixel coordinates to obtain a new grayscale bitmap.

[0247] In one embodiment of the present application, when the computer program is executed by a processor, the following steps are further implemented:

[0248] Obtaining the current control instruction sent by the server through the instruction processing module of the workstation; wherein the current control instruction includes a start instruction;

[0249] Initiate a build task based on the start-up instruction, and generate a target digital population based on the build task through a build execution module on the workstation side;

[0250] The data transmission module based on the workstation side sends the target digital population to the server side.

[0251] In one embodiment of the present application, when the computer program is executed by a processor, the following steps are further implemented:

[0252] Based on the construction task, construct the basic attribute information corresponding to each member of the population;

[0253] For each crowd member, obtain the preset tag association relationship corresponding to the crowd member;

[0254] Use a preset binary search algorithm to find and obtain the label values ​​and probability distribution of all target associated labels corresponding to the preset label association relationship;

[0255] Based on the target associated tags and the preset attribute addition rules, the basic attribute information is subjected to attribute addition processing to generate the target attribute information; wherein the preset attribute addition rules include full addition rules or mutually exclusive addition rules, the full addition rules are used to add the label values ​​and probability distributions of all target associated tags to the crowd members, and the mutually exclusive addition rules are used to add the label values ​​and probability distributions of the target associated tags to the crowd members according to the preset addition quantity;

[0256] Based on the target attribute information, the target digital population is constructed.

[0257] In one embodiment of the present application, when the computer program is executed by a processor, the following steps are further implemented:

[0258] When the number of population members of the target digital population reaches a preset number threshold, the data corresponding to the target digital population is written into the target file;

[0259] The target file is segmented according to the preset file segment size to obtain multiple segment files;

[0260] Each segment file is transmitted to the server side respectively, and each segment file is marked with a preset marking field; wherein the preset marking field is used to indicate whether the segment file has been transmitted.

[0261] The computer-readable storage medium provided in this embodiment has similar implementation principles and technical effects to those of the above method embodiments, and will not be described in detail here.

[0262] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0263] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0264] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for generating a digital crowd, characterized in that: A system for generating digital crowds, the system comprising a server and a plurality of workstations, the server and the workstations communicating with each other using a preset efficient task distribution and data collection protocol, the method comprising: The server determines the workload of each workstation based on the preset total workload and the number of currently idle workstations, and enables a work thread for each workstation; The server-side performs task division for the scale of the digital population to be constructed, and sends corresponding construction tasks to each of the workstations based on the workload of the workstation and the work thread; wherein the construction task includes a preset total workload and preset related construction parameters, and the preset related construction parameters are obtained from a preset label database, and the preset label database is a clickhouse database; the preset label database includes multiple industry type features and digital population label groups corresponding to each of the industry type features, and the digital population label group includes multiple target digital population labels, target label feature values ​​corresponding to each of the target digital population labels, and target label distribution probabilities, and the digital population label group is stored in an initial grayscale bitmap of a preset size, and the pixel value corresponding to each pixel in the initial grayscale bitmap is used to represent the target label feature value, and the coordinates of the pixel point are used to represent the number of the target digital population label, and the preset size is obtained by the target digital population label. The number is determined; the construction process of the preset label database includes: obtaining initial population data of multiple industries; performing data preprocessing operations on the initial population data to generate target population data; wherein the data preprocessing operations include data cleaning, data standardization, data format conversion, data integration and data desensitization; extracting and constructing labels for the target population data to obtain full labels for populations in various industries; parsing and converting the full labels for populations in various industries to obtain a label pool that can be identified by the database; based on preset feature recognition rules, generating multiple industry type features corresponding to the target population data; organizing the labels in the label pool according to the characteristics of each industry type to obtain multiple digital population label groups; matching each industry type feature with each digital population label group to obtain a digital population label group corresponding to each industry type feature; based on the multiple industry type features and the digital population label groups corresponding to each industry type feature, constructing the preset label database; Receiving the construction task through the workstation, generating a target digital population based on the construction task, and sending the target digital population to the server; The target digital population is received through the server and stored.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining initial attribute weights corresponding to each member of the target digital population; wherein each initial attribute weight is determined based on a pixel value corresponding to each pixel point in the initial grayscale bitmap; For each of the crowd members, obtain the current operating environment information in the target business system, and determine the current behavior characteristics of the crowd members based on the current operating environment information; wherein the current operating environment information includes the number of business orders, time information, user status information, commodity market information in the business order, and commodity price fluctuation information; The initial attribute weights are updated based on the current behavior characteristics of the crowd members to obtain new attribute weights; The initial grayscale bitmap is updated based on the new attribute weights and the coordinates of the pixel points to obtain a new grayscale bitmap.

3. The method according to claim 1, characterized in that The step of receiving the construction task through the workstation, generating a target digital population based on the construction task, and sending the target digital population to the server includes: Acquiring the current control instruction sent by the server through the instruction processing module of the workstation; wherein the current control instruction includes a start instruction; Starting the construction task based on the start instruction, and generating a target digital population based on the construction task through the construction execution module on the workstation side; The data transmission module based on the workstation sends the target digital population to the server.

4. The method according to claim 1, characterized in that: The generating of the target digital population based on the construction task also includes: Based on the construction task, construct basic attribute information corresponding to each group member; For each of the crowd members, obtaining a preset tag association relationship corresponding to the crowd member; Using a preset binary search algorithm to search and obtain the tag values ​​and probability distribution of all target associated tags corresponding to the preset tag association relationship; Based on the target associated tags and preset attribute addition rules, attribute addition processing is performed on the basic attribute information to generate target attribute information; wherein the preset attribute addition rules include full addition rules or mutually exclusive addition rules, the full addition rule is used to add the label values ​​and probability distributions of all the target associated tags to the crowd members, and the mutually exclusive addition rule is used to add the label values ​​and probability distributions of the target associated tags to the crowd members according to the preset addition quantity; Based on the target attribute information, the target digital population is constructed.

5. The method according to claim 1, characterized in that The step of sending the target digital population to the server side includes: When the number of members of the target digital population reaches a preset number threshold, writing the data corresponding to the target digital population into the target file; Slice the target file according to a preset file slice size to obtain multiple slice files; Each of the segment files is transmitted to the server end respectively, and each of the segment files is marked with a preset mark field; wherein the preset mark field is used to indicate whether the segment file has been transmitted.

6. A digital crowd generation system, characterized in that: The system includes a server end and a plurality of workstation ends, wherein the server end and each of the workstation ends communicate with each other using a preset efficient task distribution and data collection protocol, wherein: The server is used to determine the workload of each workstation based on the preset total workload and the number of currently idle workstations, and enable a work thread for each of the workstations; The server side is also used to divide the tasks according to the scale of the digital population to be constructed, and send corresponding construction tasks to each of the workstations based on the workload of the workstation and the work thread; wherein the construction task includes a preset total workload and preset related construction parameters, and the preset related construction parameters are obtained from a preset label database, and the preset label database is a clickhouse database; the preset label database includes multiple industry type features and digital population label groups corresponding to each of the industry type features, and the digital population label group includes multiple target digital population labels, target label feature values ​​corresponding to each of the target digital population labels, and target label distribution probabilities, and the digital population label group is stored using an initial grayscale bitmap of a preset size, and the pixel values ​​corresponding to each pixel in the initial grayscale bitmap are used to represent the target label feature values, and the coordinates of the pixel points are used to represent the number of the target digital population label, and the preset size is obtained by the target digital population label. The number of people in different industries is determined; the construction process of the preset label database includes: obtaining initial population data of multiple industries; performing data preprocessing operations on the initial population data to generate target population data; wherein the data preprocessing operations include data cleaning, data standardization, data format conversion, data integration and data desensitization; performing label extraction and construction on the target population data to obtain full labels of people in different industries; performing parsing and conversion processing on the full labels of people in different industries to obtain a label pool identifiable by the database; based on preset feature recognition rules, generating multiple industry type features corresponding to the target population data; organizing the labels in the label pool according to the characteristics of each industry type to obtain multiple digital population label groups; matching each industry type feature with each digital population label group to obtain a digital population label group corresponding to each industry type feature; constructing the preset label database based on the multiple industry type features and the digital population label groups corresponding to each industry type feature; The workstation is used to receive the construction task, generate a target digital population based on the construction task, and send the target digital population to the server; The server side is also used to receive the target digital population and save the target digital population.

7. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method for generating a digital crowd as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method for generating a digital crowd as described in any one of claims 1-5.

Images