A data synchronization method and device, electronic equipment and storage medium

Abstract

The application discloses a data synchronization method and device, electronic equipment and a storage medium. The method comprises the following steps: obtaining position information; determining a first target model according to the difference between the position information and historical position information recorded in a feature database, and version update data of a position feature generation model used by the historical position information, wherein the historical position information comprises a historical position identifier and historical position description information corresponding to the historical position identifier; calling the first target model, and analyzing first field data in first position description information through the first target model to obtain target position feature information corresponding to the first position description information; storing the first position description information and the target position feature information in the feature database corresponding to the first position identifier, so as to provide consumption data to a data consumer through the feature database. The embodiment of the application can improve the synchronization efficiency of recruitment information.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a data synchronization method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the rapid development of the Internet, the application scenarios of solving work and life through online services are also more and more extensive. For example, a recruitment service provider can provide a recruitment information publishing portal to the recruitment party, and by analyzing and promoting the position information included in the recruitment information, the matching degree of the job seekers and the recruitment demand can be improved.

[0003] However, as the enterprise scale and the complexity of the enterprise branches involved by the recruitment party increase, and the recruitment information collection service provided by the recruitment website becomes more and more detailed, the database storing the position information receives updates of the position information from multiple data sources. In the face of a large amount of updated position information data, there may be multiple data with the same processing logic, and thus, there may be repeated processing of the position information, which not only causes waste of calculation, but also reduces the data processing speed, causes the position information to be unable to be updated to the downstream data section in time, and affects the recruitment progress of the recruitment party. SUMMARY

[0004] Therefore, the embodiments of the present application provide a data synchronization method and device, electronic equipment and storage medium, which improve the synchronization efficiency of the recruitment information.

[0005] The embodiments of the present application provide a data synchronization method, which comprises the following steps:

[0006] obtaining position information, the position information comprising a first position identifier and first position description information corresponding to the first position identifier, and the first position description information comprising first field data corresponding to a preset field;

[0007] determining a first target model according to the difference between the position information and historical position information recorded in a feature database, and version update data of a position feature generation model used by the historical position information, wherein the historical position information comprises a historical position identifier and historical position description information corresponding to the historical position identifier;

[0008] calling the first target model, and analyzing the first field data in the first position description information through the first target model to obtain target position feature information corresponding to the first position description information;

[0009] storing the first position description information and the target position feature information into the feature database corresponding to the first position identifier, so as to provide consumption data to a data consumer through the feature database.

[0010] Optionally, according to the method of the embodiment of the application, the number of the first target models is at least one, and the target position characteristic information comprises at least one position characteristic.

[0011] The first target model is called, and the first field data in the first position description information is parsed by the first target model to obtain the target position characteristic information corresponding to the first position description information, comprising:

[0012] The preset field required by each first target model is determined.

[0013] Corresponding to each first target model, the first field data corresponding to the preset field required by the first target model is input into the first target model, and the position characteristic is output by the first target model.

[0014] The position characteristic output by each first target model is obtained to obtain the target position characteristic information corresponding to the first position description information.

[0015] Optionally, according to the method of the embodiment of the application, the first target model is determined according to the difference between the position information and the historical position information recorded in the characteristic database, and the version update data of the position characteristic generation model used by the historical position information.

[0016] Each historical position identifier in the characteristic database is traversed.

[0017] In the case that the characteristic database does not record the first position identifier, the model version of each position characteristic generation model in the model library is determined.

[0018] Each latest version of the position characteristic generation model is determined as the first target model.

[0019] Optionally, according to the method of the embodiment of the application, the number of the position characteristic generation models corresponding to each historical position identifier is at least one.

[0020] The first target model is determined according to the difference between the position information and the historical position information recorded in the characteristic database, and the version update data of the position characteristic generation model used by the historical position information, and further comprising:

[0021] In the case that the characteristic database records the first position identifier, the latest version and the first model version of each first position characteristic generation model are determined according to the version update data, wherein the first position characteristic generation model of the first model version is used to parse the historical position description information corresponding to the first position identifier.

[0022] In the case that the first model version is different from the latest version, the first position characteristic generation model is determined as the first target model.

[0023] Optionally, according to the method of the embodiment of the application, the historical position description information corresponding to the first position identifier comprises second field data corresponding to a preset field;

[0024] According to the difference between the position information and the historical position information recorded in the feature database, and the version update data of the position feature generation model used by the historical position information, the first target model is determined, and the method further comprises:

[0025] In a case where the first position identifier is recorded in the feature database, and the first model version of the first position feature generation model is the latest version, the first field data and the second field data of the preset field required by each first position feature generation model are determined;

[0026] In a case where the first field data and the second field data of the preset field required by the first position feature generation model are inconsistent, the first position feature generation model is determined as the first target model.

[0027] Optionally, according to the method of the embodiment of the application, the method further comprises:

[0028] The first field data of the preset field required by the first position feature generation model is obtained from the first position description information, and the first field data is subjected to hash calculation to obtain a first hash value;

[0029] The second field data of the preset field required by the first position feature generation model is obtained from the historical position information, and the second field data is subjected to hash calculation to obtain a second hash value;

[0030] In a case where the first hash value and the second hash value are inconsistent, it is determined that the first field data and the second field data of the preset field required by the first position feature generation model are inconsistent.

[0031] Optionally, according to the method of the embodiment of the application, before the first position description information and the target position feature information corresponding to the first position identifier are stored in the feature database, the method further comprises:

[0032] In a case where the first field data and the second field data of the preset field required by the first position feature generation model are consistent, the position feature information corresponding to the historical position description information of the first position identifier is determined as the target position feature information corresponding to the first position description information.

[0033] Optionally, according to the method of the embodiment of the application, the method further comprises:

[0034] A model version update notification is received, and the model update notification comprises a model update time of the second target model in the model library;

[0035] obtain the latest position description information corresponding to each second position identifier in the feature database, wherein the latest position description information corresponding to each second position identifier is stored in the feature database at a time earlier than the model update time of the second target model;

[0036] invoke the latest version of the second target model in the model library, and parse the latest position description information corresponding to each second position identifier through the latest version of the second target model, to obtain the second position feature information corresponding to each second position identifier;

[0037] corresponding to each second position identifier, store the second position feature information and the model update time of each second target model in the feature database.

[0038] Optionally, according to the method of the embodiment of the present application, the method further comprises:

[0039] in response to the feature database storing the first position description information and the target position feature information corresponding to the first position identifier, generate a position synchronization record, and the position synchronization record comprises a data synchronization type of the first position identifier;

[0040] store the position synchronization record in a preset message queue, so that the data consumer determines whether to read the position synchronization record according to the data synchronization type;

[0041] wherein the position synchronization record further comprises the first position description information and the target position feature information corresponding to the first position identifier.

[0042] The embodiment of the present application provides a data synchronization device, comprising:

[0043] an acquisition module, configured to acquire position information, wherein the position information comprises a first position identifier and first position description information corresponding to the first position identifier, and the first position description information comprises first field data corresponding to a preset field;

[0044] a processing module, configured to determine a first target model according to a difference between the position information and historical position information recorded in a feature database, and version update data of a position feature generation model used by the historical position information, wherein the historical position information comprises a historical position identifier and historical position description information corresponding to the historical position identifier;

[0045] the processing module is further configured to invoke the first target model, and parse the first field data in the first position description information through the first target model, to obtain target position feature information corresponding to the first position description information;

[0046] the processing module is further configured to, corresponding to the first position identifier, store the first position description information and the target position feature information in the feature database, so as to provide consumption data to a data consumer through the feature database.

[0047] An electronic device is provided, which includes a processor and a memory storing computer program instructions; the processor implements the steps of the method as above when executing the computer program instructions.

[0048] A computer readable storage medium is provided, which stores computer program instructions; the computer program instructions are executed by a processor to implement the steps of the method as above.

[0049] A computer program product is provided, which includes computer program instructions; the computer program instructions are executed by a processor to implement the steps of the method as above.

[0050] According to the embodiments of the present application, by providing a feature database, and taking the position identifier as the primary key, the position information corresponding to different position identifiers is stored. When receiving the newly added position information, the difference between the position information and the historical position information recorded in the feature database can be determined according to the first position identifier included in the position information, and the first target model used for updating the feature database is determined in combination with the version update data of the position feature generation model used by the historical position information, so as to reduce repeated calling of the model, save the computing resources, and improve the processing efficiency. In addition, the position feature data and the position description information corresponding to each position identifier are standardized and stored in the feature database with the position identifier as the primary key before being sent to the downstream consumer, so that the consumption data provided by the feature database to the downstream data consumer can conveniently meet the data use requirements of different consumers, and the flexibility and scalability of the business service are improved. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings in the embodiments of the present application.

[0052] Figure 1 is a schematic diagram of the system architecture of the embodiments of the present application.

[0053] Figure 2 is a flow chart of a data synchronization method according to an embodiment of the present application.

[0054] Figure 3 is a schematic diagram of the processing process of data synchronization according to an embodiment of the present application.

[0055] Figure 4 is a flow chart of another data synchronization method provided by an embodiment of the present application.

[0056] Figure 5 is a structural block diagram of a data synchronization device according to an embodiment of the present application.

[0057] Figure 6 This is a schematic diagram of an electronic device used to implement the data synchronization method of the embodiments of this application. Detailed Implementation

[0058] The principles and spirit of this application will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided to make the principles and spirit of this application clearer and more thorough, enabling those skilled in the art to better understand and implement the principles and spirit of this application. The exemplary embodiments provided herein are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments described herein without inventive effort are within the scope of protection of this application.

[0059] The embodiments of this application relate to terminal devices and / or servers. Those skilled in the art will understand that the embodiments of this application can be implemented as a system, apparatus, device, method, computer-readable storage medium, or computer program product. Therefore, this disclosure can be specifically implemented in at least one of the following forms: entirely hardware, entirely software, or a combination of hardware and software. According to embodiments of this application, this application claims protection for a data synchronization method, apparatus, electronic device, computer-readable storage medium, and computer program product. Figure 1 A schematic diagram of a system architecture according to an embodiment of this application is shown. For example... Figure 1 As shown, the system includes a terminal device 102 and a server 104. The terminal device 102 may include at least one of the following: a smartphone, tablet, laptop, desktop computer, smart TV, various wearable devices, augmented reality (AR) devices, virtual reality (VR) devices, etc. A client can be installed on the terminal device 102. For example, the client may be a client specifically designed to perform a particular function (such as an application app), or a client with multiple embedded application applets (with different functions), or a client accessed via a browser. Users can perform operations on the terminal device 102. For example, a user can open the client installed on the terminal device 102 and input commands through the client, or a user can open the browser installed on the terminal device 102 and input commands through the browser. After receiving the user's input command, the terminal device 102 sends a request message containing the command to the server 104.

[0060] For example, recruiters can access recruitment service interfaces such as websites, applications, or mini-programs for employers through terminal device 102, and use terminal device 102 to input recruitment information, modify recruitment information, and update job-related information such as the company's operating status.

[0061] The server 104 receives the request information, performs corresponding processing, and then returns the processing result information to the terminal device 102. Through a series of data processing and information interaction, the user instruction is completed. Alternatively, in the embodiment of the present application, the data synchronization method can be executed by the server 104. The server 104 can receive the position-related information input by the employer on the terminal device 102 in the recruitment service interface, extract the position identifier and the position identifier corresponding position description information, and then synchronize the position identifier and the position identifier corresponding position description information to the feature database, so as to provide consumption data to downstream data consumers through the feature database, which can conveniently meet the data use requirements of different consumers and improve the flexibility and scalability of business services.

[0062] In this document, terms such as first, second, third, and the like are used merely to distinguish one entity (or action) from another, without necessarily requiring or implying any order or sequence between the entities (or actions).

[0063] The data synchronization method provided by the embodiment of the present application will be introduced below in combination with a specific application scenario. Figure 2 A flowchart of the data synchronization method of the embodiment of the present application is shown, which includes the following steps 201 to 204.

[0064] Step 201, obtaining position information, the position information including a first position identifier and first position description information corresponding to the first position identifier, the first position description information including first field data corresponding to a preset field;

[0065] Step 202, determining a first target model according to the difference between the position information and historical position information recorded in the feature database, and version update data of a position feature generation model used by the historical position information, wherein the historical position information includes a historical position identifier and historical position description information corresponding to the historical position identifier;

[0066] Step 203, calling the first target model, and analyzing the first field data in the first position description information through the first target model to obtain target position feature information corresponding to the first position description information;

[0067] Step 204, storing the first position description information and the target position feature information into the feature database corresponding to the first position identifier, so as to provide consumption data to data consumers through the feature database.

[0068] The above steps will be described in detail in combination with specific embodiments, as follows.

[0069] Involving the step 201, the server can obtain the newly added position information by reading the related database, parsing the log file, etc.

[0070] In some embodiments, as the recruitment information collection service provided by the recruitment website is more and more refined, the information of the recruitment party can be divided into a plurality of sub-service modules, and the information of the recruitment party is maintained by modules. Therefore, the position information obtained by the server can come from different service modules. Figure 3 is a schematic diagram of the processing process of data synchronization of the embodiments of the present application, in combination with Figure 3 The service modules include, for example, a member basic information module, a member value-added service information module, a company basic information module, a company portrait information module, a position basic information module, a position value-added service information module, and a position portrait information module.

[0071] For example, each member account can correspond to one or more companies, and each company includes one or more positions. Therefore, each member account can correspond to one or more positions. In the case where one member account corresponds to a plurality of companies, the plurality of companies can belong to one general company or belong to different companies respectively, which is not specifically limited here.

[0072] For example, each service module can obtain updated data in real time from the database and log file related to the service module, and put the updated data into the message queue in combination with the event time of the updated data. In some optional embodiments, each service module can use the Flank CDC tool to capture the updated data and synchronize the updated data to the message queue. Optionally, the message queue can be a Kafka message queue, which is not specifically limited here. Based on this, the server can read the data in the message queue to obtain the newly added position information.

[0073] In order to facilitate the subsequent synchronization process and improve the application convenience of the stored data, the position information can be expanded with the position identifier as the primary key to obtain the position description information, that is, the position information is parsed to obtain the field data corresponding to the preset field. For example, the preset field includes, for example, position name, salary, education, work experience, work location, etc. The specific preset field is not listed here. In order to introduce the data synchronization method provided by the embodiments of the present application, the position identifier to be synchronized is described as a first position identifier, and the position description information corresponding to the first position identifier is described as first position description information.

[0074] As a specific example, taking the first position identifier as the primary key, the first position identifier corresponding position description information can include: position information: ID: 10001, position name: big data development engineer, salary: 10000, education: master, work experience: 5 years, major: computer.

[0075] After obtaining the position information, next, step 202 is involved, the first position description information can be compared with the historical position description information recorded in the feature database according to the first position identifier included in the position information, so as to determine the difference between the first position information and the historical position information recorded in the feature database, and generate the version update data of the position feature generation model used in combination with the historical position information, to determine the first target model used to update the feature database.

[0076] It can be understood that the feature database can be used to store the historical position information that has been received, wherein the historical position information can include a historical position identifier, and the historical position identifier corresponding historical position description information. Wherein the event occurrence time corresponding to the historical position information is earlier than the event occurrence time of the first position description information.

[0077] Exemplarily, the existence of the difference between the position information and the historical position information recorded in the feature database includes but is not limited to: the feature database does not record the position information at all, the feature database records the first position identifier and the field data of part or all of the preset fields in the first position description information, etc. Wherein the feature database does not record the position information at all, for example, there is no historical position identifier consistent with the first position identifier in the feature database, etc.

[0078] Optionally, the difference between the position information and the historical position information recorded in the feature database can be found by comparing whether the position identifier is consistent, whether the preset fields in the position description information are consistent.

[0079] In some embodiments, the position feature generation model used by the historical position information can be pre-configured in the server, or can be pre-configured in the third party service platform. Each position feature model corresponds to a version update data, and through the version update data, the number of model updates can be understood, and the latest version of the model can be determined. Optionally, the model update data can include but is not limited to the version identifier of the model after each update of the model, or the update time of the model after each update.

[0080] The first target model can be determined in combination with the difference between the position information and the historical position information recorded in the feature database, and the version update data of the position feature generation model used by the historical position information. For example, whether the position feature generation model used by the historical position information has a version update can be determined according to the version update data of the position feature generation model used by the historical position information, and in the case of a version update, the latest version of the position feature model is directly determined as the first target model. Based on this, the position feature information generated by the first target model can be directly sent to the downstream consumer, and the server and the downstream consumer do not need to repeatedly verify whether the position feature information generated based on the latest version of the feature generation model, thereby effectively saving computing resources.

[0081] Involving the above step 203, the called first target model can be pre-configured in the server, and can be a model library accessible by the server. The number of first target models can be one or more.

[0082] Optionally, in the case where the number of first target models is multiple, by configuring the position feature generation model in the model library, the position feature model in the feature library can be conveniently called in an asynchronous manner in parallel. Especially in the case where the number of position information to be processed is large, or the number of first target models to be called corresponding to each position information is large, the model can be called in parallel in an asynchronous manner, which can effectively save the time of data in communication, and improve the data processing efficiency and the speed of providing synchronous data to the downstream consumer.

[0083] In the embodiments of the present application, the number of first target models is at least one, corresponding to one first position description information, and the target position feature information includes at least one position feature.

[0084] For example, in the above step 203, the first target model is called, and the first field data in the first position description information is analyzed by the first target model to obtain the target position feature information corresponding to the first position description information. Specifically, it can include the following steps: determining the preset field required by each first target model; corresponding to each first target model, inputting the first field data corresponding to the preset field required by the first target model into the first target model, and outputting the position feature by the first target model; obtaining the position feature output by each first target model to obtain the target position feature information corresponding to the first position description information.

[0085] As a specific example, the preset field required by the first target model is, for example, salary, education, and major. The field data of the required field obtained by the first position description information can be used as the input information of the first target model, and the input information is, for example, 10000, master, and computer.

[0086] It can be understood that in the case that the number of the first target models is multiple, different first target models have different calculation logics, and in addition, the preset fields required by different first target models can be different or the same.

[0087] The position feature information corresponding to the first position description information is obtained by acquiring the position feature output by each first target model, so that the target position feature information includes at least one position feature.

[0088] According to the embodiment of the present application, the position feature data can be quickly generated by calling the position feature generation model, which is beneficial to improve the speed of providing synchronous data to downstream consumers.

[0089] After obtaining the target position feature information corresponding to the first position description information, the step 204 is involved, and the first position description information and the target position feature information are stored in the feature database corresponding to the first position identifier.

[0090] Since the storage link of each position identifier, the position description information corresponding to each position identifier, and the position feature information corresponding to each position identifier in the feature database is clear, the storage efficiency of the data and the query efficiency of the data can be improved. When accessing the related information corresponding to each position identifier in the feature database, the feature database can provide the latest information corresponding to each position identifier by default, and the old related information will not be queried. The related information includes but is not limited to the position description information and the position feature information. Alternatively, the feature database can select a database based on HBase, based on which the convenience of providing consumption data to downstream consumers can be further improved.

[0091] Taking the historical position description information corresponding to the first position identifier and the position feature information generated based on the historical position description information recorded in the feature database as an example, after generating the target position feature information of the first position description information and storing the first position description information and the target position feature information corresponding to the first position identifier in the feature database, the related information of the first position identifier is read by the consumer when accessing the feature database each time, and the first position description information and the target position feature information are provided to the consumer by default.

[0092] According to the embodiment of the present application, by providing the feature database, when receiving the newly added position information, the difference between the position information and the historical position information recorded in the feature database can be determined according to the first position identifier included in the position information, and the first target model used to update the feature database can be determined in combination with the version update data of the position feature generation model used by the historical position information, so as to reduce repeated calling of the model, save the calculation resources, and improve the processing efficiency.

[0093] Based on this, because the data synchronization speed is fast, any change in the details of the recruitment related information can be captured by the server in time, and the changed information is adjusted to the position characteristic information of the position identifier, so as to facilitate the job seeker to understand the latest dynamics of the position information in time, and also facilitate to match the person who meets the needs of the recruitment party in time.

[0094] In addition, the position characteristic data and the position description information corresponding to each position identifier are standardized and stored in the characteristic database with the position identifier as the primary key before being sent to the downstream consumer. Therefore, by providing the consumption data to the downstream data consumer through the characteristic database, the data use demand of different consumers can be conveniently met, and the flexibility and scalability of business services can be improved.

[0095] In some optional embodiments, in order to further improve the speed of data synchronization and provide effective data to the downstream consumer in time and accurately, the first target model can also be determined by the following manner, so as to cope with the situation of a large amount of newly added data and frequent version update of the position characteristic generation model.

[0096] In one example, the step 203 is involved, the first target model is determined according to the difference between the position information and the historical position information recorded in the characteristic database, and the version update data of the position characteristic generation model used by the historical position information. Specifically, it can be: traversing each historical position identifier in the characteristic database; in the case that the characteristic database does not record the first position identifier, determining the model version of each position characteristic generation model in the model library; determining each latest version of the position characteristic generation model as the first target model.

[0097] By traversing each historical position identifier in the characteristic database, it can be determined whether the characteristic database has stored the first position identifier. It can be understood that, if the characteristic database does not record the first position identifier, it means that the first position identifier is a newly added position, and all logical operations need to be performed on the position corresponding to the first position identifier in order to obtain comprehensive position characteristic information.

[0098] The position characteristic generation models used by the historical position information are all configured in the model library. Correspondingly, in combination with the version update data of the position characteristic generation model used by the historical position information, the model version of each position characteristic generation model in the model library can be determined, and each latest version of the position characteristic generation model is determined as the first target model.

[0099] Based on the embodiments of the present application, by analyzing the first position description information with the latest version of the position feature model, the position feature data calculated based on the latest operation logic can be obtained, which can facilitate the consumer to directly use the position feature information, and especially the downstream consumer does not need to repeatedly verify whether the position feature information generated based on the latest version of the feature generation model, thereby effectively saving the computing resources.

[0100] In some embodiments, the number of corresponding position feature generation models of each historical position identifier is at least one. Again referring to step 203, according to the difference between the position information and the historical position information recorded in the feature database, and the version update data of the position feature generation model used by the historical position information, the first target model is determined, which can further include:

[0101] In the case that the first position identifier is recorded in the feature database, according to the version update data, the latest version and the first model version of each first position feature generation model are determined, wherein the first position feature generation model of the first model version is used to analyze the historical position description information corresponding to the first position identifier; in the case that the first model version is different from the latest version, the first position feature generation model is determined as the first target model.

[0102] Specifically, the first position identifier is recorded in the feature database, that is, the historical position identifier identical to the first position identifier is included in the feature database. Therefore, it can be determined that the corresponding position of the first position identifier may have established changes.

[0103] In order to make the feature data stored in the feature database generated by using the latest version of the model, therefore, in the case that it is determined that the first position identifier is recorded in the feature database, the position feature generation model used by the historical position description information corresponding to the first position identifier can be determined, that is, the first position feature generation model. The number of the first position generation model can be one or multiple.

[0104] Combined with the model update data, the model version of the first position feature generation model used for analyzing the historical position description information corresponding to the first position identifier can be determined, that is, the first model version.

[0105] Optionally, when storing the position description information and the position feature information corresponding to the position identifier, the feature database can also store the version update time of the model generating the position feature information. That is, a preset field corresponding to the position identifier can be added to store the version update time of the model or the model version, wherein the version update time is, for example, 20xx x month xx day, yy:00. Therefore, it can be determined whether the first model version is the latest version.

[0106] Compared with the first model version and the latest version of the first position feature generation model, if the first model version and the latest version of the first position feature generation model are different, it indicates that the first position feature generation model has been updated, the first position feature model can be used as the first target model, and it is not necessary to compare the difference between the historical description information corresponding to the first position identifier and the first position identifier description information, thereby simplifying the data synchronization process.

[0107] Based on the embodiments of the present application, in the case that the feature database stores the first position identifier, by judging whether the model used by the historical position description information corresponding to the first position identifier is the latest version, and taking the latest version of the position feature generation model as the first target model to be called. Next, the position feature data calculated based on the latest operation logic can be directly obtained, which can facilitate the consumer to directly use the position feature information, especially the downstream consumer does not need to repeatedly verify whether the position feature information generated based on the latest version of the feature generation model, effectively saving the computing resources.

[0108] In some embodiments, the historical position description information corresponding to the first position identifier includes second field data corresponding to a preset field; referring again to step 203, according to the difference between the position information and the historical position information recorded in the feature database, and the version update data of the position feature generation model used by the historical position information, the first target model is determined, further comprising:

[0109] In the case that the feature database records the first position identifier, and the first model version of the first position feature generation model is the latest version, the first field data and the second field data of the preset field required by each first position feature generation model are determined; in the case that the first field data and the second field data of the preset field required by the first position feature generation model are inconsistent, the first position feature generation model is determined as the first target model.

[0110] For example, model 1 is a position feature generation model used by the historical position description information corresponding to the first position identifier, and the first model version of model 1 is the latest version. If the preset field required by model 1 is salary, education, and major, the field data corresponding to salary, education, and major respectively can be obtained from the first position description information, that is, the first field data, and the field data corresponding to salary, education, and major respectively can be obtained from the historical position description information corresponding to the first position identifier, that is, the second field data. By comparing whether there is a difference between the first field data and the second field data, it is determined whether model 1 is determined as the first target model.

[0111] According to the embodiment of the present application, in the case that the first field data and the second field data of the preset field required by the first position feature generation model are inconsistent, the first position feature generation model is determined as the first target model, thereby effectively reducing the occurrence of repeated data processing by calling the model, reducing the number of model calls, and saving computing resources.

[0112] Optionally, whether there is a difference between the first field data and the second field data can be compared by a preset comparison algorithm.

[0113] In yet another optional embodiment, the first field data of the preset field required by the first position feature generation model can be obtained from the first position information, and the first field data is subjected to hash calculation to obtain a first hash value; the second field data of the preset field required by the first position feature generation model can be obtained from the historical position information, and the second field data is subjected to hash calculation to obtain a second hash value; in the case that the first hash value and the second hash value are inconsistent, it is determined that the first field data and the second field data of the preset field required by the first position feature generation model are inconsistent.

[0114] Taking model 1 as an example, the field data corresponding to the salary, education, and major respectively is obtained from the first position description information, and subjected to hash calculation to obtain a first hash value, such as b2f5ff47436671b6e533d8dc3614845d; the field data corresponding to the salary, education, and major respectively is obtained from the historical position description information, and subjected to hash calculation to obtain a second hash value, such as c938cbdc10b9e1d0b8b760f9e7342c36. Finally, by comparing whether the first hash value and the second hash value are the same, it can be quickly determined whether there is a difference between the first field data and the second field data.

[0115] Optionally, the algorithm used for hash calculation includes but is not limited to MD5 algorithm, and the specific algorithm can be set according to actual application requirements and performance, which is not specifically limited here.

[0116] According to the embodiment of the present application, by comparing the hash values of two field data, it can be quickly and accurately determined whether there is a difference between the two field data, which is beneficial to improve the data synchronization efficiency.

[0117] In some embodiments, before the first position description information and the target position feature are stored in the feature database corresponding to the first position identifier, the method further comprises:

[0118] If the first field data and the second field data of the preset fields required by the first job feature generation model are consistent, the job feature information corresponding to the historical job description information of the first job identifier will be determined as the target job feature information corresponding to the first job description information.

[0119] Specifically, if the first job feature generation model remains unchanged, and the first field data and the second field data of the preset fields required by the first job feature generation model are consistent, then the job feature information corresponding to the historical job description information of the first job identifier stored in the feature database can be directly determined as the target job feature information corresponding to the first job description information. Since there is no need to call the model, computing resources are effectively saved and computing efficiency is improved.

[0120] For example, the member account associated with the job identifier subscribed to the job push service and renewed the subscription during the validity period. Thus, although a job-related information change event is generated, the job description information corresponding to the job identifier does not change in this event. Therefore, based on the embodiments of this application, the specific change information in each job-related information change event can be fully explored, and the situation where the model does not need to be called can be identified, thereby effectively saving computing resources and improving data synchronization efficiency.

[0121] According to the embodiments of this application, since the storage links of each job identifier in the feature database, as well as the job description information and job feature information corresponding to each job identifier, are clear, the data query efficiency can be effectively improved, thereby helping to improve the data synchronization speed.

[0122] In some embodiments of this application, the data usage needs of downstream consumers are considered, along with efforts to improve the service quality of recruitment services. Job feature generation models often require version updates to provide downstream consumers with more reliable feature data. Based on this, embodiments of this application may further include steps 401 to 404.

[0123] Step 401: Receive the model version update notification, which includes the model update time of the second target model in the model library;

[0124] Step 402: Obtain the latest job description information corresponding to each second job identifier in the feature database. The latest job description information corresponding to the second job identifier is stored in the feature database earlier than the model update time of the second target model.

[0125] Step combination Figure 4 As shown, the latest version of the second target model in the model library is called, and the latest job description information corresponding to each second job identifier is parsed through the latest version of the second target model to obtain the second job feature information corresponding to each second job identifier;

[0126] At step 404, the second job feature information and the model update time of each second target model are stored in the feature database corresponding to each second job identification.

[0127] Specifically, each job identification is recorded in the feature database, and the job identification corresponds to job description information and job feature information. The job description information corresponds to a time of being stored in the feature database. Optionally, the time of storing the job description information in the feature database can correspond to the job identification, and data query is facilitated.

[0128] Since the latest version of the model is used to process the job description information in each data synchronization process, after receiving the update notification each time, the latest time of storing the job description information corresponding to the job identification in the feature database can be obtained, the second job identification corresponding to the job description information is obtained, and the second target model and performance processing are used to obtain the job feature information of the job description information corresponding to the second job identification, that is, the second job feature information.

[0129] According to the embodiments of the present application, after the model version is updated each time, the historical job feature information can be easily found, and the data processing of the job description information corresponding to all job identifications in the feature database is not required, thereby effectively reducing the data processing amount, realizing the data processing logic of quickly synchronizing the generation of job feature data of each job description information, and providing more reliable feature data to downstream consumers to meet the data use requirements of consumers.

[0130] It can be understood that, according to the embodiments of the present application, if the latest job description information corresponding to the job identification in the feature database is stored in the feature database at a time later than the update time of the second target model, the latest version of the second target model is directly used, thereby avoiding repeated calling of the second target model and repeated calculation of the job description information stored at a time later than the update time. Especially when there are two or more downstream consumers that need to perform unified logical processing on a certain preset field, the newly added job description information can be processed before being stored in the job feature database, thereby avoiding repeated calculation and saving resources.

[0131] In some embodiments of the present application, after new data is stored in the feature database, the following processing mode can be included for data synchronization processing to downstream consumers.

[0132] Exemplarily, in response to the feature database storing the first job description information and the target job feature information corresponding to the first job identification, a job synchronization record is generated, the job synchronization record including a data synchronization type of the first job identification; the job synchronization record is stored in a preset message queue, so that a data consumer determines whether to read the job synchronization record according to the data synchronization type; wherein the job synchronization record further includes the first job description information and the target job feature information corresponding to the first job identification.

[0133] The type of data synchronization can be included in the job information, and the data synchronization type can be synchronized and stored in the feature database corresponding to the job identification. The data synchronization type includes but is not limited to reprint job import, job File import, job label Heat import, industry label synchronization, Oracle library (super exposure business), member dimension real-time data, incremental fusion data, job value-added service data, sample calculation storage, job filter, and the like. The specific data synchronization type can be set according to actual application requirements, and is not listed here.

[0134] Optionally, when the feature database responds to model version update to perform data synchronization processing, the data synchronization type of the job feature data generated based thereon can be set according to synchronization requirements. By setting the data synchronization type, the downstream consumer can automatically identify whether the newly added data in the feature database is the required data.

[0135] In the embodiments of the present application, after new data is stored in the feature database, any newly added data is captured and processed to ensure the timeliness and accuracy of data synchronization. Optionally, the Hbase coprocessor can be used to listen to and capture new data stored in the feature database, such as job description information and job feature information.

[0136] Optionally, the conversion field can also be included in the job description information stored in the feature database, for example, the job information often includes a large amount of text information. In order to facilitate data processing of the processor, the text information can be split by word segmentation and the like, and then the keywords are extracted from the word segmentation, or the labels are added according to the word segmentation, wherein the keywords and the labels can be used as the conversion field. Corresponding to the job identification, the preset field of the job description information, the conversion field and the job feature information are stored in the job feature database to ensure efficient storage and retrieval of data.

[0137] In some embodiments, optionally, the preset message queue is a data synchronization channel of the feature database and the downstream consumer. Optionally, the Kafka message queue can be used as the data synchronization channel, which is not specifically limited here.

[0138] According to the embodiment of the present application, through the preset message queue, the downstream consumer can conveniently know the updated data in the feature database, and after the downstream consumer identifies the required data through the data synchronization type, the downstream consumer can read the data in time, thereby realizing the quick synchronization of the message.

[0139] Corresponding to the method embodiment of the present application, the present application also provides a data synchronization device, as shown in the figure, Figure 5 The data synchronization device 500 comprises:

[0140] The acquisition module 501 is configured to acquire position information, wherein the position information comprises a first position identifier and first position description information corresponding to the first position identifier, and the first position description information comprises first field data corresponding to a preset field;

[0141] The processing module 502 is configured to determine a first target model according to the difference between the position information and historical position information recorded in the feature database, and version update data of a position feature generation model used by the historical position information, wherein the historical position information comprises a historical position identifier and historical position description information corresponding to the historical position identifier;

[0142] The processing module 502 is further configured to call the first target model, and parse the first field data in the first position description information through the first target model, so as to obtain target position feature information corresponding to the first position description information;

[0143] The processing module 502 is further configured to store the first position description information and the target position feature information in the feature database corresponding to the first position identifier, so as to provide consumption data to the data consumer through the feature database.

[0144] In some embodiments, the number of the first target models is at least one, and the target position feature information comprises at least one position feature;

[0145] The processing module 502 is further configured to determine the preset field required by each first target model;

[0146] The processing module 502 is further configured to input the first field data corresponding to the preset field required by each first target model into the first target model, and output the position feature through the first target model;

[0147] The acquisition module 501 is further configured to acquire the position feature output by each first target model, so as to obtain the target position feature information corresponding to the first position description information.

[0148] In some embodiments, the processing module 502 is further configured to traverse each historical position identifier in the feature database;

[0149] The processing module 502 is further configured to determine a model version of each position feature generation model in the model library in a case where the feature database does not record the first position identifier.

[0150] The processing module 502 is further configured to determine each latest version of the position feature generation model as the first target model.

[0151] In some embodiments, the number of the position feature generation models corresponding to each historical position identifier is at least one.

[0152] The processing module 502 is further configured to determine, in a case where the feature database records the first position identifier, a latest version and a first model version of each first position feature generation model according to the version update data, wherein the first position feature generation model of the first model version is used to parse historical position description information corresponding to the first position identifier.

[0153] The processing module 502 is further configured to determine the first position feature generation model as the first target model in a case where the first model version is different from the latest version.

[0154] In some embodiments, the historical position description information corresponding to the first position identifier includes second field data of a preset field.

[0155] The processing module 502 is further configured to determine, in a case where the feature database records the first position identifier and the first model version of the first position feature generation model is the latest version, first field data and second field data of the preset field required by each first position feature generation model.

[0156] The processing module 502 is further configured to determine the first position feature generation model as the first target model in a case where the first field data and the second field data of the preset field required by the first position feature generation model are inconsistent.

[0157] In some embodiments, the processing module 502 is further configured to obtain, in the first position description information, the first field data of the preset field required by the first position feature generation model, and perform hash calculation on the first field data to obtain a first hash value.

[0158] The processing module 502 is further configured to obtain, in the historical position information, the second field data of the preset field required by the first position feature generation model, and perform hash calculation on the second field data to obtain a second hash value.

[0159] The processing module 502 is further configured to determine, in a case where the first hash value and the second hash value are inconsistent, that the first field data and the second field data of the preset field required by the first position feature generation model are inconsistent.

[0160] In some embodiments, the processing module 502 is further configured to, in a case where the first field data and the second field data of the preset field required by the first position feature generation model are consistent, determine the historical position description information corresponding to the position feature information of the first position identification as the target position feature information corresponding to the first position description information.

[0161] In some embodiments, the processing module 502 is further configured to receive an update notification of the model version, the update notification including a model update time of the second target model in the model library;

[0162] The acquisition module 501 is further configured to acquire the latest position description information corresponding to each second position identification in the feature database, wherein the latest position description information corresponding to each second position identification is stored in the feature database at a time earlier than the model update time of the second target model;

[0163] The processing module 502 is further configured to call the latest version of the second target model in the model library, and parse the latest position description information corresponding to each second position identification by using the latest version of the second target model to obtain the second position feature information corresponding to each second position identification;

[0164] The second position feature information and the model update time of each second target model are stored in the feature database corresponding to each second position identification.

[0165] In some embodiments, the processing module 502 is further configured to, in response to the feature database storing the first position description information and the target position feature information corresponding to the first position identification, generate a position synchronization record, the position synchronization record including a data synchronization type of the first position identification;

[0166] The processing module 502 is further configured to store the position synchronization record in a preset message queue, so that a data consumer determines whether to read the position synchronization record according to the data synchronization type;

[0167] The position synchronization record further includes the first position description information and the target position feature information corresponding to the first position identification.

[0168] It can be understood that the data synchronization apparatus of the embodiments of the present application can correspond to the execution subject of the data synchronization method provided by the embodiments of the present application, and the specific details of the operation and / or function of each module / unit of the data synchronization apparatus can be referred to the description of the corresponding part in the data synchronization method provided by the embodiments of the present application. For the sake of brevity, it will not be repeated here.

[0169] The electronic device in the embodiments of the present application can be a user terminal device, can be a server, can also be other computing devices, and can also be a cloud server. Figure 6A hardware structure schematic diagram of an electronic device of an embodiment of the present application is shown, which can include a processor 601 and a memory 602 storing computer program instructions, and the processor 601 implements the flow or function of the method of any of the above embodiments when executing the computer program instructions.

[0170] Specifically, the processor 601 can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or can be configured to implement one or more integrated circuits of the embodiments of the present application. The memory 602 can include a mass storage for data or instructions. For example, the memory 602 can be at least one of a hard disk drive (HDD), a read-only memory (ROM), a random access memory (RAM), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, a universal serial bus (USB) drive, or other physical / tangible memory storage device. For another example, the memory 602 can include a removable or non-removable (or fixed) medium. For yet another example, the memory 602 can be internal or external to the integrated gateway disaster recovery device. The memory 602 can be a non-volatile solid-state memory. In other words, generally the memory 602 includes a tangible (non-transitory) computer-readable storage medium (such as a memory device) encoded with computer-executable instructions, and when the software is executed (such as by one or more processors), can perform the operations described in the method of the embodiments of the present application. The processor 601 implements the flow or function of the method of any of the above embodiments by reading and executing the computer program instructions stored in the memory 602.

[0171] In one example, Figure 6The electronic device shown can also include a communication interface 603 and a bus 610. Among them, the processor 601, the memory 602, the communication interface 603 are connected through the bus 610 and complete the communication between each other. The communication interface 603 is mainly used to realize the communication between the modules, devices, units and / or equipment in the embodiments of the application. The bus 610 includes hardware, software or both, which can couple the components of the online data traffic billing device to each other. For example, the bus can include at least one of the following: an accelerated graphics port (AGP) or other graphics bus, an enhanced industry standard architecture (EISA) bus, a front side bus (FSB), a hyper transport (HT) interconnect, an industry standard architecture (ISA) bus, an infiniband interconnect, a low pin count (LPC) bus, a memory bus, a micro channel architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a serial advanced technology attachment (SATA) bus, a video electronics standards association local (VLB) bus or other suitable bus. The bus 610 can include one or more buses. Although the embodiments of the application describe or show a specific bus, any suitable bus or interconnection method can be considered by the embodiments of the application.

[0172] In combination with the method in the above embodiments, the embodiments of the application further provide a computer readable storage medium, which has stored thereon computer program instructions, and the computer program instructions are executed by a processor to implement the flow or function of any of the methods in the above embodiments.

[0173] In addition, the embodiments of the application also provide a computer program product, which has stored thereon computer program instructions, and the computer program instructions are executed by a processor to implement the flow or function of any of the methods in the above embodiments.

[0174] The above exemplary describes the flowcharts and / or block diagrams of the methods, devices, systems and computer program products of the embodiments of the application, and describes the related aspects. It should be understood that each block in the flowchart and / or block diagram, or a combination thereof, can be implemented by computer program instructions, or by special hardware that performs specified functions or actions, or by a combination of special hardware and computer instructions. For example, these computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, to form a machine, so that the instructions executed by the processor enable the implementation of the functions / actions specified in each block or a combination thereof in the flowchart and / or block diagram. Such a processor can be a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit.

[0175] The functional blocks shown in the structural block diagram of the embodiments of the present application can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc.; when implemented in software, it is a program or code segment used to perform the required tasks. The program or code segment can be stored in a memory or transmitted through a data signal carried in a carrier wave over a transmission medium or communication link. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0176] It should be noted that the present application is not limited to the specific configurations and processes described above or shown in the drawings. The above description is merely a specific implementation of the present application, and those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the described systems, devices, modules or units can refer to the corresponding processes in the method embodiments, which need not be described again. It should be understood that the protection scope of the present application is not limited thereto, and any person skilled in the art can think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be covered within the protection scope of the present application.

Claims

1. A data synchronization method, characterized by, The method comprises the following steps: obtaining position information, wherein the position information comprises a first position identifier and first position description information corresponding to the first position identifier, and the first position description information comprises first field data corresponding to a preset field; determining a first target model according to a difference between the position information and historical position information recorded in a feature database and version update data of a position feature generation model used by the historical position information, wherein the historical position information comprises a historical position identifier and historical position description information corresponding to the historical position identifier, and the number of position feature generation models corresponding to each historical position identifier is at least one; calling the first target model and analyzing the first field data in the first position description information by using the first target model to obtain target position feature information corresponding to the first position description information; storing the first position description information and the target position feature information corresponding to the first position identifier in the feature database to provide consumption data to a data consumer through the feature database; wherein the determination of the first target model according to the difference between the position information and the historical position information recorded in the feature database and the version update data of the position feature generation model used by the historical position information comprises: traversing each historical position identifier in the feature database; in the case that the feature database does not record the first position identifier, determining the model version of each position feature generation model in a model library; and determining each latest version of the position feature generation model as the first target model; in the case that the feature database records the first position identifier, determining the latest version and a first model version of each first position feature generation model according to the version update data, wherein the first position feature generation model of the first model version is used to analyze the historical position description information corresponding to the first position identifier; and in the case that the first model version is different from the latest version, determining the first position feature generation model as the first target model.

2. The method of claim 1, wherein, The number of the first target models is at least one, and the target position feature information comprises at least one position feature. The calling of the first target model and the analysis of the first field data in the first position description information by using the first target model to obtain the target position feature information corresponding to the first position description information comprises: determining preset fields required by each first target model; inputting the first field data corresponding to the preset fields required by each first target model into the first target model and outputting position features by using the first target model; obtaining the position features output by each first target model to obtain the target position feature information corresponding to the first position description information.

3. The method of claim 1, wherein, The historical position description information corresponding to the first position identifier comprises second field data corresponding to the preset field. The first target model is determined according to a difference between the position information and historical position information recorded in the feature database, and version update data of a position feature generation model used by the historical position information, and the method further comprises: In a case where the first position identifier is recorded in the feature database, and a first model version of the first position feature generation model is a latest version, first field data and second field data of a preset field required by each of the first position feature generation models are determined; In a case where the first field data and the second field data of the preset field required by the first position feature generation model are inconsistent, the first position feature generation model is determined as the first target model.

4. The method of claim 3, wherein, The method further comprises: The first field data of the preset field required by the first position feature generation model is obtained from the first position description information, and a first hash value is obtained by performing hash calculation on the first field data; The second field data of the preset field required by the first position feature generation model is obtained from the historical position information, and a second hash value is obtained by performing hash calculation on the second field data; In a case where the first hash value and the second hash value are inconsistent, it is determined that the first field data and the second field data of the preset field required by the first position feature generation model are inconsistent.

5. The method of claim 3, wherein, Before the first position description information and the target position feature information corresponding to the first position identifier are stored in the feature database, the method further comprises: In a case where the first field data and the second field data of the preset field required by the first position feature generation model are consistent, historical position description information corresponding position feature information of the first position identifier is determined as target position feature information corresponding to the first position description information.

6. The method of claim 1, wherein, The method further comprises: An update notification of a model version is received, and the update notification comprises a model update time of a second target model in a model library; The latest position description information corresponding to each second position identifier in the feature database is obtained, wherein the latest position description information corresponding to each second position identifier is stored in the feature database at a time earlier than the model update time of the second target model; The latest version of the second target model in the model library is called, and the latest version of the second target model is used to analyze the latest position description information corresponding to each second position identifier to obtain second position feature information corresponding to each second position identifier; The second position feature information and the model update time of each second target model are stored in the feature database corresponding to each second position identifier.

7. The method of claim 1, wherein, The method further comprises: In response to the feature database storing the first position description information and the target position feature information corresponding to the first position identifier, a position synchronization record is generated, and the position synchronization record comprises a data synchronization type of the first position identifier; The position synchronization record is stored in a preset message queue, so that the data consumer determines whether to read the position synchronization record according to the data synchronization type. The position synchronization record further includes the first position description information corresponding to the first position identifier and the target position characteristic information.

8. A data synchronization apparatus, characterized by comprising: The method comprises: The acquisition module is configured to acquire position information, the position information comprising a first position identifier and first position description information corresponding to the first position identifier, the first position description information comprising first field data corresponding to a preset field; The processing module is configured to determine a first target model based on a difference between the position information and historical position information recorded in a characteristic database, and version update data of a position characteristic generation model used by the historical position information, wherein the historical position information comprises a historical position identifier and historical position description information corresponding to the historical position identifier, and the number of position characteristic generation models corresponding to each historical position identifier is at least one; The processing module is further configured to call the first target model and analyze the first field data in the first position description information by using the first target model to obtain target position characteristic information corresponding to the first position description information; The processing module is further configured to store the first position description information and the target position characteristic information in the characteristic database corresponding to the first position identifier, so as to provide consumption data to a data consumer through the characteristic database; The processing module is further configured to traverse each historical position identifier in the characteristic database; in a case where the first position identifier is not recorded in the characteristic database, determine a model version of each position characteristic generation model in a model library; determine each latest version of the position characteristic generation model as the first target model; in a case where the first position identifier is recorded in the characteristic database, determine a latest version and a first model version of each first position characteristic generation model based on the version update data, wherein the first position characteristic generation model of the first model version is used to analyze historical position description information corresponding to the first position identifier; in a case where the first model version is different from the latest version, determine the first position characteristic generation model as the first target model.

9. An electronic device, comprising: The electronic device comprises a processor and a memory storing computer program instructions; and the electronic device implements the method of any one of claims 1-7 when executing the computer program instructions.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer program instructions, and the computer program instructions are executed by the processor to implement the method of any one of claims 1-7.

11. A computer program product, characterised in that, The computer program instructions are executed by the processor to implement the method of any one of claims 1-7.

Images