Personnel archive test data generation method, system and device and medium

Abstract

The invention discloses a personnel archive test data generation method, system and device and a medium, and the method comprises the steps: responding to a generation request of a user, starting a service system, and obtaining a pre-storage address of personnel archive test data; the service system comprises an AI face changing service interface and a feature vector extraction interface; calling an AI face changing service interface, and generating a plurality of personnel archive pictures; determining a first hash value corresponding to each personnel file picture; determining a target picture set according to the first hash value and a pre-established database; the database comprises a plurality of preset pictures, and the target picture set comprises a personnel archive picture and at least one preset picture; calling a feature vector extraction interface, performing feature extraction on preset pictures in the target picture set, and obtaining a feature vector corresponding to each preset picture; and assembling the target picture set, the first hash value, the feature vector and the storage address into Kafka message data. The method can be widely applied to the technical field of data processing.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, system, device and storage medium for generating personnel profile test data. Background Art

[0002] In the modern information society, with the expansion of the scale of enterprises and institutions and the complexity of their business, personnel file management has become an indispensable part of organizational management. However, in actual operations, the generation and management of personnel files face many challenges. Especially in scenarios such as software development, system testing, or business process simulation, it is often necessary to build a large amount of simulated personnel file data to verify the stability, accuracy, and efficiency of the system.

[0003] Traditional methods usually rely on manual input or script generation based on specific business logic. These methods are not only time-consuming and laborious, but also difficult to ensure the diversity and authenticity of data, which will affect the test efficiency and the accuracy of data analysis. Therefore, there are still technical problems that need to be solved in the relevant technology. Summary of the invention

[0004] The purpose of this application is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0005] To this end, an object of an embodiment of the present application is to provide a method, system, device and storage medium for generating personnel profile test data, which can improve the efficiency and flexibility of testing work.

[0006] In order to achieve the above-mentioned technical objectives, the technical scheme adopted by the embodiments of the present application includes: a method for generating personnel profile test data, comprising: in response to a user's generation request, starting a service system and obtaining a pre-set storage address of personnel profile test data; the service system includes an AI face-changing service interface and a feature vector extraction interface; calling the AI ​​face-changing service interface to generate a number of personnel profile pictures; determining a first hash value corresponding to each of the personnel profile pictures; determining a target picture set based on the first hash value and a pre-established database; the database includes multiple preset pictures, and the target picture set includes the personnel profile picture and at least one preset picture; calling the feature vector extraction interface to perform feature extraction on the preset pictures in the target picture set to obtain a feature vector corresponding to each of the preset pictures; assembling the target picture set, the first hash value, the feature vector and the storage address into Kafka message data.

[0007] This embodiment can generate several personnel profile pictures by calling the AI ​​face-changing service interface; determine the first hash value corresponding to each personnel profile picture; determine the target picture set from the database according to the first hash value; call the feature vector extraction interface to extract features from the target picture set to obtain feature vectors; assemble the target picture set, the first hash value, the feature vector and the storage address into Kafka message data. This application stores key test data such as personnel profile pictures and feature vectors persistently and establishes corresponding data archives. When the system environment is switched, only the relevant data needs to be pushed to the new test environment through message queue technologies such as Kafka, and the same test scenarios and data sets as the original ones can be quickly constructed. This application can avoid the tedious process of repeatedly generating test data, save a lot of time and resources, and improve the efficiency and flexibility of testing work.

[0008] In addition, the method for generating personnel profile test data according to the above embodiment of the present invention may also have the following additional technical features:

[0009] Furthermore, in an embodiment of the present application, determining the target picture set based on the first hash value and a pre-established database specifically includes: determining the second hash value corresponding to each of the preset pictures; when the first hash value corresponding to the personnel profile picture is the same as the second hash value corresponding to any of the preset pictures, determining the personnel profile picture as the profile picture to be processed; re-calling the AI ​​face-changing service interface to generate a personnel profile picture that has a different second hash value from that corresponding to all the preset pictures; replacing the profile picture to be processed with the newly generated personnel profile picture, and taking the personnel profile picture and the set of preset pictures in the database as the target picture set.

[0010] Furthermore, in an embodiment of the present application, assembling the target picture set, the first hash value, the feature vector and the storage address into Kafka message data includes: assembling the target picture set, the first hash value, the feature vector and the storage address into a facial feature data table; regularly reading updated picture information in the target picture set from the facial feature data table according to a preset time interval, and assembling the updated picture information into Kafka message data.

[0011] Furthermore, in an embodiment of the present application, determining the first hash value corresponding to each of the personnel profile pictures specifically includes: filling each of the personnel profile pictures with picture data of a preset length; dividing the picture data into a number of picture data blocks of fixed length; and determining the first hash value based on the picture data blocks.

[0012] Furthermore, in an embodiment of the present application, determining the first hash value based on the picture data block specifically includes: configuring an initial hash value; for any data block, inputting the initial hash value and the current picture data block into a compression function to obtain an intermediate hash value; using the intermediate hash value as the initial hash value for processing the next data block, and returning to the step of inputting the initial hash value and the current picture data block into the compression function to obtain the intermediate hash value, until all the picture data blocks are compressed to obtain the first hash value.

[0013] Furthermore, in an embodiment of the present application, the AI ​​face-changing service interface is an initialized service interface, and the initialization process of the AI ​​face-changing service interface includes: parameterizing the facial structure features in the AI ​​face-changing service interface, and the facial structure features include facial clothing features, skin color features, and facial features.

[0014] Furthermore, in an embodiment of the present application, the method further includes: in response to an operation of the business side subscribing to a Kafka topic, pushing the Kafka message data to the business side so that the business side generates batch personnel profile data.

[0015] On the other hand, an embodiment of the present application also provides a personnel profile test data generation system, including: a first processing unit, used to respond to a user's generation request, start a service system and obtain a preset storage address of personnel profile test data; the service system includes an AI face-changing service interface and a feature vector extraction interface; a second processing unit, used to call the AI ​​face-changing service interface to generate a number of personnel profile pictures; a third processing unit, used to determine the first hash value corresponding to each of the personnel profile pictures; a fourth processing unit, used to determine a target picture set based on the first hash value and a pre-established database; the database includes multiple preset pictures, and the target picture set includes the personnel profile picture and at least one preset picture; a fifth processing unit, used to call the feature vector extraction interface, perform feature extraction on the preset pictures in the target picture set, and obtain a feature vector corresponding to each of the preset pictures; a sixth processing unit, used to assemble the target picture set, the first hash value, the feature vector and the storage address into Kafka message data.

[0016] On the other hand, the present application also provides a personnel profile test data generating device, comprising:

[0017] at least one processor;

[0018] at least one memory for storing at least one program;

[0019] When the at least one program is executed by the at least one processor, the at least one processor implements a method for generating personnel profile test data as described in any one of the invention contents.

[0020] In addition, the present application also provides a computer-readable storage medium, which stores processor-executable instructions, and the processor-executable instructions are used to execute a personnel profile test data generation method as described in any of the above items when executed by the processor.

[0021] The advantages and benefits of the present application will be partially given in the following description, and partially become apparent from the following description, or be understood through the practice of the present application:

[0022] This application can generate several personnel profile pictures by calling the AI ​​face-changing service interface; determine the first hash value corresponding to each personnel profile picture; determine the target picture set from the database according to the first hash value; call the feature vector extraction interface to extract features from the target picture set to obtain feature vectors; assemble the target picture set, the first hash value, the feature vector and the storage address into Kafka message data. This application stores key test data such as personnel profile pictures and feature vectors persistently and establishes corresponding data archives. When the system environment is switched, only the relevant data needs to be pushed to the new test environment through message queue technologies such as Kafka, and the same test scenarios and data sets as the original ones can be quickly constructed. This application can reduce the tedious process of repeatedly generating test data, save a lot of time and resources, and improve the efficiency and flexibility of testing work. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A schematic diagram of the steps of a method for generating personnel profile test data in a specific embodiment of the present invention;

[0024] Figure 2 A schematic diagram of the steps of assembling a target picture set, a first hash value, a feature vector, and a storage address into Kafka message data in a specific embodiment of the present invention;

[0025] Figure 3 A schematic diagram of the steps of determining the first hash value corresponding to each personnel profile picture in a specific embodiment of the present invention;

[0026] Figure 4 A schematic diagram of a step of determining a first hash value according to a picture data block in a specific embodiment of the present invention;

[0027] Figure 5 A schematic diagram of the steps of a method for generating personnel profile test data in another specific embodiment of the present invention;

[0028] Figure 6 A schematic diagram of a flow chart of a method for generating personnel profile test data in a specific embodiment of the present invention;

[0029] Figure 7 A schematic diagram of a flow chart of a method for generating personnel profile test data in another specific embodiment of the present invention;

[0030] Figure 8 It is a structural schematic diagram of a personnel profile test data generation system in a specific embodiment of the present invention;

[0031] Fig. 9 It is a structural schematic diagram of a personnel profile test data generating system in another specific embodiment of the present invention;

[0032] Fig.10 It is a structural schematic diagram of a personnel profile test data generating device in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0033] The embodiments of the present invention are described in detail below with reference to the accompanying drawings to illustrate the principles and processes of the personnel profile test data generating method, system, device and storage medium in the embodiments of the present invention.

[0034] The following are necessary explanations of the nouns appearing in this application:

[0035] Kafka Message: Apache Kafka is a distributed stream processing platform, mainly used to build real-time data pipelines and stream applications. In Kafka messages, "Message" is the basic data unit of Kafka. It is the basic information block processed, stored and transmitted in the Kafka cluster. The following are some key concepts and features about Kafka messages:

[0036] 1. Message structure

[0037] Messages in Kafka usually consist of two parts:Key: Optional, an identifier used to determine which partition the message is sent to. If a key is set, Kafka will assign messages to specific partitions based on the hash value of the key, which helps ensure that messages with the same key are sent to the same partition, which is very useful for implementing ordered message processing.Value: The actual message content, which is required and can be any byte sequence, such as JSON, XML, or other custom formats.

[0038] 2. Message sending

[0039] A producer is a client that sends messages in Kafka. The producer sends messages to a specified topic, which is a logical collection of messages that can be subscribed to by one or more consumers.

[0040] 3. Message Storage

[0041] Kafka uses partitions to store messages. Each partition is an ordered, immutable sequence of messages that are continuously appended to the partition. Partitions are distributed across multiple nodes (Brokers) in the Kafka cluster to achieve high availability and scalability.

[0042] 4. Message consumption

[0043] Consumers are clients that read and process messages in Kafka. Consumers can subscribe to one or more topics and read messages from partitions of these topics. Kafka supports the concept of consumer groups. Consumers belonging to the same consumer group can read messages from a topic together, but each message will only be read by one consumer in the group (ensuring that each message is only processed once).

[0044] 5. Message persistence

[0045] Kafka provides a highly reliable and persistent message storage mechanism. Once a message is written to a Kafka cluster, it is replicated to multiple nodes to ensure that the message is not lost even in the event of a node failure.

[0046] 6. Message Order

[0047] Since Kafka guarantees the order of messages within a partition, if you need to guarantee the order of message processing, you should ensure that related messages are sent to the same partition. This is usually achieved by specifying the same key when sending messages.

[0048] 7. Message Compression

[0049] Kafka supports multiple compression algorithms (such as GZIP, Snappy, and LZ4, etc.), which can compress the message content when the producer sends the message, and then decompress it when the consumer receives it. This helps reduce the use of network bandwidth and storage space.

[0050] The technical defects of the prior art are described below:

[0051] In the modern information society, with the expansion of the scale of enterprises and institutions and the complexity of their business, personnel file management has become an indispensable part of organizational management. However, in actual operations, the generation and management of personnel files face many challenges. Especially in scenarios such as software development, system testing, or business process simulation, it is often necessary to build a large amount of simulated personnel file data to verify the stability, accuracy, and efficiency of the system.

[0052] Traditional methods usually rely on manual input or script generation based on specific business logic. These methods are not only time-consuming and laborious, but also difficult to ensure the diversity and authenticity of data, which will affect the test efficiency and the accuracy of data analysis. Therefore, there are still technical problems that need to be solved in the relevant technology.

[0053] In view of the defects existing in the above-mentioned related technologies, refer to Figure 1 , Figure 1 Schematic diagram of the steps of a method for generating personnel profile test data provided in an embodiment of the present application. Figure 1 In the method for generating personnel profile test data, the method may include but is not limited to steps S101 to S106.

[0054] S101. In response to a user's generation request, start the service system and obtain a pre-set storage address for personnel profile test data; the service system includes an AI face-changing service interface and a feature vector extraction interface.

[0055] In some embodiments of the present application, in response to a user's generation request, the processor can start a service system and obtain a pre-set storage address of personnel profile test data, wherein the service system includes an AI face-changing service interface and a feature vector extraction interface.

[0056] S102. Call the AI ​​face-changing service interface to generate several personnel profile pictures.

[0057] It is understandable that the AI ​​face-changing service interface can be a preprocessed service interface or an initialized service structure. Initialization and preprocessing can be parameter configuration or parametric processing of facial structural features. Parametric processing can be a process of converting key feature points (such as eyes, nose or mouth, etc.) and facial contours in a face image into quantifiable mathematical parameters. These parameters not only describe the shape and position information of the face, but can also be used in a variety of applications such as face recognition, face editing, and face animation. The parametric processing of facial structural parameters involves multiple aspects such as image processing, computer vision, and machine learning.

[0058] In some feasible embodiments of the application, a processor or other chip with data processing function can establish a wired or wireless connection with a processing unit capable of calling a data interface. The processor can call the AI ​​face-changing service interface, and after calling the interface, the processor runs the corresponding image generation program to generate several personnel file images.

[0059] It should be noted that the wired connection method mentioned above may include the connection between the mobile device and the processing module, and may also include the connection between the processing module and the hardware device, and other wired connections between the devices and the processing module that are currently known or will be developed in the future. The wireless connection method mentioned above may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (Ultra Wide Band) connection, and other wireless connection methods that are currently known or will be developed in the future.

[0060] S103: Determine the first hash value corresponding to each personnel file picture.

[0061] It is understandable that the first hash value may be a unique hash value corresponding to each image. In the data processing process, the first hash value may represent the corresponding image. If the first hash values ​​are the same, the corresponding images may be considered to be two identical images.

[0062] In some embodiments of the present application, the processor may determine the first hash value corresponding to each personnel profile picture according to a hash value determination algorithm. Among them, the hash value determination algorithm may be the existing secure hash algorithm 2. Secure Hash Algorithm 2 (SHA-2) is a cryptographic hash function algorithm standard issued by the National Institute of Standards and Technology (NIST). SHA-2 is the successor to SHA-1 and is designed to provide higher security and better performance. The SHA-2 algorithm family includes multiple different versions, each of which generates hash values ​​of different lengths to meet different security requirements. These versions mainly include: SHA-224, SHA-256, SHA-384, SHA-512, and SHA-512 / 224 and SHA-512 / 256. The basic structure of these algorithms is the same, but there are slight differences in the length of the generated summary and the number of times the loop is run.

[0063] S104. Determine a target picture set according to the first hash value and a pre-established database; the database includes a plurality of preset pictures, and the target picture set includes a personnel file picture and at least one preset picture.

[0064] It is understandable that the database may include preset historical personnel profile pictures, that is, several personnel profile pictures that existed before the service interface was called. The target picture set may include several historical personnel profile pictures that already exist in the database, and may also include several personnel profile pictures generated by calling the AI ​​face-changing service interface.

[0065] In some feasible embodiments of the present application, the processor can determine whether there are pictures in the plurality of personnel profile pictures that are identical to the pictures in the plurality of personnel profile pictures already existing in the database according to the first hash value. If there are identical pictures, the corresponding identical pictures are discarded, and the generated plurality of mutually different personnel profile pictures and the plurality of historical personnel profile pictures are stored in the database, and then the target picture set is determined from the database.

[0066] S105 . Call a feature vector extraction interface to perform feature extraction on preset pictures in the target picture set to obtain a feature vector corresponding to each preset picture.

[0067] It can be understood that each picture in the target picture set can correspond to a feature vector.

[0068] In some feasible embodiments of the present application, the processor may call a feature vector extraction interface and run a feature extraction program to perform feature extraction on a target picture set to obtain a feature vector corresponding to each picture.

[0069] S106: Assemble the target picture set, the first hash value, the feature vector, and the storage address into Kafka message data.

[0070] After obtaining the feature vector, the processor can run a program to write the target image set, the first hash value, the feature vector, and the storage address into a data table, and then periodically read the updated image information from the data table and assemble the updated image information into Kafka message data. The updated image information also includes information such as the first hash value, the feature vector, and the storage address.

[0071] In summary, this embodiment can generate several personnel profile pictures by calling the AI ​​face-changing service interface. Determine the first hash value corresponding to each personnel profile picture. According to the first hash value, determine the target picture set from the database. Call the feature vector extraction interface to extract features from the target picture set to obtain a feature vector. Assemble the target picture set, the first hash value, the feature vector, and the storage address into Kafka message data. This application stores key test data such as personnel profile pictures and feature vectors persistently, and establishes corresponding data archives. When the system environment is switched, only the relevant data needs to be pushed to the new test environment through message queue technologies such as Kafka, and the same test scenarios and data sets as the original ones can be quickly constructed. This application can avoid the tedious process of repeatedly generating test data, save a lot of time and resources, and improve the efficiency and flexibility of testing work.

[0072] Specifically, in some other embodiments, the AI ​​face-changing service interface is called, and the processor generates several personnel file pictures. After hash comparison, the processor can remove duplicate pictures. Then, the processor can save the deduplicated pictures to the storage server, and save the storage address and hash value to the database; call the feature vector extraction interface, the processor can read data from the database, extract features to obtain feature vectors, and then store the picture address and feature vectors persistently and assemble the feature vectors and picture storage addresses into a Kafka message body for push. This embodiment can store key test data such as pictures and feature vectors persistently and establish corresponding data archives. When the system environment is switched, the relevant data can be pushed to the new test environment through message queue technologies such as Kafka to quickly construct the same test scenario and data set as the original one. That is, the persistent picture address and feature vector data are directly read and assembled into a Kafka message for push.

[0073] Furthermore, the step of determining the target picture set according to the first Hash value and the pre-established database may specifically include but is not limited to steps S201 to S204.

[0074] S201. Determine a second hash value corresponding to each preset image.

[0075] S202: When the first hash value corresponding to the personnel file picture is the same as the second hash value corresponding to any preset picture, the personnel file picture is determined as the file picture to be processed.

[0076] S203. Recall the AI ​​face-changing service interface to generate a personnel profile picture that has a different second hash value corresponding to all preset pictures.

[0077] S204, using the newly generated personnel file picture to replace the to-be-processed file picture, and taking the personnel file picture and a collection of preset pictures in the database as a target picture set.

[0078] In some feasible embodiments of the present application, the processor may first determine the second hash value corresponding to each personnel profile picture in the database. After obtaining the second hash value, when the first hash value corresponding to any personnel profile picture is the same as the second hash value corresponding to any preset picture in the database, the AI ​​face-changing service interface is re-called to regenerate a personnel profile picture that is different from the second hash value corresponding to all preset pictures, and use the newly generated personnel profile picture to replace the profile picture to be processed, and use the personnel profile picture and the set of preset pictures in the database as the target picture set. When the first hash value corresponding to any personnel profile picture is different from the second hash value corresponding to all preset pictures in the database, a set of several personnel profile pictures and preset pictures is used as the target picture set.

[0079] Further, refer to Figure 2 , Figure 2 This is a schematic diagram of the steps of assembling the target image set, the first hash value, the feature vector and the storage address into Kafka message data in an embodiment of the present application. Figure 2 In the step, the step may include but is not limited to step S301-step S302.

[0080] S301, assembling a target picture set, a first hash value, a feature vector and a storage address and writing them into a facial feature data table.

[0081] S302: According to a preset time interval, regularly read updated picture information in the target picture set from the facial feature data table, and assemble the updated picture information into Kafka message data.

[0082] In some feasible embodiments of the present application, the target picture set, the first hash value, the feature vector and the storage address are assembled and written into the facial feature data table. The facial feature data table may be a table with a preset format and a preset category, and the preset category may include pictures, hash values ​​and feature vectors, etc. As the processor continuously calls the AI ​​face-changing service interface. The pictures in the facial feature data table are constantly updated, and what becomes the Kafka message data is the picture information in the constantly updated facial feature data table. Therefore, the processor of this embodiment can regularly read the updated picture information in the target picture set from the facial feature data table according to a preset time interval, and assemble the updated picture information into Kafka message data.

[0083] Further, refer to Figure 3 , Figure 3This is a schematic diagram of the steps of determining the first hash value corresponding to each personnel profile picture in the embodiment of the present application. Figure 3 In the step, the step may include but is not limited to step S401 and step S403.

[0084] S401, filling each personnel file picture with picture data of a preset length.

[0085] S402: Divide the picture data into a number of picture data blocks of fixed length.

[0086] S403: Determine a first hash value according to the picture data block.

[0087] In some feasible embodiments of the present application, the processor can fill each personnel profile picture with picture data of a preset length, such as a 512-bit length binary number. After obtaining the picture data, the processor can divide the picture data into a number of picture data blocks of fixed length. The specific number of picture data blocks can be adjusted according to user needs. After obtaining the picture data blocks, the processor can run a preset program according to the picture data blocks to determine the first hash value.

[0088] Further, refer to Figure 4 , Figure 4 This is a schematic diagram of the steps of determining a first hash value according to a picture data block in an embodiment of the present application. Figure 4 In the step, the step may include but is not limited to step S501-step S503.

[0089] S501. Configure an initial hash value.

[0090] S502: For any data block, input the initial hash value and the current image data block into a compression function to obtain an intermediate hash value.

[0091] S503: Use the intermediate hash value as the initial hash value for the next data block to be processed, and return to the step to input the initial hash value and the current picture data block into the compression function to obtain the intermediate hash value, until all picture data blocks are compressed to obtain the first hash value.

[0092] In some feasible embodiments of the present application, the processor can configure an initial hash value. After obtaining the initial hash value, for any data block, the processor can input the initial hash value and the current image data block into a compression function to obtain an intermediate hash value. The intermediate hash value can be a hash value obtained by inputting the compression function. After obtaining the intermediate hash value corresponding to the current image data block, the processor can use the intermediate hash value as the initial hash value for the next data block to be processed, and return to the step of inputting the initial hash value and the current image data block into the compression function to obtain the intermediate hash value, until all image data blocks are compressed, and finally the first hash value can be obtained.

[0093] Furthermore, the AI ​​face-changing service interface is an initialized service interface, and the initialization of the AI ​​face-changing service interface may include: parameterizing facial structure features in the AI ​​face-changing service interface.

[0094] In some feasible embodiments of the present application, the facial structure features in the AI ​​face-changing service interface are parameterized, and the facial structure features include facial clothing features, skin color features, and facial features.

[0095] Further, refer to Figure 5 , Figure 5 FIG. 1 is a schematic diagram of the steps of another method for generating personnel profile test data of the present application. Figure 5 In addition to steps S101-S106, the personnel profile test data generation method of the present application also includes step S107.

[0096] S107. In response to the operation of the business side subscribing to the Kafka topic, push Kafka message data to the business side, so that the business side generates batch personnel file data.

[0097] In some feasible embodiments of the present application, after the processor assembles the target image set, the first hash value, the feature vector, and the storage address into Kafka message data, when the user subscribes to the Kafka topic on the business side, the processor can push the Kafka message data to the business side, so that the business side generates batch personnel profile data.

[0098] Specifically, the processor can call the AI ​​face-changing service interface to generate several personnel profile pictures. After generating several personnel profile pictures, the processor can determine the first hash value corresponding to each personnel profile picture. After obtaining the first hash value, the processor can determine the target picture set from the database according to the first hash value. After obtaining the target picture set, the processor can call the feature vector extraction interface to perform feature extraction on the target picture set to obtain a feature vector. After obtaining the feature vector, the processor can assemble the target picture set, the first hash value, the feature vector, and the storage address into Kafka message data. After the processor assembles the target picture set, the first hash value, the feature vector, and the storage address into Kafka message data, when the user subscribes to the Kafka topic on the business side, the processor can push the Kafka message data to the business side, so that the business side generates batch personnel profile data.

[0099] The specific data processing principle of this application is explained below with reference to the accompanying drawings:

[0100] Reference Figure 6 as well as Figure 7 The steps of the personnel profile test data generation method of this embodiment include steps S1 to S4.

[0101] S1, AI face-changing technology interface call and parameterization. Step S1 includes steps S11 and S12.

[0102] S11. Interface selection and configuration: Use the open source AI face-changing service interface and configure the interface parameters according to business needs to ensure that diverse samples can be generated.

[0103] S12. Parametric processing: Perform detailed parametric processing on the facial structural parameters in the AI ​​face-changing interface, including but not limited to wearing glasses, masks, hats, skin color, and facial features, to enrich the diversity of samples.

[0104] S2, designing an efficient storage database. Step S2 may include steps S21 and S22.

[0105] S21. Database design: Create a special data table faces containing the necessary fields: id (auto-increment primary key), image_hash (photo hash value SHA-256), image_feature (photo feature vector) and image_url (photo storage address).

[0106] S22. Index optimization: Create a unique index for the image_hash field to ensure fast deduplication and reduce storage space usage. At the same time, consider indexing fields such as image_url or image_feature to support fast queries.

[0107] S3, photo deduplication storage and feature extraction. Step S3 may include steps S31 to S33.

[0108] S31. Photo deduplication: The SHA-256 hash value of each photo generated by the AI ​​face-changing interface is calculated and compared with the hash values ​​already in the database. If a duplicate is found, the AI ​​face-changing interface is called again to generate a new photo until a unique photo is generated.

[0109] S32. For photos that pass the deduplication check, use the S3 storage service and use Python's botocore library to implement the writing of photos, generate a photo path, and return the path as a reference to the photo. S3 storage is a highly scalable, secure, and cost-effective object storage service. The botocore library is a powerful low-level interface library that allows developers to interact with multiple services in Web Services (AWS). The main functions of the botocore library are: API design: botocore adopts a low-level API design, allowing developers to directly call various AWS services such as EC2, S3, and DynamoDB. Client instance: With botocore, developers can create client instances, send requests, and receive responses without the need for an additional abstraction layer. Dynamic discovery: botocore supports dynamic discovery of new AWS services, as well as automatic handling of retries and error control.

[0110] S33, feature extraction and storage: For photos that pass the deduplication check, call the existing business feature vector extraction interface to generate a feature vector, and write the photo, hash value, feature vector and storage address into the face feature data table (faces data table).

[0111] S4, Kafka data push and business processing. Step S4 may include steps S41 to S43.

[0112] S41. Data assembly: Read new or updated photo information from the facial feature data table regularly or on demand, and assemble it into Kafka message format.

[0113] S42. Batch push: Use Python's confluent-kafka library to implement batch push of Kafka messages.

[0114] S43, Business processing: The business side subscribes to the Kafka topic, receives and processes the pushed facial photo information, performs operations such as file generation and data analysis, and realizes batch construction of personnel file data.

[0115] In summary, the personnel profile test data generation method of the present application has the following advantages:

[0116] 1. Traditional methods of generating simulated data manually or by script often fail to ensure the diversity and authenticity of the data, thus affecting the test results and the accuracy of data analysis. This application can automatically generate highly realistic and diverse simulated data by introducing advanced technologies such as AI face-changing to improve the authenticity and reliability of data simulation.

[0117] 2. In a big data environment, data processing efficiency is one of the key factors affecting system performance. This application can achieve rapid data processing and circulation and improve the overall efficiency of data processing by designing an efficient data processing process, including steps such as data deduplication, feature extraction, and database optimization.

[0118] 3. In the traditional test environment, whenever the system environment is switched or updated, it is often necessary to regenerate the full-link test data, which is not only time-consuming and labor-intensive, but also easily leads to a waste of resources. The present application solves this problem by realizing the multiple use of test data. Specifically, the present invention stores key test data such as pictures and feature vectors persistently, and establishes corresponding data archives. When the system environment is switched, the relevant data can be pushed to the new test environment through message queue technologies such as Kafka to quickly construct the same test scenario and data set as the original one. The present application can avoid the tedious process of repeatedly generating test data, save a lot of time and resources, and improve the efficiency and flexibility of testing work. At the same time, since the test data is generated based on the real environment, it also has higher authenticity and reliability.

[0119] In addition, refer to Figure 8 ,and Figure 1 Corresponding to the method, a personnel profile test data generation system is also provided in the embodiment of the present application. The system may include: a first processing unit 1001, a second processing unit 1002, a third processing unit 1003, a fourth processing unit 1004, a fifth processing unit 1005 and a sixth processing unit 1006. Among them, the first processing unit 1001 can start the service system and obtain the preset storage address of the personnel profile test data in response to the user's generation request; the service system includes an AI face-changing service interface and a feature vector extraction interface. The second processing unit 1002 can be used to call the AI ​​face-changing service interface to generate several personnel profile pictures. The third processing unit 1003 can be used to determine the first hash value corresponding to each personnel profile picture. The fourth processing unit 1004 can be used to determine the target picture set according to the first hash value and a pre-established database; the database includes multiple preset pictures, and the target picture set includes a personnel profile picture and at least one preset picture. The fifth processing unit 1005 can be used to call the feature vector extraction interface to extract features from the preset pictures in the target picture set to obtain the feature vector corresponding to each preset picture. The sixth processing unit 1006 may be configured to assemble the target picture set, the first hash value, the feature vector, and the storage address into Kafka message data.

[0120] It should be noted that the acquisition unit can be any integrated circuit unit or microprocessor unit obtained by integrating a chip having a processing function and its peripheral circuits through existing integration technology. The first processing unit and the second processing unit can also be any integrated circuit module or microprocessor module obtained by integrating a chip having a processing function and its peripheral circuits through existing integration technology. The first processing unit and the second processing unit can also include one or more memories. One or more memories can be used to store the specific algorithm used for compression adjustment processing in this application.

[0121] In some embodiments of the present application, the first processing unit 1001 can be set in the same gateway or device with a processor as the second processing unit 1002. The first processing unit 1001 can start the service system and obtain the preset storage address of the personnel file test data in response to the user's generation request; the service system includes an AI face-changing service interface and a feature vector extraction interface. The second processing unit 1002 can call the AI ​​face-changing service interface through the chip inside its own processor to generate several personnel file pictures. The third processing unit 1003 can receive several personnel file pictures, determine the first hash value corresponding to each personnel file picture and send it to the fourth processing unit 1004. The fourth processing unit 1004 can receive the first hash value, and determine the target picture set according to the first hash value and a pre-established database. The fifth processing unit 1005 can receive the target picture set and call the feature vector extraction interface to extract features from the preset pictures in the target picture set, obtain the feature vector corresponding to each preset picture and send it to the sixth processing unit 1006. The sixth processing unit 1006 can receive the target image set, the first hash value, the feature vector and the storage address, and assemble them into Kafka message data. The first processing unit 1001 can be any unit connected to the processor inside the gateway or device. The second processing unit 1002 can transmit the acquired data to the processor through a wired or wireless connection with the processor of the third processing unit 1003. The processor of the third processing unit 1003 can process the data through the internal chip and finally obtain the retrieval result. Among them, the specific device connection mode and device setting of the second processing unit 1002 and the third processing unit 1003, the third processing unit 1003 and the fourth processing unit 1004, the fourth processing unit 1004 and the fifth processing unit 1005, and the fifth processing unit 1005 and the sixth processing unit 1006 are not limited.

[0122] Further, refer to Fig. 9In some feasible embodiments of the present application, the personnel profile test data generation system may further include a seventh processing unit 1007. The seventh processing unit 1007 may be used to push Kafka message data to the business side in response to the operation of the business side subscribing to the Kafka topic, so that the business side generates batch personnel profile data. Specifically, the first processing unit 1001 responds to the user's generation request, starts the service system and obtains the preset storage address of the personnel profile test data; the service system includes an AI face-changing service interface and a feature vector extraction interface. The second processing unit 1002 may call the AI ​​face-changing service interface through the chip inside its own processor to generate several personnel profile pictures. The second processing unit 1003 may receive several personnel profile pictures, determine the first hash value corresponding to each personnel profile picture and send it to the fourth processing unit 1004. The fourth processing unit 1004 may receive the first hash value, and determine the target picture set from the database according to the first hash value. The fifth processing unit 1005 may receive the target picture set and call the feature vector extraction interface to extract features from the target picture set, obtain the feature vector and send it to the sixth processing unit 1006. The sixth processing unit 1006 can receive the target picture set, the first hash value, the feature vector, and the storage address, and assemble them into Kafka message data. After assembling the Kafka message data, in response to the business side's operation of subscribing to the Kafka topic, the seventh processing unit 1007 can push the Kafka message data to the business side, so that the business side generates batch personnel file data. It can be understood that the specific device connection method and device setting of the sixth processing unit 1006 and the seventh processing unit 1006 are not limited.

[0123] Similarly, the personnel profile test data generation system of this embodiment can call the AI ​​face-changing service interface through the second processing unit 1002 to generate several personnel profile pictures. The second processing unit 1003 determines the first hash value corresponding to each personnel profile picture. The fourth processing unit 1004 determines the target picture set from the database according to the first hash value. The fifth processing unit 1005 calls the feature vector extraction interface to extract features from the target picture set to obtain a feature vector. The sixth processing unit 1006 assembles the target picture set, the first hash value, the feature vector and the storage address into Kafka message data. This embodiment stores key test data such as personnel profile pictures and feature vectors persistently and establishes corresponding data archives. When the system environment is switched, the relevant data only needs to be pushed to the new test environment through message queue technologies such as Kafka, and the same test scenario and data set as the original can be quickly constructed. This embodiment can reduce the cumbersome process of repeatedly generating test data, save a lot of time and resources, and improve the efficiency and flexibility of testing work.

[0124] and Figure 1Corresponding to the method, the embodiment of the present application also provides a personnel file test data generation device, the specific structure of which can be referred to Fig.10 ,include:

[0125] at least one processor 1011;

[0126] At least one memory 1012, used to store at least one program;

[0127] When the at least one program is executed by the at least one processor, the at least one processor implements the personnel profile test data generating method.

[0128] The contents of the above method embodiments are all applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0129] and Figure 1 Corresponding to the method, an embodiment of the present application further provides a computer-readable storage medium, which stores processor-executable instructions, and the processor-executable instructions are used to execute the personnel profile test data generation method when executed by the processor.

[0130] The contents of the above-mentioned personnel profile test data generation method embodiment are all applicable to the present storage medium embodiment. The functions specifically implemented by the present storage medium embodiment are the same as those of the above-mentioned personnel profile test data generation method embodiment, and the beneficial effects achieved are also the same as those achieved by the above-mentioned personnel profile test data generation method embodiment.

[0131] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the application is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are expected, wherein the order of various operations is changed and the sub-operation described as a part of a larger operation is performed independently.

[0132] In addition, although the present application is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present application. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional techniques of the engineer. Therefore, those skilled in the art can implement the present application set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present application, which is determined by the full scope of the attached claims and their equivalents.

[0133] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several programs to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0134] The logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable programs for implementing the logical functions, and may be embodied in any computer-readable medium for use by a program execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch and execute a program from a program execution system, device or apparatus), or in conjunction with such program execution systems, devices or apparatuses. For purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by a program execution system, device or apparatus, or in conjunction with such program execution systems, devices or apparatuses.

[0135] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0136] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable program execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0137] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0138] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

[0139] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the described embodiments. Technical personnel familiar with the field may make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A method for generating personnel file test data, characterized in that: The following steps are involved: In response to the user's generation request, the service system is started and the storage address preset for the personnel file test data is obtained; the service system includes an AI face-changing service interface and a feature vector extraction interface; Call the AI ​​face-changing service interface to generate several personnel profile pictures; Determine a first hash value corresponding to each of the personnel profile pictures; Determine a target picture set according to the first hash value and a pre-established database; the database includes a plurality of preset pictures, and the target picture set includes the personnel file picture and at least one preset picture; Calling the feature vector extraction interface to perform feature extraction on the preset pictures in the target picture set to obtain a feature vector corresponding to each of the preset pictures; The target picture set, the first hash value, the feature vector, and the storage address are assembled into Kafka message data.

2. A method for generating personnel file test data according to claim 1, characterized in that: The determining the target picture set according to the first hash value and a pre-established database specifically includes: Determine a second hash value corresponding to each of the preset pictures; When the first hash value corresponding to the personnel profile picture is the same as the second hash value corresponding to any one of the preset pictures, the personnel profile picture is determined as the profile picture to be processed; Recall the AI ​​face-changing service interface to generate a personnel profile picture that has a different second hash value from all the preset pictures; The newly generated personnel profile picture is used to replace the profile picture to be processed, and the personnel profile picture and the set of preset pictures in the database are used as the target picture set.

3. A method for generating personnel file test data according to claim 1, characterized in that: The step of assembling the target picture set, the first hash value, the feature vector, and the storage address into Kafka message data includes: Assemble the target picture set, the first hash value, the feature vector and the storage address and write them into a facial feature data table; According to a preset time interval, the updated picture information in the target picture set is read from the facial feature data table regularly, and the updated picture information is assembled into Kafka message data.

4. A method for generating personnel file test data according to claim 1, characterized in that: Determining the first hash value corresponding to each of the personnel profile pictures specifically includes: Fill each of the personnel profile pictures with picture data of a preset length; Dividing the picture data into a plurality of picture data blocks of fixed length; The first hash value is determined according to the picture data block.

5. A method for generating personnel file test data according to claim 4, characterized in that: The determining the first hash value according to the picture data block specifically includes: Configure the initial hash value; For any data block, the initial hash value and the current image data block are input into a compression function to obtain an intermediate hash value; The intermediate hash value is used as the initial hash value for the next data block to be processed, and the step of inputting the initial hash value and the current image data block into the compression function to obtain the intermediate hash value is returned until all the image data blocks are compressed to obtain the first hash value.

6. A method for generating personnel file test data according to claim 1, characterized in that: The AI ​​face-changing service interface is an initialized service interface, and the initialization process of the AI ​​face-changing service interface includes: The facial structure features in the AI ​​face-changing service interface are parameterized, and the facial structure features include facial clothing features, skin color features, and facial features.

7. A method for generating personnel file test data according to claim 1, characterized in that: The method further comprises: In response to the operation of the business side subscribing to the Kafka topic, the Kafka message data is pushed to the business side, so that the business side generates batch personnel file data.

8. A personnel file test data generation system, characterized in that: include: A first processing unit, configured to start a service system and obtain a preset storage address of personnel profile test data in response to a user's generation request; the service system includes an AI face-changing service interface and a feature vector extraction interface; The second processing unit is used to call the AI ​​face-changing service interface to generate a number of personnel file pictures; A third processing unit, configured to determine a first hash value corresponding to each of the personnel profile pictures; A fourth processing unit, configured to determine a target picture set according to the first hash value and a pre-established database; the database includes a plurality of preset pictures, and the target picture set includes the personnel file picture and at least one preset picture; A fifth processing unit, configured to call the feature vector extraction interface, perform feature extraction on the preset pictures in the target picture set, and obtain a feature vector corresponding to each of the preset pictures; The sixth processing unit is used to assemble the target picture set, the first hash value, the feature vector and the storage address into Kafka message data.

9. A device for generating personnel file test data, characterized in that include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a method for generating personnel profile test data as described in any one of claims 1-7.

10. A computer-readable storage medium storing instructions executable by a processor, characterized in that: The processor-executable instructions are used to execute a personnel profile test data generating method as described in any one of claims 1-7 when executed by the processor.

Images