Streaming data testing method and device, equipment, storage medium and product
By obtaining the initial data file and the dictionary to be replaced, the default key value pairs in the initial data file are replaced, the replaced target data file is generated, and stream data is tested, which solves the problem of inefficient stream data testing in the existing technology. Implement automated testing of streaming data, improving efficiency and accuracy.
Patent Information
- Application Number
- CN202411072457.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-11-08
AI Technical Summary
In the prior art, streaming data testing is inefficient, and reliance on manual configuration data leads to repeated operations, which is not very efficient.
By obtaining the initial data file and the dictionary to be replaced, the default key value pairs in the initial data file are replaced based on the target key value pairs, the replaced target data file is generated, and stream data is tested to achieve automated testing.
Improve the efficiency of streaming data testing and realize automated streaming data testing without manually performing repeated configuration operations, improving testing efficiency and accuracy.
Smart Images

Figure CN118916288A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a streaming data testing method, device, equipment, storage medium and product. Background Art
[0002] At present, with the development of computer technology, software system testing involves all walks of life. For example, the banking system developed by financial enterprises needs to be able to process and respond to data within a specified time. In order to ensure the performance and stability of the above systems under real-time conditions, system flow data testing is an indispensable link.
[0003] In the related art, the testing of quasi-real-time projects usually relies on manual configuration of data to implement system stream data testing. However, the manual data configuration method requires manual repetitive configuration operations, resulting in low efficiency of stream data testing.
[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention
[0005] The main purpose of this application is to provide a streaming data testing method, aiming to solve the technical problem of low efficiency of streaming data testing.
[0006] To achieve the above object, the present application proposes a flow data testing method, the flow data testing method comprising:
[0007] Obtain an initial data file and a dictionary to be replaced, wherein the initial data file is a file containing serialized data, the serialized data is a default key-value pair, and the dictionary to be replaced contains a target key-value pair to be replaced;
[0008] Based on the target key-value pair, a default key-value pair under the initial data file is replaced to obtain a replaced target data file;
[0009] A stream data test is performed on the replaced target data file to obtain a stream data test result.
[0010] Optionally, the default key-value pair includes a default key and a default key value, the target key-value pair includes a target key and a target key value, and the step of replacing the default key-value pair under the initial data file based on the target key-value pair to obtain the replaced target data file includes:
[0011] Traversing the default key-value pairs under the initial data file, and matching the default key with the target key;
[0012] If the match is successful, the default key value corresponding to the successfully matched default key is replaced with the target key value corresponding to the matched target key to obtain a replaced target key-value pair, and based on the replaced target key-value pair, a replaced target data file is formed.
[0013] Optionally, if the match is successful, then after the step of replacing the default key value corresponding to the successfully matched default key with the target key value corresponding to the matched target key, and forming a replaced target data file based on the replaced target key-value pair, the method comprises:
[0014] If the match fails, determine whether the default key value has a function identifier;
[0015] If the default key value has a function identifier, execute the target function corresponding to the function identifier on the default key value to obtain the generated key value;
[0016] The default key value is replaced by the generated key value to obtain a generated key-value pair, and a replaced target data file is formed based on the generated key-value pair and the replaced target key-value pair.
[0017] Optionally, if the default key value has a function identifier, after the step of executing the target function corresponding to the function identifier on the default key value to obtain the generated key value, the method includes:
[0018] If the default key fails to match the target key and the default key value does not have a function identifier, the default key value does not perform a replacement operation, and a replaced target data file is formed based on the default key-value pair, the generated key-value pair, and the replaced target key-value pair.
[0019] Optionally, the step of performing a stream data test on the replaced target data file to obtain a stream data test result includes:
[0020] Sending the replaced target data file to the data source topic of the target cluster;
[0021] Acquire the data to be tested from the data source topic of the target cluster;
[0022] The data to be tested is verified and tested to obtain a stream data test result.
[0023] Optionally, the step of replacing a default key-value pair under the initial data file based on the target key-value pair to obtain a replaced target data file includes:
[0024] Acquire subject category information of the initial data file;
[0025] Selecting a target key-value pair after category screening corresponding to the subject category information from the dictionary to be replaced, wherein the dictionary to be replaced includes key-value pairs of multiple subject categories;
[0026] Based on the target key-value pairs after the category screening, the default key-value pairs under the initial data file are replaced to obtain a replaced target data file, wherein the replaced target data file adopts a subject format corresponding to the subject category information.
[0027] In addition, to achieve the above-mentioned purpose, the present application also proposes a flow data testing device, the flow data testing device comprising:
[0028] An acquisition module is used to acquire an initial data file and a dictionary to be replaced, wherein the initial data file is a file containing serialized data, the serialized data is a default key-value pair, and the dictionary to be replaced contains a target key-value pair to be replaced;
[0029] A replacement module, used to replace the default key-value pair under the initial data file based on the target key-value pair to obtain a replaced target data file;
[0030] The test module is used to perform a stream data test on the replaced target data file to obtain a stream data test result.
[0031] In addition, to achieve the above-mentioned purpose, the present application also proposes a stream data testing device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the stream data testing method described above.
[0032] In addition, to achieve the above objectives, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the stream data testing method described above are implemented.
[0033] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the stream data testing method described above are implemented.
[0034] One or more technical solutions proposed in this application have at least the following technical effects:
[0035] Compared with the related art in the related art about the test of quasi-real-time projects, which usually relies on manual configuration data to implement system flow data testing. However, the manual configuration data method requires manual repeated configuration operations, resulting in low efficiency of flow data testing. In this application, the default key-value pairs under the initial data file are replaced by the target key-value pairs that need to be replaced in the dictionary to be replaced, and the replaced target data file is obtained, thereby realizing automatic generation of test data, and then performing flow data testing on the replaced target data file to obtain flow data test results, realizing automatic flow data testing, and no manual repeated configuration operations are required, thereby improving the efficiency of flow data testing. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0038] Figure 1 A flow chart of the first embodiment of the flow data testing method of the present application;
[0039] Figure 2 This is a schematic diagram of the automated testing framework structure of the flow data testing method of this application;
[0040] Figure 3 A flow chart of the second embodiment of the flow data testing method of the present application;
[0041] Figure 4 A flow chart of the third embodiment of the flow data testing method of the present application;
[0042] Figure 5 This is a schematic diagram of the module structure of the stream data testing device according to an embodiment of the present application;
[0043] Figure 6 Schematic diagram of the device structure of the hardware operating environment involved in the streaming data testing method in the embodiment of the present application.
[0044] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0045] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.
[0046] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.
[0047] The main solution of the embodiment of the present application is: obtaining an initial data file and a dictionary to be replaced, wherein the initial data file is a file containing serialized data, the serialized data is a default key-value pair, and the dictionary to be replaced contains a target key-value pair that needs to be replaced; based on the target key-value pair, replacing the default key-value pair under the initial data file to obtain a replaced target data file; performing a stream data test on the replaced target data file to obtain a stream data test result.
[0048] In this embodiment, the stream data test device is used as the execution subject. For the convenience of description, it is hereinafter referred to as "device" for explanation.
[0049] In the related art, the test of quasi-real-time projects usually relies on manual configuration of data to implement system stream data testing. However, the manual data configuration method requires manual repetitive configuration operations, resulting in low efficiency of stream data testing.
[0050] The present application provides a solution to realize automated streaming data testing, thereby improving streaming data testing efficiency.
[0051] It can be seen from the above embodiments that the present application replaces the default key-value pairs under the initial data file by including the target key-value pairs that need to be replaced in the dictionary to be replaced, and obtains the replaced target data file, thereby realizing automatic generation of test data, and then performing stream data testing on the replaced target data file to obtain stream data test results, thereby realizing automatic stream data testing, and eliminating the need for manual repetitive configuration operations, thereby improving the efficiency of stream data testing.
[0052] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a terminal system, etc. The following takes the streaming data test device as an example to illustrate this embodiment and the following embodiments.
[0053] Based on this, the present application embodiment provides a flow data testing method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the stream data testing method of the present application.
[0054] In this embodiment, the flow data testing method includes steps S100 to S300:
[0055] Step S100, obtaining an initial data file and a dictionary to be replaced, wherein the initial data file is a file containing serialized data, the serialized data is a default key-value pair, and the dictionary to be replaced contains a target key-value pair to be replaced;
[0056] It should be noted that the initial data file is a file containing serialized data, and the serialized data is a default key-value pair, wherein the default key-value pair includes a default key name (field name) and a default key value. The serialized data specifically refers to stream data, and the stream data (Stream Data) refers to a continuously generated data sequence, usually composed of multiple data points, which arrive in chronological order and are usually infinite or very large.
[0057] It can be understood that the dictionary to be replaced is a data dictionary containing target key-value pairs that need to be replaced, wherein the target key-value pairs that need to be replaced are key-value pairs set by the user and / or tester. The data dictionary is a collection used to define data structures and attributes in a database or data warehouse. The data dictionary contains information such as field names, data types, lengths, constraints, etc.
[0058] In a specific implementation, the initial data file is specifically a yaml file, and the stream data of the present application is preferably Kafka stream data, wherein Kafka is a distributed stream processing platform that can process real-time data streams with high throughput. In Kafka, stream data generally refers to a continuous sequence of data that is generated and consumed in the form of a time series. Stream data can be any form of data, such as log data, events, sensor data, etc., which are usually sent to the Kafka cluster in the form of messages. Kafka uses Topics to organize messages, producers send messages to specific topics, and consumers read messages from topics. Each topic can have multiple partitions to support parallel processing and high availability of data.
[0059] Further, refer to Figure 2, the automated testing framework of this application is divided into 5 key modules in the code architecture: test case layer, task management layer, general service layer, data storage layer, and Kafka service layer. The core test data generation method is embedded in the Kafka service layer, which involves building data on multiple topics, as well as methods for managing and consuming these data. The test case layer relies on the pytest framework to structure the test collection, and directly connects to the comparison function of the task management layer to achieve test verification and result comparison for multiple scenarios; the task management layer calls the public functions provided by the general layer to adapt to the special needs of different businesses and perform customized data processing tasks; the general layer includes system common configuration, general handlers, exception handlers, etc., to provide services for different tasks; the data storage layer is used to integrate online library queries, Redis caches, and Hbase cache query methods.
[0060] Step S200, based on the target key-value pair, replacing the default key-value pair under the initial data file to obtain a replaced target data file;
[0061] In a specific implementation, the device replaces the default key-value pair under the initial data file based on the target key-value pair to obtain the replaced target data file, thereby realizing the number generation of quasi-real-time projects, that is, the code value to be replaced is configured into the yaml file once and reused, thereby improving the accuracy and efficiency of number generation.
[0062] For example, the default key-value pair is (X1, Y1), and the target key-value pair is (X1, Y2). This application requires replacing the default key-value pair with the target key-value pair to generate test data that meets user testing requirements.
[0063] In a specific implementation, the device replaces the default key-value pair under the initial data file based on the target key-value pair to obtain the replaced target data file, including:
[0064] The default key-value pairs under the initial data file are traversed to match the default key with the target key; if the match is successful, the default key value corresponding to the successfully matched default key is replaced with the target key value corresponding to the matched target key to obtain a replaced target key-value pair, and a replaced target data file is formed based on the replaced target key-value pair.
[0065] In a specific implementation, the device traverses the default key-value pairs under the initial data file and matches the default key with the target key. If the match is successful, that is, the default key (field name) in the default key-value pair and the target key (field name) in the target key-value pair are the same, the device replaces the default key value corresponding to the successfully matched default key with the target key value corresponding to the matched target key to obtain a replaced target key-value pair, and forms a replaced target data file based on the replaced target key-value pair.
[0066] For example, the default key-value pair is (X1, Y1), and the target key-value pair is (X1, Y2), where the default key X1 and the target key X1 are the same, that is, the match is successful, and the target key-value pair after replacement is (X1, Y2).
[0067] In a specific implementation, if the device matches successfully, the default key value corresponding to the successfully matched default key is replaced with the target key value corresponding to the matched target key, and after the step of forming a replaced target data file based on the replaced target key value pair, the method includes:
[0068] If the match fails, determine whether the default key value has a function identifier; if the default key value has a function identifier, execute the target function corresponding to the function identifier on the default key value to obtain a generated key value; replace the default key value with the generated key value to obtain a generated key-value pair, and form a replaced target data file based on the generated key-value pair and the replaced target key-value pair.
[0069] In a specific implementation, the function identifier is the identifier of the function containing the custom number-generating method in the default key value, that is, the target function is the function containing the custom number-generating method. If the function identifier exists, the device first needs to execute the corresponding statement to introduce the external package, and then execute the target function corresponding to the function identifier on the default key value to obtain the key value after the number-generating, and based on the key-value pair after the number-generating and the target key-value pair after the replacement, form a replaced target data file, so as to realize the one-time configuration of the custom number-generating method and the code value of the special field into the yaml file and reuse it, thereby improving the accuracy and efficiency of number-generating.
[0070] In a specific implementation, if the default key value has a function identifier, after the device executes the target function corresponding to the function identifier on the default key value to obtain the generated key value, the method includes:
[0071] If the default key fails to match the target key and the default key value does not have a function identifier, the default key value does not perform a replacement operation, and a replaced target data file is formed based on the default key-value pair, the generated key-value pair, and the replaced target key-value pair.
[0072] In a specific implementation, if the default key fails to match the target key and the default key value does not have a function identifier, the default key value is used to perform a stream data test, that is, the default key value does not perform a replacement operation, and the device combines the above default key value pair, the generated key value pair, and the replaced target key value pair to form a replaced target data file.
[0073] Step S300, performing a stream data test on the replaced target data file to obtain a stream data test result.
[0074] In a specific implementation, after the device realizes the generation of test data (i.e., obtains the replaced target data file), it performs a stream data test on the replaced target data file to obtain a stream data test result, specifically using Figure 2 The testing framework realizes the full process automated testing of data generation, delivery, and acceptance of quasi-real-time projects.
[0075] In a specific implementation, the device performs a stream data test on the replaced target data file to obtain a stream data test result, including:
[0076] The replaced target data file is sent to the data source topic of the target cluster; the data to be tested is obtained from the data source topic of the target cluster; and the data to be tested is verified and tested to obtain a stream data test result.
[0077] In a specific implementation, the device sends the replaced target data file to the data source topic of the target cluster, and obtains the data to be tested from the data source topic of the target cluster. Specifically, the device sends the replaced target data file to the source Topic of the Kafka cluster, which is equivalent to the device producing the replaced target data file to the source Topic of the Kafka cluster. Furthermore, the device consumes the result topic of the Kafka cluster to obtain the data to be tested, and finally verifies the data to be tested to obtain the streaming data test result.
[0078] In the specific implementation, this application provides an automated testing framework that realizes the full-process automated testing of data generation, sending, and acceptance of quasi-real-time projects. This method can generate data while sending it, which can simulate large quantities of data, thereby fully simulating the actual dynamic data flow and improving the accuracy of the test.
[0079] Compared with the related art in the related art about the test of quasi-real-time projects, which usually relies on manual configuration data to implement system flow data testing. However, the manual configuration data method requires manual repeated configuration operations, resulting in low efficiency of flow data testing. In this application, the default key-value pairs under the initial data file are replaced by the target key-value pairs that need to be replaced in the dictionary to be replaced, and the replaced target data file is obtained, thereby realizing automatic generation of test data, and then performing flow data testing on the replaced target data file to obtain flow data test results, realizing automatic flow data testing, and no manual repeated configuration operations are required, thereby improving the efficiency of flow data testing.
[0080] Based on the above first embodiment, the present application also proposes another embodiment, referring to Figure 3 , the flow data testing method comprises:
[0081] Step 1: The device reads the incoming YAML file content and parses it into a Python object;
[0082] Step 2: The device determines whether "IMPORT" is configured in the yaml file. If so, it parses the content after "IMPORT" and executes the corresponding statement to import the external package;
[0083] It is understandable that "IMPORT" is an identifier written at the top of the yml file, which means that the program needs to import the function package after import.
[0084] Step 3: Traverse each key (field name) value pair in the yaml file. If the key exists in the input parameter dictionary to be replaced (that is, the default key matches the target key successfully), directly take the input parameter value as the new key value; if the key value starts with a function identifier, identify the content after the function identifier as the function name, execute the function and use the execution result as the new key value; if the above situation is not met, take the value in the yaml file as the default key value;
[0085] Step 4: According to step 3, replace all fields in the yaml file in turn and return the replaced result. The returned result is the number creation result.
[0086] Step 5: Perform a stream data test on the number-generating result to obtain a stream data test result.
[0087] Based on the above first and second embodiments, the present application also proposes another embodiment, referring to Figure 4 , the flow data testing method comprises:
[0088] In a specific implementation, the device replaces the default key-value pair under the initial data file based on the target key-value pair to obtain the replaced target data file, and further includes:
[0089] Step A100, obtaining subject category information of the initial data file;
[0090] In a specific implementation, the subject category information refers to different topic format information, and it is understandable that when the device constructs Kakfa stream data, it will involve a variety of topic themes, each of which may have a unique data structure and rich fields. If the test data is manually maintained or constructed by traditional field value replacement, the efficiency is low. Therefore, the purpose of this embodiment is to provide a test data generation method to solve the technical problem of adapting to different topic formats.
[0091] Step A200, selecting a target key-value pair after category screening corresponding to the subject category information from the dictionary to be replaced, wherein the dictionary to be replaced includes key-value pairs of multiple subject categories;
[0092] In a specific implementation, the device selects a target key-value pair after category screening of the subject category information from the dictionary to be replaced, wherein the dictionary to be replaced includes key-value pairs of multiple subject categories. Specifically, the dictionary to be replaced includes key-value pairs of each subject category, that is, each file of the dictionary to be replaced corresponds to a subject to adapt to different subjects, thereby realizing the generation of test data with subjects in different formats, and after a one-time configuration, the configuration can be reused in the streaming data testing process.
[0093] Step A300, based on the target key-value pairs after the category screening, replace the default key-value pairs under the initial data file to obtain a replaced target data file, wherein the replaced target data file adopts the subject format corresponding to the subject category information.
[0094] In a specific implementation, the device replaces the default key-value pairs under the initial data file based on the target key-value pairs after the category screening, and obtains a replaced target data file, wherein the replaced target data file adopts the topic format corresponding to the topic category information, that is, the present application provides a test data generation method that can adapt to different Topic topic formats, thereby not only improving the efficiency and accuracy of the test, but also reducing manual intervention and improving the repeatability and quality of the test.
[0095] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the flow data testing method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.
[0096] This application also provides a streaming data testing device, please refer to Figure 5 , the streaming data testing device comprises:
[0097] The acquisition module 10 is used to acquire an initial data file and a dictionary to be replaced, wherein the initial data file is a file containing serialized data, the serialized data is a default key-value pair, and the dictionary to be replaced contains a target key-value pair to be replaced;
[0098] A replacement module 20, configured to replace the default key-value pair under the initial data file based on the target key-value pair to obtain a replaced target data file;
[0099] The testing module 30 is used to perform a stream data test on the replaced target data file to obtain a stream data test result.
[0100] Optionally, the replacement module 20 includes:
[0101] A matching module, used for traversing the default key-value pairs under the initial data file, and matching the default key with the target key;
[0102] The key-value replacement module replaces the default key value corresponding to the successfully matched default key with the target key value corresponding to the matched target key if the match is successful, obtains the replaced target key-value pair, and forms the replaced target data file based on the replaced target key-value pair.
[0103] Optionally, the flow data testing device further includes:
[0104] A judgment module, used for judging whether the default key value has a function identifier if the match fails;
[0105] A number generation module, configured to execute a target function corresponding to the function identifier on the default key value if the default key value has a function identifier, to obtain a generated key value;
[0106] The default key value is replaced by the generated key value to obtain a generated key-value pair, and a replaced target data file is formed based on the generated key-value pair and the replaced target key-value pair.
[0107] Optionally, the flow data testing device further includes:
[0108] A formation module is used for not performing a replacement operation on the default key value if the default key fails to match the target key and the default key value does not have a function identifier, and forming a replaced target data file based on the default key value pair, the created key value pair and the replaced target key value pair.
[0109] Optionally, the testing module 30 includes:
[0110] A sending module, used for sending the replaced target data file to a data source topic of a target cluster;
[0111] A test data acquisition module, used to acquire the data to be tested from the data source topic of the target cluster;
[0112] The verification test module is used to verify the data to be tested and obtain the flow data test result.
[0113] Optionally, the replacement module 20 includes:
[0114] A subject category acquisition module, used to acquire subject category information of the initial data file;
[0115] A screening module, used for selecting target key-value pairs corresponding to the category screening of the subject category information from the dictionary to be replaced, wherein the dictionary to be replaced includes key-value pairs of multiple subject categories;
[0116] The file replacement module is used to replace the default key-value pairs under the initial data file based on the target key-value pairs after the category screening to obtain a replaced target data file, wherein the replaced target data file adopts the theme format corresponding to the theme category information.
[0117] The stream data testing device provided by the present application adopts the stream data testing method in the above embodiment, which can solve the technical problems of stream data testing. Compared with the prior art, the beneficial effects of the stream data testing device provided by the present application are the same as the beneficial effects of the stream data testing method provided by the above embodiment, and other technical features in the stream data testing device are the same as the features disclosed in the above embodiment method, which will not be repeated here.
[0118] The present application provides a stream data testing device, which includes: at least one processor; and a memory that is communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the stream data testing method in the above-mentioned embodiment 1.
[0119] Reference below Figure 6, which shows a schematic diagram of the structure of a stream data test device suitable for implementing the embodiment of the present application. The stream data test device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The streaming data testing device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0120] like Figure 6 As shown, the stream data test device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the stream data test device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the streaming data test device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a streaming data test device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.
[0121] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.
[0122] The stream data test device provided by the present application adopts the stream data test method in the above embodiment to solve the technical problem of stream data testing. Compared with the prior art, the beneficial effects of the stream data test device provided by the present application are the same as the beneficial effects of the stream data test method provided by the above embodiment, and other technical features in the stream data test device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.
[0123] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.
[0124] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0125] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the stream data testing method in the above-mentioned embodiment.
[0126] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.
[0127] The computer-readable storage medium may be included in the stream data test device; or may exist independently without being assembled into the stream data test device.
[0128] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the stream data testing device, the stream data testing device performs stream data testing.
[0129] Computer program code for performing the operations of the present application may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0130] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0131] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.
[0132] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned stream data testing method, and can solve the technical problems of stream data testing. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the stream data testing method provided in the above-mentioned embodiment, and will not be repeated here.
[0133] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned stream data testing method when executed by a processor.
[0134] The computer program product provided by the present application can solve the technical problem of stream data testing. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the stream data testing method provided by the above embodiment, which will not be described in detail here.
[0135] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.
Claims
1. A streaming data testing method, It is characterized in that The method for the flow data test comprises: Obtain an initial data file and a dictionary to be replaced, wherein the initial data file is a file containing serialized data, the serialized data is a default key-value pair, and the dictionary to be replaced contains a target key-value pair to be replaced; Based on the target key-value pair, a default key-value pair under the initial data file is replaced to obtain a replaced target data file; A stream data test is performed on the replaced target data file to obtain a stream data test result.
2. The streaming data testing method according to claim 1, It is characterized in that The default key-value pair includes a default key and a default key value, the target key-value pair includes a target key and a target key value, and the step of replacing the default key-value pair under the initial data file based on the target key-value pair to obtain the replaced target data file includes: Traversing the default key-value pairs under the initial data file, and matching the default key with the target key; If the match is successful, the default key value corresponding to the successfully matched default key is replaced with the target key value corresponding to the matched target key to obtain a replaced target key-value pair, and based on the replaced target key-value pair, a replaced target data file is formed.
3. The streaming data testing method according to claim 2, It is characterized in that If the match is successful, then the default key value corresponding to the successfully matched default key is replaced with the target key value corresponding to the matched target key, and after the step of forming a replaced target data file based on the replaced target key-value pair, the method includes: If the match fails, determine whether the default key value has a function identifier; If the default key value has a function identifier, execute the target function corresponding to the function identifier on the default key value to obtain the generated key value; The default key value is replaced by the generated key value to obtain a generated key-value pair, and a replaced target data file is formed based on the generated key-value pair and the replaced target key-value pair.
4. The streaming data testing method according to claim 3, It is characterized in that After the step of executing the target function corresponding to the function identifier on the default key value to obtain the generated key value if the default key value has a function identifier, the method includes: If the default key fails to match the target key and the default key value does not have a function identifier, the default key value does not perform a replacement operation, and a replaced target data file is formed based on the default key-value pair, the generated key-value pair, and the replaced target key-value pair.
5. The streaming data testing method according to claim 1, It is characterized in that The step of performing a stream data test on the replaced target data file to obtain a stream data test result comprises: Sending the replaced target data file to the data source topic of the target cluster; Acquire the data to be tested from the data source topic of the target cluster; The data to be tested is verified and tested to obtain a stream data test result.
6. The streaming data testing method according to claim 1, It is characterized in that The step of replacing the default key-value pair under the initial data file based on the target key-value pair to obtain the replaced target data file includes: Acquire subject category information of the initial data file; Selecting a target key-value pair after category screening corresponding to the subject category information from the dictionary to be replaced, wherein the dictionary to be replaced includes key-value pairs of multiple subject categories; Based on the target key-value pairs after the category screening, the default key-value pairs under the initial data file are replaced to obtain a replaced target data file, wherein the replaced target data file adopts a subject format corresponding to the subject category information.
7. A streaming data testing device, It is characterized in that The device comprises: An acquisition module is used to acquire an initial data file and a dictionary to be replaced, wherein the initial data file is a file containing serialized data, the serialized data is a default key-value pair, and the dictionary to be replaced contains a target key-value pair to be replaced; A replacement module, used to replace the default key-value pair under the initial data file based on the target key-value pair to obtain a replaced target data file; The test module is used to perform a stream data test on the replaced target data file to obtain a stream data test result.
8. A streaming data testing device, It is characterized in that The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the streaming data testing method according to any one of claims 1 to 6.
9. A storage medium, It is characterized in that The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the flow data testing method according to any one of claims 1 to 6 are implemented.
10. A computer program product, It is characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the streaming data testing method according to any one of claims 1 to 6 are implemented.
