Data processing method, system, apparatus, and storage medium

Abstract

This application discloses a data processing method, system, apparatus, and storage medium. Relating to the field of Internet technology, the method includes: upon receiving a target instruction, determining the target script corresponding to the target business type indicated in the target instruction from multiple preset scripts; using the target script, extracting the request content of the target business request from the system log of the source system based on string recognition, wherein the system log records the request content of multiple business types received by the source system, and the request content of the target business request is used to perform business testing on the target system, where the target system is the system to which the business migration from the source system is directed. This application solves the problem of high labor costs associated with related technologies in business migration scenarios between old and new systems, where production test cases are manually extracted from the old system for testing the new system based on these production test cases.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet, in particular to a data processing method, system, device and storage medium. BACKGROUND

[0002] With the continuous development of technology, systems in various fields are gradually shifting from localized services to cloud systems. During the transition process, the cloud system needs to be tested, and a large number of production cases are needed for comparison in the testing process to comprehensively test multiple business scenarios corresponding to each functional module, so as to ensure that the cloud system is consistent with the localized service in function and return result.

[0003] At present, in the related technology, the requests (i.e. production cases) under different functional scenarios are recorded in the same production log, and these types are mixed together, and the number of files and the occupied space of each file are very large. When manually extracting production cases from them, if you want to manually filter out each type of request and store it in a specified folder, it is time-consuming and laborious, thereby there is a problem of high labor cost.

[0004] For the above problems in the related technology, no effective solution has been proposed so far. SUMMARY

[0005] The main purpose of the present application is to provide a data processing method, system, device and storage medium, so as to solve the problem that in the related technology, production cases are manually extracted from the old system in the new and old system business migration scenario, and the new system is tested based on the production cases, thereby the problem of high labor cost exists.

[0006] In order to achieve the above purpose, according to one aspect of the present application, a data processing method is provided. The method comprises: in the case of receiving a target instruction, determining a target script corresponding to a target business type indicated in the target instruction from a plurality of preset scripts, wherein the target instruction is used to indicate the request content of the target business request, and the target business request refers to the business request corresponding to the target business type; using the target script, the request content of the target business request is extracted from the system log of the source system in a string recognition manner, wherein the system log is used to record the request content of the business request of a plurality of business types received by the source system, and the request content of the target business request is used for business testing of the target system, and the target system is the system pointed by the business migration of the source system.

[0007] Optionally, the data processing method further includes: using the target script to identify multiple business requests from the system logs; for each business request, determining whether the request content of the business request includes a type identifier string corresponding to the target business type; if the request content of the business request includes a type identifier string, determining the request content of the business request as the request content of the target business request.

[0008] Optionally, the data processing method also includes: identifying information headers from system logs, wherein the information headers are used to segment the request content of different business requests; and identifying multiple business requests from the system logs based on the information headers.

[0009] Optionally, the data processing method further includes: determining whether the request content of the business request includes the key field name corresponding to the target business type; if the request content of the business request includes the key field name corresponding to the target business type, determining the request content of the business request as the request content of the target business request.

[0010] Optionally, the data processing method further includes: after extracting the request content of the target business request from the system log of the source system based on string recognition, generating a target file based on the request content of the target business request, and determining the file name of the target file based on the request content; and storing the target file in the storage path corresponding to the target business type.

[0011] Optionally, the data processing method further includes: extracting target parameters from the target instruction before determining the target script corresponding to the target business type indicated in the target instruction from multiple preset scripts; determining the business type that matches the target parameters from the correspondence between multiple parameters and multiple business types based on the target parameters; and determining the business type that matches the target parameters as the target business type.

[0012] Optionally, the data processing method further includes: before determining the target script corresponding to the target business type indicated in the target instruction from multiple preset scripts, obtaining a preset script template; for each of the multiple business types, importing the parameter set corresponding to the business type into the script template to obtain the preset script corresponding to that business type.

[0013] To achieve the above objectives, according to another aspect of this application, a data processing system is provided. The system includes: a source system for receiving business requests of multiple business types and performing business processing on the requests; a target processing system for executing the aforementioned data processing method; and a target system, which is the system to which the business migration from the source system is directed.

[0014] To achieve the above objectives, according to another aspect of this application, a data processing apparatus is provided. The apparatus includes: a first determining module, configured to, upon receiving a target instruction, determine from a plurality of preset scripts a target script corresponding to a target service type indicated in the target instruction, wherein the target instruction is used to instruct the acquisition of request content for a target service request, and the target service request refers to a service request corresponding to the target service type; and an extraction module, configured to, using the target script, extract the request content of the target service request from the system log of a source system based on string recognition, wherein the system log records the request content of multiple service types received by the source system, and the request content of the target service request is used to perform service testing on the target system, the target system being the system to which the service migration from the source system is directed.

[0015] Optionally, the extraction module further includes: an identification submodule, used to identify multiple business requests from system logs using the target script; a judgment submodule, used to determine whether the request content of each business request includes a type identifier string corresponding to the target business type; and a first determination submodule, used to determine the request content of the business request as the request content of the target business request if the request content of the business request includes a type identifier string.

[0016] Optionally, the identification submodule further includes: a first identification unit, used to identify the information header from the system log, wherein the information header is used to segment the request content of different business requests; and a second identification unit, used to identify multiple business requests from the system log based on the information header.

[0017] Optionally, the first determining submodule further includes: a judging unit, used to judge whether the request content of the business request includes the key field name corresponding to the target business type; and a determining unit, used to determine the request content of the business request as the request content of the target business request if the request content of the business request includes the key field name corresponding to the target business type.

[0018] Optionally, the data processing device further includes: a second determining module, used to generate a target file based on the request content of the target business request, and determine the file name of the target file based on the request content; and a storage module, used to store the target file in the storage path corresponding to the target business type.

[0019] Optionally, the data processing device further includes: an extraction module for extracting target parameters from the target instruction; a second determination module for determining the business type matching the target parameters from the correspondence between multiple parameters and multiple business types based on the target parameters; and a third determination module for determining the business type matching the target parameters as the target business type.

[0020] Optionally, the data processing device further includes: an acquisition module for acquiring a preset script template; and an import module for importing the parameter set corresponding to each of the multiple business types into the script template to obtain the preset script corresponding to that business type.

[0021] To achieve the above objectives, according to another aspect of this application, a computer-readable storage medium is provided, which includes a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the above-described data processing method.

[0022] To achieve the above objectives, according to another aspect of this application, an electronic device is provided, the electronic device including a memory storing an executable program; and a processor for running the program, wherein the program executes the above-described data processing method during runtime.

[0023] To achieve the above objectives, according to another aspect of this application, a computer program product is provided, including computer instructions that, when executed by a processor, implement the steps of the data processing method described above.

[0024] In this embodiment, by determining the target script corresponding to the target business type indicated in the target instruction from multiple preset scripts, the target script for extracting specific production use cases is dynamically determined according to actual needs. By using the target script, the request content of the target business request is extracted from the system log of the source system based on string recognition. This realizes the automatic acquisition of production use cases using the target script, avoiding manual extraction of production use cases, thereby effectively reducing labor costs.

[0025] Therefore, the method provided in this application achieves the goal of automatically extracting production test cases using scripts, realizes the technical effect of reducing labor costs, and solves the technical problem of high labor costs in the scenario of business migration between old and new systems, where production test cases are extracted manually from the old system to test the new system. Attached Figure Description

[0026] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0027] Figure 1 This is a hardware structure block diagram of a computer terminal provided according to an embodiment of this application;

[0028] Figure 2 This is a flowchart of a data processing method provided according to an embodiment of this application;

[0029] Figure 3 This is a schematic diagram of a data processing method provided according to an embodiment of this application;

[0030] Figure 4 This is a schematic diagram of a data processing apparatus provided according to an embodiment of this application;

[0031] Figure 5 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0033] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0034] It should be noted that the information collected in this application (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data all comply with relevant laws, regulations, and standards, and necessary measures have been taken to ensure compliance with public order and good morals. Corresponding access points are provided for users to choose whether to authorize or refuse. For example, interfaces are established between this system and relevant users or organizations, providing users with corresponding access points to choose whether to agree to or refuse automated decision-making results; if the user chooses to refuse, the process proceeds to the expert decision-making stage.

[0035] Example 1

[0036] According to an embodiment of this application, an embodiment of a data processing method is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0037] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a data processing method is shown. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor (MCU) or a field-programmable gate array (FPGA), etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output (I / O) interface, a Universal Serial Bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0038] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0039] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the data processing method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the aforementioned data processing method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0040] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0041] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0042] Under the aforementioned operating environment, this application provides the following: Figure 2 The data processing method shown. Figure 2 This is a flowchart of the data processing method according to Embodiment 1 of this application.

[0043] Step S201: Upon receiving a target instruction, determine the target script corresponding to the target service type indicated in the target instruction from multiple preset scripts. The target instruction is used to indicate the request content of the target service request, and the target service request refers to the service request corresponding to the target service type.

[0044] Optionally, electronic devices, application systems, servers, and other devices can be used as the execution subject of this application. In this embodiment, the target processing system is used as the execution subject to execute the above-described data processing method.

[0045] In an optional embodiment, the target instruction is used to indicate the request content for obtaining the target service request. The target instruction can be an instruction entered by the user in the command line, and the target instruction can include parameter strings (i.e., target parameters), such as -fd, -sfn, -price, etc. These parameters are used to represent different service types, and the service type corresponding to the parameter string in the target instruction is the target service type.

[0046] In an optional embodiment, the target business request is an original request triggered by a real user or system in a production environment that belongs to a specific business type (i.e., the target business type), such as a complete fare inquiry message. The target business request is used to request the business system (e.g., the source system) to perform corresponding business processing. The target business request can also be understood as a production use case.

[0047] Depending on the specific application scenario, the types of business (including the target business type) involved in the processing of this embodiment may vary. For example, in the civil aviation field, the various business types involved may include: fare inquiry, rule inquiry, dynamic transfer, fare calculation, ticket refund, and rescheduling. The target business type may be any of the aforementioned business types. In this embodiment, the content of the business types (including the target business type) is not specifically limited, but is only provided as an example.

[0048] Optionally, different preset scripts among the aforementioned preset scripts correspond to different business types. After receiving the target instruction, the target processing system can identify the preset script that matches the target business type as the target script. For example, assuming the parameter "-fd" corresponds to the business type "fare inquiry," when the target parameter in the target instruction is -fd, the target script is the script responsible for processing the production use case extraction for "fare inquiry." In an optional embodiment, the target script can be in the form of a function block.

[0049] Step S202: Using the target script, extract the request content of the target business request from the system log of the source system based on string recognition. The system log is used to record the request content of multiple business types of business requests received by the source system. The request content of the target business request is used to perform business testing on the target system. The target system is the system to which the business migration of the source system is directed.

[0050] Optionally, after the target script is determined, the target processing system can execute it. During the execution of the target script, it determines whether a log entry in the system log belongs to the target business type by precisely matching a specific string (i.e., a type identifier string). For example, when the target business type is "fare inquiry," the target processing system can search the system log entries for the string "ENGINE REQUEST:1POST / fd." If a log entry contains this string, it is determined that the log entry is the request content of the target business request and is extracted from it.

[0051] It should be noted that by using scripts to automatically extract production test cases, the need for manual extraction is avoided, thereby effectively reducing labor costs and improving the efficiency of production test case extraction.

[0052] In an optional embodiment, the source system and the target system are the old and new systems involved in the business migration scenario. The source system is the old system, for example, the system currently being migrated and originally deployed locally. Its runtime logs (i.e., system logs) record a massive amount of real requests (i.e., business requests), serving as the source of production test cases. The target system is the new system, for example, a cloud-based system under a new architecture that is being developed or deployed. Its goal is to maintain complete consistency with the source system in terms of functionality and results; therefore, it needs to use the same inputs (i.e., production test cases) for testing. For example, after extracting the production test cases (i.e., the request content of the target business request), the production test cases are sent as test requests to the target system, and the request processing results from the target system in response to the test requests are obtained. The request processing results from the target system are compared with the request processing results from the source system in response to the production test cases, thereby determining whether the target system has any anomalies based on the comparison results, thus achieving reliability testing.

[0053] In this embodiment, by determining the target script corresponding to the target business type indicated in the target instruction from multiple preset scripts, the target script for extracting specific production use cases is dynamically determined according to actual needs. By using the target script, the request content of the target business request is extracted from the system log of the source system based on string recognition. This realizes the automatic acquisition of production use cases using the target script, avoiding manual extraction of production use cases, thereby effectively reducing labor costs.

[0054] Therefore, the method provided in this application achieves the goal of automatically extracting production test cases using scripts, realizes the technical effect of reducing labor costs, and solves the technical problem of high labor costs in the scenario of business migration between old and new systems, where production test cases are extracted manually from the old system to test the new system.

[0055] Optionally, in the data processing method provided in this application embodiment, the target script is used to extract the request content of the target business request from the system log of the source system based on string recognition. This includes: using the target script to identify multiple business requests from the system log; for each business request, determining whether the request content of the business request includes a type identifier string corresponding to the target business type; and if the request content of the business request includes a type identifier string, determining the request content of the business request as the request content of the target business request.

[0056] Optionally, the target script can use a specific header as a separator for log entries in the system log, splitting the entire system log into individual log segments (i.e., log entries). Each segment is a complete record of a business request, which may include the request time, request method, request path, request headers, and JSON (JavaScript Object Notation) data as the request body.

[0057] After identifying multiple business requests, for each business request, it is determined whether the request content includes a type identifier string corresponding to the target business type. For example, different business types are matched with different type identifier strings. The target script records the type identifier strings corresponding to the target business type, and the target script performs the aforementioned determination based on the type identifier strings recorded within it.

[0058] For example, after the target script identifies the complete log fragment for each business request, it extracts the request header from the log fragment, such as the content from the beginning of the log fragment to the first JSON's left parenthesis "{". Next, the target script performs an exact string match between this request header and the type identifier string corresponding to the target business type. For example, when the target business type is "fare inquiry", the script will determine whether the request header of the current log fragment completely contains the string "ENGINE REQUEST:1POST / fd".

[0059] Optionally, if the request content of the business request includes a type identifier string, then the request content of the business request is determined to be the request content of the target business request. If the request content of the business request does not include a type identifier string, then the request content of the business request is determined not to be the request content of the target business request.

[0060] In an optional embodiment, the correspondence between business types and type identifier strings can be as follows:

[0061] Table 1

[0062]

[0063] It should be noted that identifying whether a business request is the target business request based on the type identifier string improves the accuracy and efficiency of determining the target business request.

[0064] Optionally, in the data processing method provided in this application embodiment, multiple business requests are identified from the system log using a target script, including: identifying information headers from the system log, wherein the information headers are used to segment the request content of different business requests; and identifying multiple business requests from the system log based on the information headers.

[0065] Optionally, the header can be understood as a fixed delimiter string used to separate each independent business request. For example, the header can be represented as "[#%& The string "^]" is not part of the log content, but rather a structure identifier automatically inserted by the system before each log entry is written. Its function is to divide the continuous, unstructured log text into independent, complete, and identifiable log blocks. In other words, the header is the information header stored for each log entry, and the subsequent information represents the complete information of different written content (i.e., business requests).

[0066] Optionally, during the system log scan, the target script will actively search for the fixed string [#%&^]. This string appears repeatedly in the log, and each appearance represents the end of the previous business request and the beginning of the next business request. The program records the location of each [#%&^] found as a boundary marker for the log block.

[0067] After successfully identifying the header positions, the target script can use these positions as dividing points to divide the entire log content into several consecutive substrings. Each substring starts at the end of one header and ends before the beginning of the next header (the last segment ends at the end of the file). Each such substring constitutes a complete business request. For example, the content of the first business request is: "[#%&^]2024-02-1610:18:10.037[INFO] ENGINE REQUEST:1POST / fd { "fareDisplay": { ...}}", the target script will remove the [#%&^] at the beginning. The `^` symbol preserves the subsequent content as a complete record of the business request. The target script can store these segmented fragments one by one into an in-memory queue for subsequent steps to process.

[0068] It should be noted that by identifying business requests in the system logs based on fixed information headers, the accuracy and completeness of the identified business requests are improved.

[0069] Optionally, in the data processing method provided in this application embodiment, determining the request content of the service request as the request content of the target service request includes: determining whether the request content of the service request includes a key field name corresponding to the target service type; if the request content of the service request includes a key field name corresponding to the target service type, determining the request content of the service request as the request content of the target service request.

[0070] In an optional embodiment, each of the multiple business types has a corresponding key field name.

[0071] For example, the correspondence between an optional business type and key field names can be shown in Table 2:

[0072] Table 2

[0073]

[0074] The target processing system can determine whether the request content of a business request includes the key field name corresponding to the target business type based on the correspondence between each business type and its corresponding key field name. For example, if the target business type is fare inquiry, the script will search for the key field name "fareDisplay" in the request content; if the target business type is "refund", it will search for the key field name "RefundRequest".

[0075] When the script successfully detects the corresponding target key field name in the request content of the business request, it determines that the business request is a compliant production use case that truly belongs to that business type, i.e., it identifies the business request as the target business request. Optionally, in this case, the script can extract the complete request content of the target business request from the logs and standardize it according to a pre-defined format: retaining the original format, not adding or deleting fields, not modifying values, and removing any extra spaces or line breaks to ensure readability. Next, the script writes the request content to the corresponding preset directory based on the target business type. For example, the fare query test case is stored in the fdreq / directory. The filename can use the transaction ID from the log to ensure that each filename is unique and avoids overwriting. Finally, the request content enters the test case set for subsequent functional consistency comparison by the cloud system (target system).

[0076] Optionally, if the request content of the business request does not include the key field name corresponding to the target business type, then the business request is determined to be not the target business request.

[0077] It should be noted that by combining key field names for secondary validation, the accuracy of identifying the content of the target business request can be further improved. Compared with identifying the target business request based solely on the type identifier string, it can effectively reduce the false positive rate.

[0078] Optionally, in the data processing method provided in this application embodiment, after extracting the request content of the target business request from the system log of the source system based on string recognition, the method further includes: generating a target file based on the request content of the target business request, and determining the file name of the target file based on the request content; storing the target file in the storage path corresponding to the target business type.

[0079] Optionally, after successfully extracting the request content of a target business request, the script can perform lightweight standardization processing on the data, such as removing unnecessary whitespace characters and standardizing line breaks to obtain text with a standardized format. Subsequently, the script can search for fields in the request content that uniquely identify this request, such as the transaction ID. For example, the transaction ID in the request content can be extracted from "transId:SAT222024021610181003701210", meaning the transaction ID is represented as "SAT222024021610181003701210". The transaction ID can be identified based on the fixed field name "transId". After determining the transaction ID, the transaction ID is used as the filename of the target file, and a new file with that filename is created. The standardized request content is then completely written into this file, resulting in the target file named with that filename.

[0080] In an optional embodiment, the data format in the target file is the same as the request format of the target business request. For example, if the request format is JSON, then the data format in the target file is also JSON.

[0081] After obtaining the target file, the script can automatically determine the corresponding storage path based on the currently processing target business type. The target processing system pre-stores storage paths corresponding to different business types. These storage paths can refer to directory names (i.e., folder names) pre-defined before the script runs, used to categorize and store various production use cases. For example, the script first checks if the directory corresponding to the storage path exists; if not, it creates the directory. Subsequently, the program writes the generated target file to the corresponding directory. For instance, after a fare inquiry request is extracted, its target file will be saved in the "fdreq" directory; a refund request will be saved in the "refundreq" directory.

[0082] For example, the correspondence between an optional business type and a storage path (i.e., folder name) can be shown in Table 3:

[0083] Table 3

[0084]

[0085] It should be noted that the above method enables effective classification and storage of the request content of the target business request, thereby improving the storage reliability of the extracted data production use cases.

[0086] Optionally, in the data processing method provided in this application embodiment, before determining the target script corresponding to the target service type indicated in the target instruction from multiple preset scripts, the method further includes: extracting target parameters from the target instruction; determining the service type matching the target parameters from the correspondence between multiple parameters and multiple service types based on the target parameters; and determining the service type matching the target parameters as the target service type.

[0087] Optionally, before receiving the target instruction, the target processing system can pre-create a mutex parameter group, which includes multiple parameters, each representing a different production use case. The system can determine the user-specified target business type based on the target parameters input by the user, where the aforementioned target parameters are among the multiple parameters. Optionally, the user can input the target parameters via command line.

[0088] For example, the correspondence between an optional business type and parameters can be shown in Table 4:

[0089] Table 4

[0090]

[0091] For example, after obtaining the target instruction, the system parses the instruction content of the target instruction and identifies the target parameters based on specific field names.

[0092] In an optional embodiment, when an incorrect parameter is passed, for example, when the target parameter does not exist in Table 4, the system can issue a target prompt message to indicate that an error has occurred. The target prompt message can be used to remind the user (e.g., staff) that the parameter was passed incorrectly, and to indicate which parameters are correct, in order to prevent the error from recurring.

[0093] In an optional embodiment, if multiple target parameters are extracted, and at least two of the multiple target parameters exist in Table 4, the system can issue an error message to guide the user to re-enter the target instruction until it is determined that only one target parameter exists in Table 4 in the target instruction, and then determine the target service type based on the target parameter.

[0094] For example, after extracting the target parameter (such as -fd) from the target instruction, the system immediately queries its internally preset "parameter-business type" mapping table. This table is loaded into memory during program initialization and is in the form of a key-value dictionary, for example: {"-fd": "fare query", "-sfn": "rule query", ...}. The program uses the target parameter as the key to look up the corresponding value in this dictionary. If a match is found (such as -fd corresponding to "fare query"), the value (i.e., "fare query") is temporarily stored as the current target business type; if no match is found (such as the user entering -abc), the program immediately terminates and displays an error message, prompting the user to enter compliant parameters, ensuring that the program does not continue execution in an unknown state.

[0095] In an optional embodiment, when the parameters are passed correctly, for example, when only one target parameter exists in Table 4 (i.e., belongs to the correspondence indicated by Table 4), the system can call different target scripts according to different passed target parameters to obtain production use cases of different business types. For example, if the target parameter is "-fd", the target business type is determined to be fare query, and the target script corresponding to fare query is called to obtain the corresponding production use case, and the extracted production use case is stored in the fdreq folder; if the target parameter is "-sfn", the target business type is determined to be rule query, and the target script corresponding to rule query is called to obtain the corresponding production use case, and the extracted production use case is stored in the sfnreq folder.

[0096] In an optional embodiment, before receiving the target instruction, the target processing system can first identify which production use cases of which business types are included in the system log. For example, the system may have a pre-defined correspondence between different business types and executable file names. The system can first traverse specific areas in the system log to identify the executable file names, and then determine the business type corresponding to the executable file name as the business type involved in the system log. Therefore, when the target instruction is received, if the target parameter in the target instruction does not belong to the business type involved in the system log, an error will be reported.

[0097] For example, the correspondence between an optional business type and an executable file name can be shown in Table 5:

[0098] Table 5

[0099]

[0100] It should be noted that by using parameter-business type matching, the target business type is determined according to the target instruction, thus achieving accurate determination of the target business type and improving the reliability of data processing.

[0101] Optionally, in the data processing method provided in this application embodiment, before determining the target script corresponding to the target business type indicated in the target instruction from multiple preset scripts, the method further includes: obtaining a preset script template; for each of the multiple business types, importing the parameter set corresponding to the business type into the script template to obtain the preset script corresponding to that business type.

[0102] In an optional embodiment, the preset script template can be understood as a general, structured script framework containing a general logical skeleton for extracting a certain type of business request from system logs, but without binding specific business parameters. For example, the script template includes command-line parameter parsing structure (e.g., determining target parameters), log reading and segmentation logic (e.g., segmentation based on information headers), request header matching logic (e.g., identifying type identifier strings), request content semantic verification logic (e.g., identifying key field names), file generation and storage logic, and functions for outputting logs and statistical information.

[0103] Optionally, the parameter set for a business type includes, but is not limited to, the parameters corresponding to the business type, the type identifier string, key field names, and storage paths. For each business type, the target processing system can import the parameters from the parameter set corresponding to that business type into the corresponding location in the script template to obtain the preset script for that business type.

[0104] It should be noted that the above method enables the effective reuse of script templates, avoiding the inefficient mode of manually writing a script for each business in related technologies, thereby effectively reducing labor costs.

[0105] In an alternative embodiment, Figure 3 This is a schematic diagram of a data processing method provided according to an embodiment of this application. Figure 3An optional application process of this embodiment will be described. Optionally, upon receiving a target instruction, target parameters are extracted from the target instruction, the target business type indicated by the user is determined based on the target parameters, and the target script corresponding to the target business type is determined from multiple preset scripts. For example, if the target business type is fare query and the system log folder is "shengchanlog", then the target instruction can be: "python get_le_request.py -fdshengchanlog", where "-fd" is the target parameter; as another example, if the target business type is rule query, then the target instruction can be: "python get_le_request.py -sfnshengchanlog", where "-sfn" is the target parameter. The target parameters in each instruction are different, such as "-fd", "-sfn", etc., which are intended to determine which business type of production test case to obtain in this execution. "shengchanlog" is the storage location of the system log, which is intended to determine which folder to read the production log (i.e., the production log).

[0106] like Figure 3 As shown, after determining the target script, the target processing system can execute the target script to obtain the production use cases corresponding to the target business type. In an optional embodiment, for example, when the target instruction reads a system log, 6106 use cases can be obtained in 30 seconds after execution, including 1267 fare query production use cases, 248 rule query production use cases, 158 dynamic transfer production use cases, 4366 fare calculation production use cases, 44 ticket refund production use cases, and 23 rescheduling production use cases. If a production use case is manually extracted from the system log and stored in a designated folder, it would take about one minute. This script, by replacing manual labor, greatly shortens the time for obtaining production use cases, saves labor costs, and buys more time for subsequent comparisons.

[0107] In this embodiment, by determining the target script corresponding to the target business type indicated in the target instruction from multiple preset scripts, the target script for extracting specific production use cases is dynamically determined according to actual needs. By using the target script, the request content of the target business request is extracted from the system log of the source system based on string recognition. This realizes the automatic acquisition of production use cases using the target script, avoiding manual extraction of production use cases, thereby effectively reducing labor costs.

[0108] Therefore, the method provided in this application achieves the goal of automatically extracting production test cases using scripts, realizes the technical effect of reducing labor costs, and solves the technical problem of high labor costs in the scenario of business migration between old and new systems, where production test cases are extracted manually from the old system to test the new system.

[0109] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0110] Example 2

[0111] According to an embodiment of this application, a data processing system for implementing the above-described data processing method is also provided, the system comprising:

[0112] The source system is used to receive business requests of multiple business types and perform business processing on the requests.

[0113] The target processing system is used to execute the data processing method in Example 1;

[0114] The target system is the system to which the business migration from the source system is directed.

[0115] In an optional embodiment, the source system and the target system are the old and new systems involved in the business migration scenario. The source system is the old system, for example, the system currently being migrated and originally deployed locally. Its runtime logs (i.e., system logs) record a massive amount of real requests (i.e., business requests), serving as the source of production test cases. The target system is the new system, for example, a cloud-based system under a new architecture that is being developed or deployed. Its goal is to maintain complete consistency with the source system in terms of functionality and results; therefore, it needs to use the same inputs (i.e., production test cases) for testing. For example, after extracting the production test cases (i.e., the request content of the target business request), the production test cases are sent as test requests to the target system, and the request processing results from the target system in response to the test requests are obtained. The request processing results from the target system are compared with the request processing results from the source system in response to the production test cases, thereby determining whether the target system has any anomalies based on the comparison results, thus achieving reliability testing.

[0116] In an optional embodiment, the target processing system may execute the data processing method in Embodiment 1 and perform system reliability testing on the target system based on the extracted request content of the target business request.

[0117] In this embodiment, by determining the target script corresponding to the target business type indicated in the target instruction from multiple preset scripts, the target script for extracting specific production use cases is dynamically determined according to actual needs. By using the target script, the request content of the target business request is extracted from the system log of the source system based on string recognition. This realizes the automatic acquisition of production use cases using the target script, avoiding manual extraction of production use cases, thereby effectively reducing labor costs.

[0118] Therefore, the method provided in this application achieves the goal of automatically extracting production test cases using scripts, realizes the technical effect of reducing labor costs, and solves the technical problem of high labor costs in the scenario of business migration between old and new systems, where production test cases are extracted manually from the old system to test the new system.

[0119] Example 3

[0120] This application also provides a data processing apparatus. It should be noted that the data processing apparatus of this application can be used to execute the data processing method provided in this application. The data processing apparatus provided in this application will be described below.

[0121] According to an embodiment of this application, an apparatus for implementing the above-described data processing method is also provided, such as... Figure 4 As shown, the device includes:

[0122] The first determining module 401 is used to determine, upon receiving a target instruction, the target script corresponding to the target service type indicated in the target instruction from a plurality of preset scripts, wherein the target instruction is used to indicate the request content for obtaining the target service request, and the target service request refers to the service request corresponding to the target service type.

[0123] Extraction module 402 is used to extract the request content of the target business request from the system log of the source system using the target script and based on string recognition. The system log is used to record the request content of multiple business requests of various business types received by the source system. The request content of the target business request is used to perform business testing on the target system. The target system is the system to which the business migration of the source system is directed.

[0124] In this embodiment, by determining the target script corresponding to the target business type indicated in the target instruction from multiple preset scripts, the target script for extracting specific production use cases is dynamically determined according to actual needs. By using the target script, the request content of the target business request is extracted from the system log of the source system based on string recognition. This realizes the automatic acquisition of production use cases using the target script, avoiding manual extraction of production use cases, thereby effectively reducing labor costs.

[0125] Therefore, the method provided in this application achieves the goal of automatically extracting production test cases using scripts, realizes the technical effect of reducing labor costs, and solves the technical problem of high labor costs in the scenario of business migration between old and new systems, where production test cases are extracted manually from the old system to test the new system.

[0126] Optionally, in the data processing apparatus provided in this application embodiment, the extraction module further includes: an identification submodule, used to identify multiple business requests from the system log using a target script; a judgment submodule, used to determine, for each business request, whether the request content of the business request includes a type identifier string corresponding to the target business type; and a first determination submodule, used to determine the request content of the business request as the request content of the target business request if the request content of the business request includes a type identifier string.

[0127] Optionally, in the data processing apparatus provided in this application embodiment, the identification submodule further includes: a first identification unit, used to identify an information header from the system log, wherein the information header is used to segment the request content of different business requests; and a second identification unit, used to identify multiple business requests from the system log based on the information header.

[0128] Optionally, in the data processing apparatus provided in this application embodiment, the first determining submodule further includes: a judging unit, used to judge whether the request content of the service request includes a key field name corresponding to the target service type; and a determining unit, used to determine the request content of the service request as the request content of the target service request when the request content of the service request includes a key field name corresponding to the target service type.

[0129] Optionally, in the data processing apparatus provided in the embodiments of this application, the data processing apparatus further includes: a second determining module, used to generate a target file based on the request content of the target business request, and determine the file name of the target file based on the request content; and a storage module, used to store the target file in the storage path corresponding to the target business type.

[0130] Optionally, in the data processing apparatus provided in the embodiments of this application, the data processing apparatus further includes: an extraction module, used to extract target parameters from target instructions; a second determination module, used to determine the business type matching the target parameters from the correspondence between multiple parameters and multiple business types based on the target parameters; and a third determination module, used to determine the business type matching the target parameters as the target business type.

[0131] Optionally, in the data processing apparatus provided in the embodiments of this application, the data processing apparatus further includes: an acquisition module, used to acquire a preset script template; and an import module, used to import the parameter set corresponding to each of the multiple business types into the script template to obtain a preset script corresponding to that business type.

[0132] It should be noted that the first determining module 401 and the extraction module 402 mentioned above correspond to steps S201 to S202 in Embodiment 1. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules or units can be hardware components or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above modules can also be part of the device and run in the computer terminal 10 provided in Embodiment 1.

[0133] Example 4

[0134] Embodiments of this application may provide an electronic device. Figure 5 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 5 As shown, the electronic device may include: one or more ( Figure 5 (Only one is shown) processor 1002, memory 1004, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0135] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the above-described methods. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0136] The processor can invoke information and applications stored in memory via a transmission device to perform the following steps: Upon receiving a target instruction, determine the target script corresponding to the target service type indicated in the target instruction from multiple preset scripts, wherein the target instruction is used to instruct the acquisition of the request content of the target service request, and the target service request refers to the service request corresponding to the target service type; Using the target script, extract the request content of the target service request from the system log of the source system based on string recognition, wherein the system log is used to record the request content of multiple service types of service requests received by the source system, and the request content of the target service request is used to perform service testing on the target system, and the target system is the system to which the service migration of the source system is directed.

[0137] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: using the target script, identify multiple business requests from the system log; for each business request, determine whether the request content of the business request includes a type identifier string corresponding to the target business type; if the request content of the business request includes a type identifier string, determine the request content of the business request as the request content of the target business request.

[0138] The processor can also access information and applications stored in memory via a transmission device to perform the following steps: identify headers from the system log, whereby the headers are used to segment the request content of different business requests; and identify multiple business requests from the system log based on the headers.

[0139] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: determine whether the request content of the service request includes the key field name corresponding to the target service type; if the request content of the service request includes the key field name corresponding to the target service type, determine the request content of the service request as the request content of the target service request.

[0140] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: after extracting the request content of the target business request from the system log of the source system based on string recognition, generating a target file based on the request content of the target business request, and determining the file name of the target file based on the request content; storing the target file in the storage path corresponding to the target business type.

[0141] The processor can also invoke information and applications stored in the memory via the transmission device to perform the following steps: extracting target parameters from the target instructions before determining the target script corresponding to the target service type indicated in the target instructions from multiple preset scripts; determining the service type that matches the target parameters from the correspondence between multiple parameters and multiple service types based on the target parameters; and determining the service type that matches the target parameters as the target service type.

[0142] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: before determining the target script corresponding to the target service type indicated in the target instruction from multiple preset scripts, obtain a preset script template; for each of the multiple service types, import the parameter set corresponding to the service type into the script template to obtain the preset script corresponding to that service type.

[0143] Those skilled in the art will understand that Figure 5 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones, tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 5 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 5 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 5 The different configurations shown.

[0144] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0145] Example 5

[0146] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the data processing method provided in Embodiment 1.

[0147] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0148] This application also provides a computer program product, which, when executed on a data processing device, is a program adapted to perform data processing method steps.

[0149] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0150] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0151] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0152] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0153] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0154] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0155] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A data processing method, characterized in that, include: Upon receiving a target instruction, a target script corresponding to the target service type indicated in the target instruction is determined from a plurality of preset scripts. The target instruction is used to indicate the request content for obtaining the target service request, and the target service request refers to the service request corresponding to the target service type. Using the target script, the request content of the target business request is extracted from the system log of the source system based on string recognition. The system log is used to record the request content of multiple business requests of various business types received by the source system. The request content of the target business request is used to perform business testing on the target system. The target system is the system to which the business migration of the source system is directed.

2. The method according to claim 1, characterized in that, Using the target script, the request content of the target business request is extracted from the system logs of the source system based on string recognition, including: Using the target script, multiple business requests are identified from the system logs; For each business request, determine whether the request content of the business request includes the type identifier string corresponding to the target business type; If the request content of the service request includes the type identifier string, then the request content of the service request is determined to be the request content of the target service request.

3. The method according to claim 2, characterized in that, Using the target script, multiple business requests are identified from the system logs, including: The header information is identified from the system log, wherein the header information is used to segment the request content of different business requests; Based on the information header, the multiple business requests are identified from the system log.

4. The method according to claim 2, characterized in that, The request content of the service request is determined as the request content of the target service request, including: Determine whether the request content of the service request includes the key field name corresponding to the target service type; If the request content of the service request includes the key field name corresponding to the target service type, then the request content of the service request is determined as the request content of the target service request.

5. The method according to claim 1, characterized in that, After extracting the request content of the target business request from the system logs of the source system using string recognition, the method further includes: A target file is generated based on the request content of the target service request, and the filename of the target file is determined based on the request content; Store the target file in the storage path corresponding to the target business type.

6. The method according to claim 1, characterized in that, Before determining the target script corresponding to the target service type indicated in the target instruction from multiple preset scripts, the method further includes: Extract the target parameters from the target instruction; Based on the target parameter, determine the business type that matches the target parameter from the correspondence between multiple parameters and multiple business types; The business type that matches the target parameter is determined as the target business type.

7. The method according to claim 1, characterized in that, Before determining the target script corresponding to the target service type indicated in the target instruction from multiple preset scripts, the method further includes: Get the preset script template; For each of the multiple business types, the parameter set corresponding to the business type is imported into the script template to obtain the preset script corresponding to that business type.

8. A data processing system, characterized in that, include: The source system is used to receive business requests of multiple business types and perform business processing on the requests. A target processing system for performing the data processing method according to any one of claims 1 to 7; The target system is the system to which the business migration of the source system is directed.

9. A data processing apparatus, characterized in that, include: The first determining module is used to determine, upon receiving a target instruction, the target script corresponding to the target service type indicated in the target instruction from a plurality of preset scripts, wherein the target instruction is used to indicate the request content for obtaining the target service request, and the target service request refers to the service request corresponding to the target service type. The extraction module is used to extract the request content of the target business request from the system log of the source system using the target script and based on string recognition. The system log is used to record the request content of multiple business requests of various business types received by the source system. The request content of the target business request is used to perform business testing on the target system. The target system is the system to which the business migration of the source system is directed.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the computer-readable storage medium is located to perform the data processing method according to any one of claims 1 to 7.

Images