Method and device for quickly reading MIR information of gzip compressed stdf file

Abstract

The application relates to the technical field of data processing, and discloses a method and device for quickly reading MIR information of a gzip-compressed stdf file, which comprises the following steps: screening a stdf file list, reading the front part of bytes, judging whether the file is a legal compressed file, decompressing the legal compressed file, judging whether the decompressed file is an illegal file, identifying MIR records and analyzing detailed contents if the file is not an illegal file, and judging whether the file is a target file that needs complete analysis according to the detailed contents. The application only needs to read part of the file information, then identify the compression algorithm according to a characteristic code, decompress in the memory, and quickly analyze MIR information in the file header, so that the program version number is obtained, the time consumption of identification is reduced from the minute level to the millisecond level, resource occupation is greatly reduced, and the processing efficiency is effectively improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of data processing, in particular to a method and device for quickly reading MIR information of a gzip-compressed stdf file. BACKGROUND

[0002] During yield monitoring, semiconductor testing generally only monitors PTR (parameter test record) and FTR (function test record) records, and does not monitor DTR (data log text record) because it only records some log information. However, sometimes human errors and non-standard operations write key PTR or FTR data into DTR records, thereby causing an RTM system (real-time monitoring system) to be unable to monitor the yield of products.

[0003] The application causes the RTM system to be unable to monitor because a test program of a customer writes FTR data in DTR. However, only data generated by a program with version No. 8 is affected, and finally the data is directly compressed and stored in a certain place after being parsed. Therefore, a software needs to be developed to identify which STDF files (program version information is stored in MIR records) are generated by the program with version No. 8, then convert the files into a normal data structure (write data in DTR into FTR after being parsed), and then parse and import the files into the system to perform offline yield analysis. Because there are a large number of stdf files, two problems exist.

[0004] 1. All files need to be decompressed before being parsed, and 300M of files is 3G after being decompressed, so the inspection efficiency is very low, and a lot of resources are occupied;

[0005] 2. An open-source parsing library needs to return a parsing result after parsing the entire file by default, so the parsing process is very long and occupies a lot of memory.

[0006] An existing technical solution is that manually screened stdf files generated by the program with version No. 8 are manually decompressed, and then data is imported into the system according to a conversion rule provided by the customer. Manual screening is not only inefficient and has a large workload, but also may miss some files. SUMMARY

[0007] The application aims to overcome the deficiencies of the prior art and provide a method and device for quickly reading MIR information of a gzip-compressed stdf file.

[0008] In a first aspect, a method for quickly reading MIR information of a gzip-compressed stdf file is provided, which includes the following steps.

[0009] Preliminarily screening a list of compressed stdf files according to a wildcard;

[0010] reading the first N bytes of the compressed stdf file, wherein N is a positive integer multiple of 1024;

[0011] determining whether the compressed stdf file is a legal compressed file according to the first two bytes of the file header in the compressed stdf file;

[0012] in response to the compressed stdf file being a legal compressed file, decompressing the first N bytes according to the algorithm of gzip;

[0013] determining whether the decompressed file is an illegal file according to the first six bytes of the decompressed file;

[0014] in response to the decompressed file not being an illegal file, identifying a MIR record from the decompressed file;

[0015] reading the complete MIR record byte stream by length and skipping the first four bytes to parse the detailed content of the MIR record;

[0016] determining whether the current MIR record is a target file that needs to be completely parsed according to the parsing result.

[0017] Further, the wildcard is any one of *.stdf.gz or *.gz.

[0018] Further, the first 2048 bytes of the compressed stef file are read, and if a complete MIR record is not read, the first 2048*n bytes are read, wherein n is the number of times of reading the compressed stdf file, until a complete MIR record is read to stop the loop.

[0019] Further, determining whether the compressed stdf file is a legal compressed file according to the first two bytes of the file header in the compressed stdf file comprises:

[0020] reading the first two bytes of the file header in the compressed stdf file;

[0021] determining whether the first two bytes of the file header in the compressed stdf file are 31 and 139;

[0022] in response to the first two bytes of the file header in the compressed stdf file being 31 and 139, determining that the current compressed stdf file is a legal compressed file;

[0023] in response to the first two bytes of the file header in the compressed stdf file not being 31 and 139, determining that the current compressed stdf file is not a legal compressed file.

[0024] Further, determining whether the decompressed file is an illegal file according to the first six bytes of the decompressed file comprises:

[0025] The byte order is determined by the fifth byte of the decompressed file. If the fifth byte is less than 2, the byte order is determined to be high-order, and the length of FAR is read from the second byte. If the fifth byte is greater than or equal to 2, the byte order is determined to be low-order, and the length of FAR is read from the first byte.

[0026] If the length of the FAR file is not 2, then the decompressed file is determined to be an invalid file.

[0027] Since the length of FAR is 2, it checks whether the third and fourth bytes are 0 and 10. If the third byte is not 0 or the fourth byte is not 10, the decompressed file is considered invalid. If the third byte is 0 and the fourth byte is 10, the decompressed file is considered not invalid.

[0028] Furthermore, MIR records were identified from the decompressed files, including:

[0029] Read four bytes starting from the seventh byte of the decompressed file. Among these four bytes: the first two bytes are the total length of the record, the third byte is the main type of the record, and the fourth byte is the subtype of the record.

[0030] If the primary type of a record is 1 and the subtype is 10, then the current record is determined to be an MIR record.

[0031] If the primary type of a record is not 1 or the subtype is not 10, it is determined that the current record is not a MIR record. The current record is skipped and four bytes are read to continue identifying MIR records until a MIR record is identified.

[0032] Furthermore, based on the parsing results, it is determined whether the current MIR record is a target file that needs to be fully parsed, including:

[0033] Determine whether the string in the parsed result contains the keyword "8" based on EXEC_TYP, and whether FAMLY_ID represents the target customer;

[0034] If the string in the parsed result contains the keyword 8, and FAMLY_ID is the target customer, then the current MIR record is the target file that needs to be parsed.

[0035] Secondly, an apparatus for quickly reading MIR information from gzip-compressed stdf files is provided, comprising:

[0036] The preliminary filtering module is used to initially filter the list of compressed stdf files based on wildcards;

[0037] The read module is used to read the first N bytes of a compressed stdf file, where N is a positive integer multiple of 1024;

[0038] The first judgment module is used to determine whether a compressed file is valid based on the first two bytes of the file header in the compressed stdf file.

[0039] The decompression module is used to decompress the first N bytes according to the gzip algorithm if the compressed stdf file is valid.

[0040] The second judgment module is used to determine whether the file is illegal based on the first 6 bytes of the decompressed file;

[0041] The identification module is used to identify MIR records from the decompressed file if the decompressed file is not an illegal file.

[0042] The parsing module is used to read the complete MIR record byte stream by length and skip the first 4 bytes to parse the detailed content of the MIR record;

[0043] The third judgment module is used to determine whether the current MIR record is a target file that needs to be fully parsed based on the parsing results.

[0044] Thirdly, a computer-readable storage medium is provided that stores program code for execution by a device, the program code including steps for performing a method as described in any implementation of the first aspect.

[0045] Fourthly, an electronic device is provided, the electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method as in any implementation of the first aspect.

[0046] This application has the following beneficial effects: It only needs to read part of the file information, then identify the compression algorithm based on the feature code, decompress it in memory, and quickly parse the MIR information in the file header to obtain the program version number. It also reduces the original identification time from minutes to milliseconds and greatly reduces resource consumption, thereby effectively improving processing efficiency. Moreover, it can be quickly fine-tuned for specific business needs, and does not require too much investment in storage and computing, which can be handled by a general server. Attached Figure Description

[0047] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an undue limitation of this application.

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and all other drawings obtained by those of ordinary skill in the art without creative effort based on these drawings are within the protection scope of the present application.

[0049] Figure 1 is a flow chart of the method for quickly reading MIR information of a gzip compressed stdf file according to the first embodiment of the present application;

[0050] Figure 2 is a flow chart of the method for quickly reading MIR information of a gzip compressed stdf file according to the first embodiment of the present application;

[0051] Figure 3 is a flow chart of the method for quickly reading MIR information of a gzip compressed stdf file according to the first embodiment of the present application;

[0052] Figure 4 is a structural block diagram of the device for quickly reading MIR information of a gzip compressed stdf file according to the second embodiment of the present application. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present application.

[0054] Embodiment One

[0055] The method for quickly reading MIR information of a gzip compressed stdf file according to the first embodiment of the application comprises the following steps: preliminarily screening a list of compressed stdf files according to a wildcard; reading the first N bytes of the compressed stdf file, wherein N is a positive integer multiple of 1024; judging whether the compressed stdf file is a legal compressed file according to the first two bytes of the file header in the compressed stdf file; in response to the compressed stdf file being a legal compressed file, decompressing the first N bytes according to the gzip algorithm; judging whether the decompressed file is an illegal file according to the first six bytes of the decompressed file; in response to the decompressed file not being an illegal file, identifying the MIR record from the decompressed file; reading the complete MIR record byte stream according to the length and skipping the first four bytes to analyze the detailed content of the MIR record; and judging whether the current MIR record is a target file that needs to be completely analyzed according to the analysis result. The technical solution of the application only needs to read partial information of the file, then identify the compression algorithm according to the characteristic code, decompress in the memory, and quickly analyze the MIR information of the file header, so as to obtain the program version number, reduce the identification time from the minute level to the millisecond level, greatly reduce the resource occupation, effectively improve the processing efficiency, and make quick fine adjustment according to specific business requirements without too much investment in storage and calculation, so that a general server can bear it.

[0056] Specifically, Figure 1 The flowchart of the method for quickly reading MIR information of a gzip compressed stdf file in the first embodiment of the application is shown, which comprises the following steps:

[0057] S101, preliminarily screening a list of compressed stdf files according to a wildcard;

[0058] The wildcard is preferably "*.stdf.gz", the file suffix after gzip compression is ".gz", and here "*.stdf.gz" is used as the wildcard, which is a kind of convention. By retaining the original file suffix, the original file type can be quickly distinguished. Of course, the wildcard can also be selected as "*.gz", so that the screened file can be a gzip compressed file with a high probability, but it cannot be guaranteed to be an stdf, which will relatively reduce the execution efficiency.

[0059] S102, reading the first N bytes of the compressed stdf file, wherein N is a positive integer multiple of 1024, which is considered from the bottom layer storage logic. A data block is generally 1024, otherwise the reading efficiency will be affected. If the number of bytes of the file is less than N, the entire file is read;

[0060] For example, N is 2048, that is, the first 2048 bytes of the compressed stef file are read, and 2048 is used here in combination with the stdf file structure and the actual compression ratio. Too little will increase the probability of insufficient reading, and too much will affect performance. The compression ratio of gzip is about 3-10 times, which means that 6144-20480 bytes of the original file are read. This can probably guarantee the inclusion of MIR record content and reduce the probability of secondary reading due to insufficient reading length. Why is it a probability? This is related to the stdf file structure. A simplest stdf file has at least four records in order: FAR->MIR->PCR->MRR. The length of FAR is fixed, which is 6 bytes. The length of MIR will vary according to the written content, but the maximum length will not exceed 7669 bytes. The only possible cause of insufficient reading is the ATR record, which is an optional record between FAR and MIR and can appear multiple times, with a maximum length of 263 bytes per record. If the first 2048 bytes of the compressed stdf file are not read to complete the MIR record, read 2048*n bytes, where n is the number of times the compressed stdf file is read, until the complete MIR record is read to stop the loop. For example: the first reading of 2048 bytes shows that the length is not enough (that is, the complete MIR record is not read), the second reading is 2048*2=4096 bytes, if the complete MIR record is still not read, the third reading is 2048*3=6144 bytes, and so on until the complete MIR record is read, as shown in Figure 2 .

[0061] S103, judging whether the compressed stdf file is a legal compressed file according to the first two bytes of the file header in the compressed stdf file;

[0062] Specifically, judging whether the compressed stdf file is a legal compressed file according to the first two bytes of the file header in the compressed stdf file, comprising:

[0063] S301, reading the first two bytes of the file header in the compressed stdf file;

[0064] S302, judging whether the first two bytes of the file header in the compressed stdf file are 31 and 139;

[0065] S303, in response to the first two bytes of the file header in the compressed stdf file being 31 and 139, judging that the current compressed stdf file is a legal compressed file;

[0066] S304, in response to the first two bytes of the file header in the compressed stdf file not being 31 and 139, judging that the current compressed stdf file is not a legal compressed file.

[0067] It should be noted that the first two bytes of the gzip compressed file are 31 and 139 respectively, so only two bytes need to be read to determine whether it is a legal gzip compressed file.

[0068] S104, in response to the compressed stdf file being legal, decompressing the first N bytes according to the algorithm of gzip;

[0069] It should be noted that in general, the decompressed file is incomplete and cannot be decompressed, but if it is a continuous file stream starting from the head, the gzip compressed file can be decompressed, which is a feature of gzip compression.

[0070] S105, judging whether the file is illegal according to the first 6 bytes of the decompressed file;

[0071] Specifically, judging whether the file is illegal according to the first 6 bytes of the decompressed file, including:

[0072] S501, judging whether the byte order is high or low according to the fifth byte of the decompressed file, if the fifth byte is less than 2, it is judged that the byte order is high, and the length of the FAR is read from the second byte, if the fifth byte is greater than or equal to 2, it is judged that the byte order is low, and the length of the FAR is read from the first byte;

[0073] S502, in response to the length of the FAR not being 2, judging that the decompressed file is an illegal file;

[0074] S503, in response to the length of the FAR being 2, judging whether the third byte and the fourth byte are 0 and 10, if the third byte is not 0 or the fourth byte is not 10, judging that the decompressed file is an illegal file, if the third byte is 0 and the fourth byte is 10, judging that the decompressed file is not an illegal file.

[0075] Here, the first 6 bytes read are the FAR record, which can be used to judge whether the decompressed file is a legal SFTD file.

[0076] It should be noted that for the decompressed gzip file, the byte order can be judged according to the fifth byte, if it is less than 2, it is high, and the length of the FAR is read from the second byte; otherwise, it is low, and the length of the FAR is read from the first byte; in addition, the length of the FAR is a fixed value of 2, if the length of the FAR is not 2, it is an illegal file; if the length of the FAR is 2, then judge whether the third byte and the fourth byte are 0 and 10, if one of them is not, it is an illegal file.

[0077] S106, in response to the decompressed file is not illegal file, then identify the MIR record from the decompressed file;

[0078] Specifically, identify the MIR record from the decompressed file, comprising:

[0079] From the seventh byte of the decompressed file, read four bytes, wherein, in the four bytes read: the first two bytes are the total length of the record, the third byte records the main type of the record, and the fourth byte records the sub-type of the record;

[0080] In response to the main type of the record being 1 and the sub-type being 10, the current record is determined to be an MIR record;

[0081] In response to the main type of the record not being 1 or the sub-type not being 10, the current record is determined to be an MIR record, and the current record is skipped to read four bytes to continue identifying the MIR record until the MIR record is identified.

[0082] Please refer to Table 1, read 4 bits from the seventh byte, wherein the first two bits are the length of the record, the third bit is the main type of the record, and the fourth bit is the sub-type of the record. If the main type is 1 and the sub-type is 10, the current record is an MIR record, otherwise skip the current record by length (4 + read record length), for example: the length of the current record is 250 bytes, then skip 250 + 4 = 254 bytes from the seventh byte, so as to skip the current record exactly, and then continue to read the next record according to the current parsing rule until the MIR record is found.

[0083] Table 1: Format of gzip file after decompression

[0084]

[0085] S107, read the complete MIR record byte stream by length, and skip the first 4 bytes to parse the detailed content of the MIR record;

[0086] It should be noted that, please refer to Figure 3 , the parsing rule is as follows (the parsing of numbers involves byte sequence, which needs to be parsed according to high and low bits, the high bit needs to be reversed, the low bit does not need to be processed, the single byte number is not affected, and the parsing process will not be described again because the length of the MIR front record is not fixed, and the starting position of the parsing can only be positioned to the first bit of the MIR record excluding the head 4 bytes) :

[0087] 1. SETUP_T is a 4-byte unsigned number, reading the first, second, third and fourth bytes, and then parsing into a number;

[0088] 2. START_T is a 4-byte unsigned number, reads the 5th, 6th, 7th, 8th byte, and then parses it into a number;

[0089] 3. STAT_NUM is a 1-byte unsigned number, reads the 9th byte, and then parses it into a number;

[0090] 4. MODE_COD is a 1-byte string, reads the 10th byte, and then parses it into a string;

[0091] 5. RTST_COD is a 1-byte string, reads the 11th byte, and then parses it into a string;

[0092] 6. PROT_COD is a 1-byte string, reads the 12th byte, and then parses it into a string;

[0093] 7. BURN_TIM is a 2-byte unsigned number, reads the 13th, 14th byte, and then parses it into a number;

[0094] 8. CMOD_COD is a 1-byte string, reads the 15th byte, and then parses it into a string;

[0095] 9. LOT_ID is a string of indefinite length, reads the 16th byte, then parses it into a string length, reads the byte array according to the length, and finally parses it into a string according to ASCII;

[0096] 10. The remaining attributes are all strings of indefinite length, which are parsed according to the above rules.

[0097] S108, according to the parsing result, judge whether the current MIR record is the target file that needs to be completely parsed.

[0098] Specifically, according to the parsing result, judge whether the current MIR record is the target file that needs to be completely parsed, including:

[0099] According to EXEC_TYP, judge whether the string of the parsing result contains the keyword 8., and whether FAMLY_ID is the target customer;

[0100] In response to the string of the parsing result containing the keyword 8., and FAMLY_ID being the target customer, the current MIR record is the target file that needs to be parsed.

[0101] Wherein, EXEC_TYP is the 16th attribute of MIR, used to record the version of test program, FAMLY_ID is the 23rd attribute of MIR, used to record the number of customer, for example, the target customer ID is ABC, and the scanning here is only for ABC customer, so the program configuration file will configure ABC, so as to determine whether it is the target customer.

[0102] As shown in Figure 2 If the target file (i.e. the MIR record of the target customer) to be identified is not found in step S108, return to step S102 and read the first N*n bytes, wherein N is the number of initial reading bytes, n is the number of times of reading the compressed stdf file, and continue to execute steps S103-S108 until the complete MIR record is read, and the loop is stopped.

[0103] Embodiment two

[0104] As shown in Figure 4 The device for quickly reading MIR information of a gzip compressed stdf file according to the second embodiment of the application comprises:

[0105] A preliminary screening module, configured to preliminarily screen a list of compressed stdf files according to a wildcard;

[0106] A reading module, configured to read the first N bytes of the compressed stdf file, wherein N is a positive integer multiple of 1024;

[0107] A first judging module, configured to judge whether the compressed stdf file is a legal compressed file according to the first two bytes of the file header in the compressed stdf file;

[0108] A decompression module, configured to, in response to the compressed stdf file being a legal compressed file, decompress the first N bytes according to the algorithm of gzip;

[0109] A second judging module, configured to judge whether the decompressed file is an illegal file according to the first six bytes of the decompressed file;

[0110] An identifying module, configured to, in response to the decompressed file not being an illegal file, identify the MIR record from the decompressed file;

[0111] An analyzing module, configured to read the complete MIR record byte stream by length and analyze the detailed content of the MIR record by skipping the first four bytes;

[0112] A third judging module, configured to judge whether the current MIR record is a target file to be completely analyzed according to the analysis result.

[0113] Embodiment three

[0114] The computer readable storage medium involved in the third embodiment of the application stores program codes for device execution, and the program codes include steps for executing the method in any one of the implementation manners in the first embodiment of the application.

[0115] The computer readable storage medium can be a read only memory (ROM), a static storage device, a dynamic storage device or a random access memory (RAM). The computer readable storage medium can store program codes, and when the program stored in the computer readable storage medium is executed by a processor, the processor is used to execute the steps of the method in any one of the implementation manners in the first embodiment of the application.

[0116] Embodiment four

[0117] The electronic device involved in the fourth embodiment of the application includes a processor, a memory and a program or instructions stored on the memory and executable on the processor. When the program or instructions are executed by the processor, the method in any one of the implementation manners in the first embodiment of the application is implemented.

[0118] The processor can be a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU) or one or more integrated circuits, and is used to execute related programs to implement the method in any one of the implementation manners in the first embodiment of the application.

[0119] The processor can also be an integrated circuit electronic device with signal processing capability. In the implementation process, each step of the method in any one of the implementation manners in the first embodiment of the application can be completed by integrated logic circuits or instruction software in the processor.

[0120] The above processor can also be a general processor, a digital signal processor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can be any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly embodied as hardware code processor execution or executed by hardware and software module combination in the code processor. The software module can be located in the random access memory, the flash memory, the read only memory, the programmable read only memory or the electrically erasable programmable memory, the register or the mature storage medium in the art. The storage medium is located in the memory, the processor reads the information in the memory, combines the hardware to complete the function required by the unit in the data processing device of the embodiments of the present application, or executes the method in any one of the implementation manners in the embodiments of the present application.

[0121] The above is only the preferred specific embodiment of the present application; however, the protection scope of the present application is not limited to this. Any skilled person in the art can make equivalent replacement or change according to the technical range disclosed in the present application and the improvement concept of the technical solution and the embodiments of the present application, which should be covered in the protection scope of the present application.

Claims

1. A method for quickly reading MIR information from a gzip-compressed stdf file, characterized in that, include: A preliminary list of compressed stdf files is generated based on wildcards; Read the first N bytes of the compressed stdf file, where N is a positive integer multiple of 1024; Determine if the compressed file is valid by checking the first two bytes of its header in the compressed stdf file; If the compressed stdf file is valid, then the first N bytes are decompressed according to the gzip algorithm; Determine if the file is invalid based on the first 6 bytes of the decompressed file; If the decompressed file is not an illegal file, then the MIR record is identified from the decompressed file; Read the complete MIR record byte stream by length, and skip the first 4 bytes to parse the detailed content of the MIR record; Based on the parsing results, determine whether the current MIR record is a target file that needs to be fully parsed; The determination of whether a compressed file is valid is based on the first two bytes of the file header in the compressed stdf file, including: Read the first two bytes of the file header from the compressed stdf file; Determine if the first two bytes of the header in a compressed stdf file are 31 and 139; If the first two bytes of the header of a compressed stdf file are 31 and 139, then the compressed stdf file is considered a valid compressed file. If the first two bytes of the header of a compressed stdf file are not 31 and 139, then the current compressed stdf file is determined to be an invalid compressed file. Determine if a file is invalid based on the first 6 bytes of the decompressed file, including: The byte order is determined by the fifth byte of the decompressed file. If the fifth byte is less than 2, the byte order is determined to be high-order, and the length of FAR is read from the second byte. If the fifth byte is greater than or equal to 2, the byte order is determined to be low-order, and the length of FAR is read from the first byte. If the length of the FAR file is not 2, then the decompressed file is determined to be an invalid file. Since the length of FAR is 2, it checks whether the third and fourth bytes are 0 and 10. If the third byte is not 0 or the fourth byte is not 10, the decompressed file is considered invalid. If the third byte is 0 and the fourth byte is 10, the decompressed file is considered not invalid.

2. The method for quickly reading MIR information of gzip-compressed stdf files according to claim 1, characterized in that, The wildcard is either *.stdf.gz or *.gz.

3. The method for quickly reading MIR information of gzip-compressed stdf files according to claim 1, characterized in that, Read the first 2048 bytes of the compressed stdf file. If the complete MIR record is not read, read the first 2048*n bytes, where n is the number of times the compressed stdf file is read, until the complete MIR record is read and the loop stops.

4. The method for quickly reading MIR information of gzip-compressed stdf files according to claim 1, characterized in that, MIR records were identified from the decompressed files, including: Read four bytes starting from the seventh byte of the decompressed file. Among these four bytes: the first two bytes are the total length of the record, the third byte is the main type of the record, and the fourth byte is the subtype of the record. If the primary type of a record is 1 and the subtype is 10, then the current record is determined to be an MIR record. If the primary type of a record is not 1 or the subtype is not 10, it is determined that the current record is not a MIR record. The current record is skipped and four bytes are read to continue identifying MIR records until a MIR record is identified.

5. The method for quickly reading MIR information of a gzip-compressed stdf file according to any one of claims 1-4, characterized in that, Based on the parsing results, determine whether the current MIR record is a target file that needs to be fully parsed, including: Determine whether the string in the parsed result contains the keyword "8" based on EXEC_TYP, and whether FAMLY_ID represents the target customer; If the string in the parsed result contains the keyword 8, and FAMLY_ID is the target customer, then the current MIR record is the target file that needs to be parsed.

6. An apparatus for quickly reading MIR information from gzip-compressed stdf files, characterized in that, include: The preliminary filtering module is used to initially filter the list of compressed stdf files based on wildcards; The read module is used to read the first N bytes of a compressed stdf file, where N is a positive integer multiple of 1024; The first judgment module is used to determine whether a compressed file is valid based on the first two bytes of the file header in the compressed stdf file. The decompression module is used to decompress the first N bytes according to the gzip algorithm if the compressed stdf file is valid. The second judgment module is used to determine whether the file is illegal based on the first 6 bytes of the decompressed file; The identification module is used to identify MIR records from the decompressed file if the decompressed file is not an illegal file. The parsing module is used to read the complete MIR record byte stream by length and skip the first 4 bytes to parse the detailed content of the MIR record; The third judgment module is used to determine whether the current MIR record is a target file that needs to be completely parsed based on the parsing results; The determination of whether a compressed file is valid is based on the first two bytes of the file header in the compressed stdf file, including: Read the first two bytes of the file header from the compressed stdf file; Determine if the first two bytes of the header in a compressed stdf file are 31 and 139; If the first two bytes of the header of a compressed stdf file are 31 and 139, then the compressed stdf file is considered a valid compressed file. If the first two bytes of the header of a compressed stdf file are not 31 and 139, then the current compressed stdf file is determined to be an invalid compressed file. Determine if a file is invalid based on the first 6 bytes of the decompressed file, including: The byte order is determined by the fifth byte of the decompressed file. If the fifth byte is less than 2, the byte order is determined to be high-order, and the length of FAR is read from the second byte. If the fifth byte is greater than or equal to 2, the byte order is determined to be low-order, and the length of FAR is read from the first byte. If the length of the FAR file is not 2, then the decompressed file is determined to be an invalid file. Since the length of FAR is 2, it checks whether the third and fourth bytes are 0 and 10. If the third byte is not 0 or the fourth byte is not 10, the decompressed file is considered invalid. If the third byte is 0 and the fourth byte is 10, the decompressed file is considered not invalid.

7. A computer-readable storage medium, characterized in that, The computer-readable medium stores program code for execution by the device, the program code including steps for performing the method as described in any one of claims 1-5.

8. An electronic device, characterized in that, The electronic device includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method as described in any one of claims 1-5.

Images