Coded data processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a coded data processing method and device, electronic equipment and a storage medium, and belongs to the technical field of data processing.The coded data processing method includes the steps that target line data of a to-be-transcoded conversion file is obtained, and the conversion file comprises coded data in a Chinese character internal code extension standard GBK format; splitting the target row data through separators to obtain a plurality of split fields; independently carrying out decoding processing on the plurality of split fields to obtain a plurality of fields after decoding processing; and connecting the plurality of fields by using the separators to obtain first coded data, performing transcoding processing on the first coded data according to a preset output coding format based on the first coded data, and outputting the transcoded first coded data, so that the processing efficiency of abnormal coded data can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, electronic device and storage medium for processing encoded data. Background Art

[0002] In the field of data management and analysis, it is often necessary to transfer data from one format or storage method to another. In particular, importing character-separated values ​​(CSV) files into a database is one of the common requirements.

[0003] In the related art, when a CSV file containing illegal Chinese Internal Code Specification (GBK) encoded data cannot be directly imported into a database, it is necessary to manually edit and remove the erroneous data, or manually modify the data. The processing of files with large amounts of data requires a lot of manpower and is prone to errors, resulting in low efficiency in processing abnormally encoded data. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a method, device, electronic device and storage medium for processing encoded data, so as to solve the problem of low processing efficiency of abnormal encoded data.

[0005] To solve the above technical problems, the embodiments of the present application are implemented as follows: In a first aspect, an embodiment of the present application provides a method for processing encoded data, comprising: obtaining target row data of a conversion file to be transcoded, the conversion file including encoded data in the Chinese character internal code extension specification GBK format; splitting the target row data by a separator to obtain a plurality of split fields; decoding the plurality of split fields separately to obtain a plurality of decoded fields; connecting the plurality of fields by using the separator to obtain first encoded data, transcoding the first encoded data based on the first encoded data according to a preset output encoding format, and outputting the transcoded first encoded data.

[0006] In a second aspect, an embodiment of the present application provides a device for processing encoded data, comprising: an acquisition module, used to acquire target row data of a conversion file to be transcoded, the conversion file including encoded data in GBK format; a splitting module, used to split the target row data by a delimiter to obtain a plurality of split fields; a decoding module, used to decode the plurality of split fields separately to obtain a plurality of fields after decoding; an output module, used to connect the plurality of fields using the delimiter to obtain first encoded data, transcode the first encoded data according to a preset output encoding format based on the first encoded data, and output the first encoded data after transcoding.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory electrically connected to the processor, the memory storing a computer program, and the processor being used to call and execute the computer program from the memory to implement the above-mentioned method for processing encoded data.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program can be executed by a processor to implement the above-mentioned method for processing encoded data.

[0009] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the above-mentioned method for processing encoded data.

[0010] The technical scheme of the embodiment of the present application is adopted to obtain the target row data of the conversion file to be transcoded, and the conversion file includes the coded data in GBK format; the target row data is split by a separator to obtain multiple split fields; the multiple split fields are decoded separately to obtain multiple fields after decoding; the multiple fields are connected by a separator to obtain first coded data, based on the first coded data, the first coded data is transcoded according to a preset output coding format, and the first coded data after transcoding is output. It can be seen that by obtaining the conversion file to be transcoded containing the coded data in GBK format, the target row data of the conversion file is first split by a separator to obtain split fields, and then the fields in the split fields are decoded separately, which can prevent the separator from being confused due to the lack of missing coded data when the target row data is decoded as a whole, and generate an erroneous decoded file, by connecting the multiple fields after decoding by a separator to obtain the first coded data, and processing the first coded data according to the preset coding format, the conversion file to be transcoded is transcoded without manually modifying the abnormal coding, and the processing efficiency of the abnormal coded data can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate one or more embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in one or more embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0012] Figure 1 is a schematic flow chart of a method for processing coded data according to an embodiment of the present application; Figure 2 is a schematic diagram of a specific process of a conversion file transcoding verification process according to an embodiment of the present application; Figure 3 is a schematic flow chart of a method for processing coded data according to another embodiment of the present application; Figure 4 is a schematic block diagram of a device for processing coded data according to an embodiment of the present application; Figure 5 It is a hardware structure diagram of a coded data processing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0013] Embodiments of the present application provide a method, device, electronic device, and storage medium for processing coded data, so as to solve the problem of low processing efficiency of abnormal coded data.

[0014] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0015] The method for processing coded data provided in the embodiment of the present application can be executed by an electronic device, or by software installed in the electronic device. Specifically, the electronic device can be a terminal device or a server device. Among them, the terminal device can include a smart phone, a laptop computer, a smart wearable device, a vehicle terminal, etc., and the server device can include an independent physical server, a server cluster composed of multiple servers, or a cloud server capable of cloud computing.

[0016] In the following, in conjunction with the accompanying drawings, a method for processing encoded data provided by an embodiment of the present application is described in detail through specific embodiments and their application scenarios.

[0017] Figure 1 A schematic flow chart of a method for processing coded data provided by an embodiment of the present invention is shown, the method comprising the following steps: S102, obtaining target row data of the conversion file to be transcoded.

[0018] The conversion file includes encoding data in GBK format.

[0019] The conversion file to be transcoded includes a file exported from a database, which includes encoding data in GBK format.

[0020] The execution subject of the present application includes an established program, which is used to replace manual processing of conversion files.

[0021] The coded data in GBK format is mainly used to store, transmit and display Chinese characters in computer systems. Specifically, the coded data in GBK format has one-byte and double-byte codes, and the first byte is in the range of 00-7F, which is consistent with the American Standard Code for Information Interchange (ASCII). In a double-byte, the first byte is the first bit of the double-byte. The range of the first byte is 81-FE (that is, excluding 80 and FF), part of the second byte is in the range of 40-7E, and the rest is in the range of 80-FE. Using two bytes to represent a character can represent more characters, including Chinese characters and graphic symbols. The following Table 1 shows an example of the detailed range of the coded data in GBK format:

[0022] Table 1 The target row data is a row in the conversion file, including data encoded in GBK format, and generally includes a row containing abnormally encoded data.

[0023] S104, splitting the target row data by using the separator to obtain a plurality of split fields.

[0024] The delimiter is usually in the form of a comma in the conversion file. The conversion file can be a CSV file, which can correctly separate different fields in the target row data with commas or other special characters (such as ASCII characters).

[0025] According to the target row data after the delimiter splitting process, multiple split fields separated by the delimiter are determined.

[0026] S106, decoding the multiple split fields separately to obtain multiple fields after decoding.

[0027] Specifically, the encoded data in each split field is decoded into corresponding Chinese characters and separators according to the decoding rules of the encoded data.

[0028] As an example, one of the multiple split fields is: d6d0 cec4 d7d6 b7fb b4ae31, which is decoded into Chinese characters: Chinese character string A through the decoding process.

[0029] S108, connecting multiple fields using a separator to obtain first encoded data, based on the first encoded data, transcoding the first encoded data according to a preset output encoding format, and outputting the transcoded first encoded data.

[0030] The preset output encoding format may include data in other formats supported by the database, such as 8-bit (Universal Character Set / Unicode Transformation Format, UTF-8) format, or if the supported format is GBK, the first encoding data is directly imported into the database.

[0031] A delimiter is used to connect multiple fields after the split field decoding process to obtain first encoded data, where the first encoded data is the target row data after complete decoding. The first encoded data is transcoded and output according to a preset output encoding format.

[0032] The technical scheme of the embodiment of the present application is adopted to obtain the target row data of the conversion file to be transcoded, and the conversion file includes the coded data in GBK format; the target row data is split by a separator to obtain multiple split fields; the multiple split fields are decoded separately to obtain multiple fields after decoding; the multiple fields are connected by a separator to obtain first coded data, based on the first coded data, the first coded data is transcoded according to a preset output coding format, and the first coded data after transcoding is output. It can be seen that by obtaining the conversion file to be transcoded containing the coded data in GBK format, the target row data of the conversion file is first split by a separator to obtain split fields, and then the fields in the split fields are decoded separately, which can prevent the separator from being confused due to the lack of missing coded data when the target row data is decoded as a whole, and generate an erroneous decoded file, by connecting the multiple fields after decoding by a separator to obtain the first coded data, and processing the first coded data according to the preset coding format, the conversion file to be transcoded is transcoded without manually modifying the abnormal coding, and the processing efficiency of the abnormal coded data can be improved.

[0033] In one embodiment, to obtain target row data of the conversion file to be transcoded (ie, S102), the following steps A1-A3 may be performed: Step A1: Obtain a conversion file to be transcoded, and read the number of fields of the conversion file.

[0034] The conversion file includes the CSV file exported from the database.

[0035] The number of fields in the conversion file is the number of columns of data in the CSV file. Each column of data is connected by a separator to form the entire CSV file. For example, the following table 2 shows 3 columns of data with 3 fields:

[0036] Table 2 Among them, there are 3 columns in Table 2 representing 3 fields. Separate by delimiter, such as after exporting the CSV file, the CSV file will correctly separate different fields with commas or other ASCII characters, such as: 1, Zhang San, Guangzhou 2. Li Si, Shanghai Step A2: Decode a line of data in the conversion file to obtain a decoded field.

[0037] A line of data can be the first line of the conversion file or any other line, without any specific limitation.

[0038] The decoding process includes converting the encoded data in the conversion file, i.e., a line of data, into Chinese characters of a string. Specifically, following the example in S106, the decoding process is to convert d6d0 cec4 d7d6 b7fb b4ae 31 represented by GBK bytes into "Chinese character string A". When the processing is improper, such as when the second byte of the character "符" is lost, the byte data becomes d6d0 cec4d7d6 b7b4 ae31. When decoding this byte data according to GBK, according to the GBK encoding rule, "b7" of the character "符" and "b4" of the character "串" will be regarded as a character b7b4 and become "反", and the subsequent ae41 forms the character "瓵". It can be seen that correctly encoded GBK Chinese double-byte characters and ASCII single-byte characters can be distinguished according to the rules. However, if the Chinese double-byte characters are not correctly truncated and bytes are lost, it will interfere with and damage the subsequent characters.

[0039] The decoding field includes the field after decoding the encoded data. For example, decoding the second row of data in Table 2 above is shown in Table 3:

[0040] Table 3 Among them, according to the rule that two bytes of the encoded data in GBK represent one character and the rule of ASCII single-byte characters, the three fields determined for this decoding field include: 32, c0ee cbc4, c9cf baa3. It is also possible to decode the data with abnormal encoded data using this decoding method.

[0041] Step A3: When the number of fields in the decoding field is not equal to the number of fields in the conversion file, take a line of data as the target line data of the conversion file to be transcoded.

[0042] Compare according to the decoding field determined in Step A2 and the number of fields in the conversion file. When the number of fields in the decoding field is not equal to the number of fields in the conversion file, take an unprocessed line of data as the target line data of the conversion file.

[0043] As an example, following the examples in Steps A1 and A2, Step A1 can obtain that the number of fields in the conversion file is 3, and the number of fields in the decoding field is also 3, indicating that the number of fields in the decoding field is equal to the number of fields in the conversion file.

[0044] When there is data loss in Step A2, i.e., in Table 3, for example, the second byte c4 of "四" is lost, and decoding this byte is shown in Table 4:

[0045] Table 4 At this time, according to the GBK encoding data that two bytes represent one character and the ASCII single-byte character rule, the decoded field in Table 3 is 2, and the number of converted text fields is 3. Therefore, the number of decoded fields is not equal to the number of fields in the conversion file. Specifically, when the data in Table 4 is exported to a CSV file, the incorrect cbee cb bytes will form a code that is not mapped to a character in the GBK code table with the separator (comma) byte 2c following this field. The code that is not mapped to a character is usually represented by?. The target row data that loses a comma separator will be considered to have only two columns of data and can no longer be imported into the database. That is, the output content of Table 4 is: 1. Zhang San, Guangzhou 2. Li Shanghai It can be seen that when the number of fields of the decoded field is not equal to the number of fields of the conversion file, it is necessary to obtain the target row data and reprocess it, and the target row data includes the data of the conversion file of the GBK encoded data.

[0046] In this embodiment, a conversion file to be transcoded is obtained, and the number of fields in the conversion file is determined, a line of data in the conversion file is obtained, and a line of data is directly decoded. Since a line of data is encoded data in a double-byte GBK format, when the number of decoded fields after decoding is not equal to the number of fields in the conversion file, a line of data that has not been decoded is re-obtained and used as the target line of data. By judging the number of decoded fields after decoding and the number of fields in the conversion file, the target line of data to be obtained can be determined, and whether a line of data after decoding is correct can be automatically judged and the target line of data can be obtained, which can improve the efficiency of checking the conversion file.

[0047] In one embodiment, a line of data in the conversion file is decoded to obtain a decoded field (ie, step A2), and specifically, step A21 may be performed: Step A21: Obtain the encoded data in a line of data of the conversion file, and decode the encoded data in the line of data to obtain a decoded field including a plurality of delimiters.

[0048] Converts the encoded data in a line of data in the file to the encoded data in GBK format.

[0049] The separator includes commonly used symbols in the conversion file CSV file, generally represented by 2c.

[0050] Specifically, for example, the encoded data with normal data is 32 2c c0 ee cb c4 2cc9 cf ba a3, and the corresponding delimiter is 2c. After CBK decoding, the decoded field obtained is: 32c0 ee cb c4c9cf ba a3, including 3 fields. The encoded data with abnormal data is 32 2c c0 ee cb 2cc9 cf ba a3, such as the encoded data in Table 4, and the corresponding delimiter is 2c. After CBK decoding, the decoded field obtained is: 32c0 ee cb2c c9cf ba a3, including 2 fields.

[0051] In this embodiment, a line of data is decoded according to the encoding rules to obtain a decoded field of multiple delimiters, which can determine how many fields are included in the decoded field after a line of data in the conversion file is decoded, thereby facilitating subsequent judgment on whether the line of data is the target line of data.

[0052] In one embodiment, the method may further perform the following step B: Step B: When the number of decoded fields is equal to the number of fields of the conversion file, the decoded fields are transcoded according to a preset output coding format, and the decoded fields after transcoding are output.

[0053] When the number of fields of the decoded field is equal to the number of fields of the conversion file, such as the example in Table 3, the number of fields of the target row data in Table 3 after decoding is equal to the number of fields of the conversion file 3. The decoded field is the data after the target row data is decoded, such as: 2, Li Si, Shanghai, and the decoded field is transcoded and output according to the preset output encoding format, such as converting the decoded field after transcoding into the UTF-8 format adapted to the database and outputting it. It should be noted that at this time, the decoded field can be transcoded and output as the first encoded data.

[0054] In this embodiment, by judging the number of fields of the decoding field and the number of fields of the conversion file, when the number of fields is equal, the decoding field is directly transcoded according to the preset output encoding format, and the decoded field after transcoding is output, which can improve the efficiency of judging abnormal coded data.

[0055] In one embodiment, the target row data is split by a separator to obtain multiple split fields (ie, S104), and the following step C may be performed: Step C: Split the encoded data in the target row data into multiple fields according to the delimiter, and determine the multiple fields as multiple split fields.

[0056] The encoded data in the target row data includes characters corresponding to Chinese characters and symbols for differentiation. The symbol for differentiation is a delimiter. The target row data is split according to the delimiter to obtain multiple split fields.

[0057] As an example, continuing with Table 4, taking the encoded data in Table 4 as the target row data for splitting processing, by scanning and searching for the delimiter symbol value 2c, the target row data can be correctly split into three fields. These three fields include: 32c0eecbc9cfbaa3.

[0058] In this embodiment, by splitting the target row data that has not been decoded first using the delimiter, it is possible to prevent the problem that the delimiter is regarded as a Chinese character byte due to byte loss during the decoding process of the target row data.

[0059] In one embodiment, to separately perform decoding processing on multiple split fields to obtain multiple fields after decoding processing (i.e., S106), the following steps D1 - D2 can be executed: Step D1: Decode each field in the multiple split fields to obtain a field containing a preset number of byte characters.

[0060] Specifically, since one Chinese character in the encoded data in GBK format consists of 2 bytes, but in the case where bytes in the encoded data in the target row data are missing, one Chinese character corresponds to one byte, and this byte cannot be decoded, then it can be represented by?. Therefore, by decoding the split fields, it is also possible to obtain fields in the conversion file where the encoding of Chinese characters corresponds to one or two bytes. It can be seen that the preset number of byte characters can be one or two.

[0061] Specifically, continuing with the example in step C above, the three split fields are: 32c0eecbc9cfbaa3. After decoding each field, the fields containing double - byte characters obtained are: 2Li? Shanghai.

[0062] Step D2: Determine the fields containing the preset number of byte characters corresponding to each field as multiple fields.

[0063] Specifically, continuing with the example in step D1, the multiple field data after splitting Table 4 can be represented as:

[0064] Table 41

[0065] Table 42

[0066] Table 43 It should be noted that the above three fields can also be connected by a separator, that is, the first encoded data is obtained by connecting the separator. Specifically, multiple fields are decoded and connected into a data that meets the specification, such as the GBK encoded data as shown in Table 5 below:

[0067] Table 5 In this embodiment, by decoding each field in the split field, multiple fields including a preset number of byte characters are obtained after decoding, and the multiple fields are connected using a separator to obtain the first encoded data corresponding to the complete target row data, which can avoid decoding errors caused by missing bytes.

[0068] In one embodiment, after the first coded data is transcoded according to a preset output coding format and the transcoded first coded data is output (ie, S108), the following steps E1-E3 may be performed: Step E1: Decode other rows of data in the conversion file to obtain second coded data that matches the number of fields in the conversion file.

[0069] The conversion file is an export file of the database. Each line of the conversion file corresponds to each line of record of the database data. Therefore, the conversion file can include multiple lines of data.

[0070] The other rows of data are the data not corresponding to the output of the first coded data after the target row of data is output and transcoded, and the other rows of data in the conversion file are verified in turn according to the transcoding rules of each row.

[0071] The second coded data that conforms to the number of fields of the conversion file includes the coded data obtained by decoding other line data of the conversion file or decoding after splitting, and the number of fields of the second coded data is consistent with the number of fields of the conversion text. The specific process of decoding other line data and decoding after splitting is consistent with the processing process of the target line data, and will not be described in detail.

[0072] Step E2: transcoding the second coded data according to a preset output coding format, and outputting the transcoded second coded data.

[0073] Step E3: determining a target file of a conversion file to be transcoded according to the first coded data after transcoding and the second coded data after transcoding.

[0074] The first encoded data and the second encoded data include the encoded data of all lines in the converted file after transcoding. After decoding all the encoded data in the converted file to be transcoded, the first encoded data and the second encoded data obtained are transcoded, and the file obtained after transcoding is the target file after the converted file to be transcoded is completely transcoded.

[0075] It should be noted that the specific process of the conversion file transcoding verification process is as follows Figure 2 As shown: start to obtain the conversion file to be transcoded, read a line of data from the conversion file, determine whether the line of data is the end of the conversion file after transcoding, that is, the last line. If yes, then after the transcoding process is completed, the process of continuing to read a line of data from the conversion file is terminated; if it is not the last line, use the GBK rule to decode the line of data, determine whether the number of fields of the decoded field after decoding is consistent with the number of fields in the conversion file, for example, if the conversion file has n columns of data, then the number of fields in the conversion file is n, determine whether the number of fields of the decoded field is equal to n, if the number of fields is equal, then output the line of character string, that is, the decoded field, according to the preset output encoding format, and then process and determine the next line of data in the conversion file; if the number of fields is not equal, then take the line of data as the target line of data, split the line of data, that is, the target line of data, according to the delimiter, obtain multiple split fields after splitting, decode each field in the multiple split fields separately, obtain multiple fields after decoding, and then use the delimiter to connect the multiple fields to obtain the first encoded data, output the first encoded data after transcoding, that is, the character string corresponding to the target line of data, according to the preset output encoding format, and then continue to process and determine the next line of data in the conversion file. The file after the coded data corresponding to the conversion file is completely transcoded is determined as the target file. Generally, the transcoding verification process starts from the first line of the conversion file.

[0076] In this embodiment, by decoding other lines of data of the conversion file, other lines of data that meet the number of fields of the conversion file, i.e., each line of data, are obtained, and each line of data is transcoded in turn according to a preset output encoding format, and each line of data after transcoding is determined as a target file.

[0077] Figure 3 is a schematic flow chart of a method for processing coded data according to another embodiment of the present application, such as Figure 3 As shown, the method comprises the following steps: S301, obtaining a CSV file with GBK format encoded data to be transcoded.

[0078] S302, obtaining the number of fields in the CSV file and reading the first row of data therein.

[0079] S303, decoding the first line of data to obtain a decoding field.

[0080] S304: When the number of decoded fields is equal to the number of fields in the CSV file, the decoded fields are transcoded according to a preset output encoding format, and the decoded fields after transcoding are output.

[0081] S305, when the number of fields of the decoded field is not equal to the number of fields of the CSV file, obtain the first line of data, split the first line of data by using a separator, and obtain a plurality of split fields.

[0082] S306, decoding the multiple split fields separately to obtain multiple decoded fields.

[0083] S307, connect multiple fields using a separator to obtain first encoded data, transcode the first encoded data according to a preset output encoding format and output it.

[0084] S308, verify each remaining row of data in the CSV file in turn, and transcode the verified data to obtain a target file after the CSV file is transcoded.

[0085] The specific process from S301 to S308 has been described in detail in the above embodiment and will not be repeated here.

[0086] The technical scheme of the embodiment of the present application is adopted to obtain the target row data of the conversion file to be transcoded, and the conversion file includes the coded data in GBK format; the target row data is split by a separator to obtain multiple split fields; the multiple split fields are decoded separately to obtain multiple fields after decoding; the multiple fields are connected by a separator to obtain first coded data, based on the first coded data, the first coded data is transcoded according to a preset output coding format, and the first coded data after transcoding is output. It can be seen that by obtaining the conversion file to be transcoded containing the coded data in GBK format, the target row data of the conversion file is first split by a separator to obtain split fields, and then the fields in the split fields are decoded separately, which can prevent the separator from being confused due to the lack of missing coded data when the target row data is decoded as a whole, and generate an erroneous decoded file, by connecting the multiple fields after decoding by a separator to obtain the first coded data, and processing the first coded data according to the preset coding format, the conversion file to be transcoded is transcoded without manually modifying the abnormal coding, and the processing efficiency of the abnormal coded data can be improved.

[0087] In summary, specific embodiments of the present subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recorded in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing can be advantageous.

[0088] The above is a method for processing coded data provided in an embodiment of the present application. Based on the same idea, an embodiment of the present application also provides a device for processing coded data.

[0089] Figure 4 is a schematic diagram of the structure of a device for processing coded data according to an embodiment of the present invention. Figure 4 As shown, the coded data processing device includes: an acquisition module 41, a splitting module 42, a decoding module 43, and an output module 44: An acquisition module 41 is used to acquire target line data of a conversion file to be transcoded, wherein the conversion file includes encoded data in the Chinese character internal code extension specification GBK format; A splitting module 42 is used to split the target row data by using a separator to obtain a plurality of split fields; A decoding module 43, configured to perform decoding processing on the multiple split fields separately to obtain multiple fields after decoding processing; The output module 44 is used to connect multiple fields using a separator to obtain first encoded data, transcode the first encoded data according to a preset output encoding format based on the first encoded data, and output the transcoded first encoded data.

[0090] In one embodiment, the acquisition module 41 further includes: A first acquisition unit is used to acquire the conversion file to be transcoded and read the number of fields in the conversion file, wherein the conversion file includes a character-separated value CSV file exported from a database; A first decoding unit, used for decoding a line of data in the conversion file to obtain a decoding field; The second acquisition unit is used to take a line of data as a target line of data of the conversion file to be transcoded when the number of fields of the decoded field is not equal to the number of fields of the conversion file.

[0091] In one embodiment, the first decoding unit is specifically used to obtain the encoded data in a line of data of the conversion file, and decode the encoded data in the line of data to obtain a decoded field including multiple delimiters.

[0092] In one embodiment, the device further includes a transcoding module, which is used to transcode the decoded fields according to a preset output encoding format when the number of fields of the decoded fields is equal to the number of fields of the conversion file, and output the decoded fields after transcoding.

[0093] In one embodiment, the splitting module 42 is specifically configured to split the encoded data in the target row data into multiple fields according to the separator, and determine the multiple fields as multiple split fields.

[0094] In one embodiment, the decoding module 43 is specifically used to decode each field in the multiple split fields to obtain a field containing a preset number of byte characters; and determine the field containing a preset number of byte characters corresponding to each field as multiple fields.

[0095] In one embodiment, the device further includes a verification module, specifically including: A decoding unit, used for decoding other line data in the conversion file to obtain second coded data that matches the number of fields in the conversion file; a transcoding unit, configured to perform transcoding processing on the second coded data according to a preset output coding format, and output the transcoded second coded data; The determination unit is used to determine a target file of a conversion file to be transcoded according to the first coded data after transcoding and the second coded data after transcoding.

[0096] The technical scheme of the embodiment of the present application is adopted to obtain the target row data of the conversion file to be transcoded, and the conversion file includes the coded data in GBK format; the target row data is split by a separator to obtain multiple split fields; the multiple split fields are decoded separately to obtain multiple fields after decoding; the multiple fields are connected by a separator to obtain first coded data, based on the first coded data, the first coded data is transcoded according to a preset output coding format, and the first coded data after transcoding is output. It can be seen that by obtaining the conversion file to be transcoded containing the coded data in GBK format, the target row data of the conversion file is first split by a separator to obtain split fields, and then the fields in the split fields are decoded separately, which can prevent the separator from being confused due to the lack of missing coded data when the target row data is decoded as a whole, and generate an erroneous decoded file, by connecting the multiple fields after decoding by a separator to obtain the first coded data, and processing the first coded data according to the preset coding format, the conversion file to be transcoded is transcoded without manually modifying the abnormal coding, and the processing efficiency of the abnormal coded data can be improved.

[0097] Those skilled in the art should understand that Figure 4The device for processing encoded data in the embodiment can be used to implement the method for processing encoded data described above, and the detailed description thereof should be similar to that described in the method part above, and will not be further described here to avoid redundancy.

[0098] Based on the same technical concept, an embodiment of the present application further provides an electronic device, which is used to execute the above-mentioned method for processing coded data. Figure 5 A schematic diagram of the structure of an electronic device for implementing various embodiments of the present application. The electronic device may have relatively large differences due to different configurations or performances, and may include a processor (processor) 510, a communication interface (Communications Interface) 520, a memory (memory) 530 and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call a computer program stored in the memory 530 and executable on the processor 510 to perform the following steps: Obtain target row data of a conversion file to be transcoded, wherein the conversion file includes encoding data in the Chinese character internal code extension specification GBK format; The target row data is split by the separator to obtain multiple split fields; Decoding the multiple split fields separately to obtain multiple decoded fields; A plurality of fields are connected by using a separator to obtain first coded data, and based on the first coded data, the first coded data is transcoded according to a preset output coding format, and the transcoded first coded data is output.

[0099] The technical scheme of the embodiment of the present application is adopted to obtain the target row data of the conversion file to be transcoded, and the conversion file includes the coded data in GBK format; the target row data is split by a separator to obtain multiple split fields; the multiple split fields are decoded separately to obtain multiple fields after decoding; the multiple fields are connected by a separator to obtain first coded data, based on the first coded data, the first coded data is transcoded according to a preset output coding format, and the first coded data after transcoding is output. It can be seen that by obtaining the conversion file to be transcoded containing the coded data in GBK format, the target row data of the conversion file is first split by a separator to obtain split fields, and then the fields in the split fields are decoded separately, which can prevent the separator from being confused due to the lack of missing coded data when the target row data is decoded as a whole, and generate an erroneous decoded file, by connecting the multiple fields after decoding by a separator to obtain the first coded data, and processing the first coded data according to the preset coding format, the conversion file to be transcoded is transcoded without manually modifying the abnormal coding, and the processing efficiency of the abnormal coded data can be improved.

[0100] The specific execution steps can refer to the various steps of the above-mentioned embodiment of the method for processing encoded data, and the same technical effect can be achieved. To avoid repetition, they will not be repeated here.

[0101] It should be noted that the electronic devices in the embodiments of the present application include: servers, terminals, or other devices except terminals.

[0102] The above electronic device structure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, the input unit may include a graphics processing unit (GPU) and a microphone, and the display unit may be configured with a display panel in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit includes a touch panel and at least one of other input devices. The touch panel is also called a touch screen. Other input devices may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0103] The memory can be used to store software programs and various data. The memory may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory may include a volatile memory or a non-volatile memory, or the memory may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DRRAM).

[0104] The processor may include one or more processing units; optionally, the processor integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to the operating system, user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor.

[0105] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned embodiment of the method for processing encoded data is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0106] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0107] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned encoding data processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0108] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0109] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the processor is used to run the program or instructions to implement the various processes of the above-mentioned product recommendation method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0110] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0111] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0112] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for processing coded data, characterized in that: The method comprises: Obtain target row data of a conversion file to be transcoded, wherein the conversion file includes encoded data in a Chinese character internal code extension specification GBK format; The target row data is split by a separator to obtain a plurality of split fields; Decoding the multiple split fields separately to obtain multiple decoded fields; The multiple fields are connected by using the delimiter to obtain first encoded data, and based on the first encoded data, the first encoded data is transcoded according to a preset output encoding format, and the transcoded first encoded data is output.

2. The method according to claim 1, characterized in that The step of obtaining target row data of the conversion file to be transcoded includes: Obtain the conversion file to be transcoded, and read the number of fields in the conversion file, wherein the conversion file includes a character-separated value CSV file exported from a database; Decoding a line of data in the conversion file to obtain a decoded field; When the number of fields of the decoded field is not equal to the number of fields of the conversion file, the line of data is used as the target line of data of the conversion file to be transcoded.

3. The method according to claim 2, characterized in that The step of decoding a line of data in the conversion file to obtain a decoded field further includes: The encoded data in the line of data of the conversion file is obtained, and the encoded data in the line of data is decoded to obtain the decoded field including the plurality of delimiters.

4. The method according to claim 2, characterized in that The method further comprises: When the number of fields of the decoded fields is equal to the number of fields of the conversion file, the decoded fields are transcoded according to the preset output encoding format, and the decoded fields after the transcoding are output.

5. The method according to claim 1, characterized in that: The target row data is split by the separator to obtain multiple split fields, including: The encoded data in the target row data is split into a plurality of fields according to a separator, and the plurality of fields are determined as the plurality of split fields.

6. The method according to claim 1, characterized in that The decoding process is performed on the multiple split fields separately to obtain multiple fields after decoding process, including: Decoding each of the multiple split fields to obtain a field containing a preset number of byte characters; The fields corresponding to each field and containing a preset number of byte characters are determined as the multiple fields.

7. The method according to claim 2, characterized in that: After the first coded data is transcoded according to a preset output coding format and the first coded data after the transcoding is output, the method further comprises: Decoding other line data in the conversion file to obtain second coded data that matches the number of fields in the conversion file; Transcoding the second coded data according to the preset output coding format, and outputting the transcoded second coded data; A target file of the conversion file to be transcoded is determined according to the first coded data after the transcoding process and the second coded data after the transcoding process.

8. A device for processing coded data, characterized in that: The device comprises: An acquisition module, used for acquiring target row data of a conversion file to be transcoded, wherein the conversion file includes encoded data in the Chinese character internal code extension specification GBK format; A splitting module, used for splitting the target row data by a separator to obtain a plurality of split fields; A decoding module, used to decode the multiple split fields separately to obtain multiple fields after decoding; The output module is used to connect the multiple fields using the delimiter to obtain first encoded data, based on the first encoded data, transcode the first encoded data according to a preset output encoding format, and output the transcoded first encoded data.

9. An electronic device, characterized in that: It comprises a processor and a memory electrically connected to the processor, the memory stores a computer program, and the processor is used to call and execute the computer program from the memory to implement a method for processing coded data as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The storage medium is used to store a computer program, and the computer program can be executed by a processor to implement a method for processing coded data as described in any one of claims 1-7.

Images