A method and related device for character encoding and decoding

Abstract

An embodiment of the present application discloses a method and related device for character encoding and decoding, which is applied to big data scenarios in cloud technology. The method includes: reading a character string to be processed in a target character string of a first encoding method through a vector reading instruction, where the target character string includes standard code characters and non-standard code characters; if the characters to be processed in the character string to be processed are all standard code characters, performing standard code encoding and decoding processing on the above-mentioned characters to be processed at the same time through the vector encoding and decoding instruction to obtain an encoded and decoded character string of a second encoding method; if the character string to be processed includes non-standard code characters, performing encoding and decoding processing on the characters to be processed in the character string to be processed in sequence through a scalar processing instruction to obtain an encoded and decoded character of the second encoding method; after each scalar processing instruction completes processing, when it is detected that the statistical data of the standard code characters corresponding to the character to be processed meet a preset condition, returning to the execution of the vector reading instruction reading step until the encoding and decoding of the target character string is completed.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a method and related device for character encoding and decoding. Background Art

[0002] With the development of computer technology, characters can generally be encoded using variable-length encoding or fixed-length encoding. Variable-length encoding means that different types of characters are encoded with different lengths, and fixed-length encoding means that each character is encoded with a fixed length.

[0003] Variable-length encodings are used during transmission and storage, allowing for different lengths to avoid wasted space and conserve transmission and storage resources. Fixed-length encodings are used during computations, allowing for consistent processing logic, simplifying computation and ensuring efficiency. Therefore, converting between these two encodings—character encoding and decoding—is crucial. Related technologies can implement character encoding and decoding through vector-based processing methods, improving efficiency.

[0004] However, in some big data business scenarios, a lot of data is actually a string of mixed standard code characters and non-standard code characters. The above-mentioned vector instruction-based processing method is used to encode and decode the above-mentioned strings, where non-standard code characters are difficult to encode and decode. That is, the above-mentioned vector instruction-based processing method is not suitable for encoding and decoding the above-mentioned strings, resulting in low encoding and decoding processing efficiency. Summary of the Invention

[0005] In order to solve the above technical problems, the present application provides a method and related devices for character encoding and decoding, which can realize fast encoding and decoding of standard code characters and simple encoding and decoding of non-standard code characters, and switch from simple encoding and decoding to fast encoding and decoding in a timely and effective manner. It is more suitable for encoding and decoding character strings of standard code characters mixed with non-standard code characters, thereby improving the processing efficiency of encoding and decoding.

[0006] The embodiments of this application disclose the following technical solutions:

[0007] In one aspect, the present application provides a method for character encoding and decoding, the method comprising:

[0008] Reading a character string to be processed in a target character string of a first encoding method based on a vector read instruction; the target character string includes standard code characters and non-standard code characters, and the standard code characters are single-byte characters that meet a preset encoding standard;

[0009] If all the characters to be processed in the character string to be processed are the standard code characters, performing standard code encoding and decoding processing on the characters to be processed in the character string to be processed simultaneously based on the vector encoding and decoding instruction to obtain an encoded and decoded character string corresponding to the second encoding mode of the character string to be processed;

[0010] If the character string to be processed includes the non-standard code character, encoding and decoding the characters to be processed in the character string to be processed in sequence based on the scalar processing instruction to obtain the encoded and decoded characters corresponding to the second encoding method of the characters to be processed;

[0011] After obtaining the encoded and decoded character of the second encoding method corresponding to the character to be processed each time, if the standard code character statistics corresponding to the character to be processed meet the preset conditions, return to continue executing the vector-based read instruction to read the character string to be processed in the target string of the first encoding method until the encoded and decoded character string of the second encoding method corresponding to the target character string is obtained.

[0012] On the other hand, the present application provides a device for character encoding and decoding, the device comprising: a reading unit, a first encoding and decoding unit, a second encoding and decoding unit, and a returning unit;

[0013] The reading unit is configured to read a character string to be processed in a target character string in a first encoding mode based on a vector read instruction; the target character string includes standard code characters and non-standard code characters, and the standard code characters are single-byte characters that meet a preset encoding standard;

[0014] The first encoding and decoding unit is configured to, if all the characters to be processed in the character string to be processed are the standard code characters, simultaneously perform standard code encoding and decoding processing on the characters to be processed in the character string to be processed based on the vector encoding and decoding instruction, to obtain a codec character string corresponding to the second encoding mode for the character string to be processed;

[0015] The second encoding and decoding unit is configured to, if the character string to be processed includes the non-standard code character, sequentially encode and decode the characters to be processed in the character string to be processed based on the scalar processing instruction to obtain the encoded and decoded characters corresponding to the second encoding method of the characters to be processed;

[0016] The return unit is used to return to continue executing the vector-based read instruction to read the character string to be processed in the target character string of the first encoding method after obtaining the encoding and decoding character corresponding to the second encoding method of the character to be processed each time, if the standard code character statistics corresponding to the character to be processed meet the preset conditions, until the encoding and decoding character string corresponding to the target character string of the second encoding method is obtained.

[0017] On the other hand, the present application provides a computer device for character encoding and decoding, the computer device comprising a processor and a memory:

[0018] The memory is used to store program code and transmit the program code to the processor;

[0019] The processor is configured to execute the character encoding and decoding method described in the above aspect according to the instructions in the program code.

[0020] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which is used to store a computer program, and the computer program is used to execute the character encoding and decoding method described in the above aspects.

[0021] On the other hand, an embodiment of the present application provides a computer program product, which includes a computer program or instructions; when the computer program or instructions are executed by a processor, the character encoding and decoding method described in the above aspects is executed.

[0022] It can be seen from the above technical solution that the character string to be processed in the target character string of the first encoding method is read by the vector read instruction, and the target character string has standard code characters and non-standard code characters, and the standard code characters are single-byte characters that meet the preset encoding standards; when the characters to be processed in the character string to be processed are all standard code characters, the vector encoding and decoding instructions are used to simultaneously perform standard code encoding and decoding on the characters to be processed in the character string to be processed, and obtain the encoded and decoded character string corresponding to the second encoding method of the character string to be processed; when the character string to be processed includes non-standard code characters, the scalar processing instructions are used to perform encoding and decoding on the characters to be processed in sequence, and obtain the encoded and decoded character string corresponding to the second encoding method of the character to be processed; after each scalar processing instruction completes processing, when it is detected that the statistical data of the standard code characters corresponding to the characters to be processed meet the preset conditions, it returns to continue executing the vector read instruction reading step until the encoded and decoded character string corresponding to the second encoding method of the target character string is obtained.

[0023] It can be seen that in the case where the target string of the first encoding method has standard code characters and non-standard code characters, after the to-be-processed string is read by the vector read instruction, the to-be-processed string that only includes standard code characters is processed by the vector encoding and decoding instruction, thereby realizing batch and fast encoding and decoding of standard code characters corresponding to the second encoding method; the to-be-processed string including non-standard code characters is processed by the scalar processing instruction, thereby realizing simple and convenient encoding and decoding of standard code characters and non-standard code characters corresponding to the second encoding method; and after each scalar processing instruction completes processing, the statistical data of the standard code characters meets the preset conditions, and the processing is switched from the scalar processing instruction to the vector read instruction in a timely and effective manner. Based on this, this method is more suitable for encoding and decoding strings of mixed standard code characters and non-standard code characters, thereby improving the processing efficiency of encoding and decoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0025] Figure 1 A schematic diagram of a text, a UTF-8 string corresponding to the text, and a UTF-16 string corresponding to the text provided in an embodiment of the present application;

[0026] Figure 2 A schematic diagram of an application scenario of a character encoding and decoding method provided in an embodiment of the present application;

[0027] Figure 3 A flowchart of a character encoding and decoding method provided in an embodiment of the present application;

[0028] Figure 4 A schematic diagram of a process for determining that all characters to be processed in a character string to be processed are standard code characters provided in an embodiment of the present application;

[0029] Figure 5 A schematic diagram of a preset character string and a character string to be processed provided in an embodiment of the present application;

[0030] Figure 6 A schematic diagram of a process for determining whether a character string to be processed includes non-standard code characters provided in an embodiment of the present application;

[0031] Figure 7 A schematic diagram of a flow chart of encoding and decoding characters to be processed in a character string to be processed in sequence based on scalar processing instructions provided in an embodiment of the present application;

[0032] Figure 8 A schematic diagram of a process for switching from scalar instruction processing to vector instruction reading provided in an embodiment of the present application;

[0033] Figure 9 A schematic diagram of the position of the last non-standard code character in a character string to be processed provided in an embodiment of the present application;

[0034] Figure 10 A processing method based on scalar instructions, a processing method based on vector instructions, and an effect diagram of character encoding and decoding implemented by an embodiment of the present application are provided;

[0035] Figure 11 A schematic diagram of a character encoding and decoding device provided in an embodiment of the present application;

[0036] Figure 12 A schematic diagram of the structure of a server provided in an embodiment of the present application;

[0037] Figure 13 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. Specific implementation method

[0038] The embodiments of the present application are described below with reference to the accompanying drawings.

[0039] Currently, character encoding and decoding (CDE) is crucial for converting between variable-length and fixed-length encodings. For example, in some big data scenarios, much data is encoded in the variable-length UTF-8 encoding. When performing operations like string segmentation and searching, these UTF-8 strings need to be converted to fixed-length UTF-16 strings. UTF-8 is a variable-length encoding for Unicode, typically using 1-4 bytes to encode different character types. UTF-16 is a fixed-length encoding for Unicode, typically using 2 bytes to encode each character.

[0040] See also Figure 1 , this figure is a schematic diagram of a text, a string corresponding to UTF-8 of the text, and a string corresponding to UTF-16 of the text provided in an embodiment of the present application; among them, some characters in the UTF-8 string are encoded using 1 byte, some characters are encoded using 2 bytes, some characters are encoded using 3 bytes, etc., and each character in the UTF-16 string is encoded using 2 bytes.

[0041] In related technologies, character strings can be encoded and decoded using vector instruction-based processing methods to improve encoding and decoding efficiency. For example, using some common vector instruction-based encoding and decoding algorithms or vector instruction-based languages ​​such as Java to process UTF-8 strings, UTF-8 strings can be decoded into UTF-16 strings. Java is an object-oriented programming language that is both powerful and easy to use.

[0042] However, in the above-mentioned big data business scenarios, a lot of data is actually a string of mixed standard code characters and non-standard code characters, for example, a string of mixed ASCII characters and non-ASCII characters, where ASCII is a universal standard code for information exchange, and ASCII characters are encoded using 1 byte. When encoding and decoding the above-mentioned strings using some of the above-mentioned general encoding and decoding algorithms based on vector instructions, the complexity is high or the overhead is too large when encoding and decoding the non-standard code words such as ASCII characters in the above-mentioned strings, and the vector instructions have poor scalability at the byte level; when encoding and decoding the above-mentioned strings using languages ​​such as Java based on vector instructions, the non-standard code characters such as ASCII characters in the above-mentioned strings cannot be encoded and decoded. Therefore, the above-mentioned processing method based on vector instructions is not suitable for encoding and decoding the above-mentioned strings, resulting in low encoding and decoding processing efficiency.

[0043] In view of this, the present application proposes a method and related device for character encoding and decoding, in which, when the target character string of the first encoding method has standard code characters and non-standard code characters, after reading the character string to be processed by the vector read instruction, the character string to be processed that only includes standard code characters is processed by the vector encoding and decoding instruction, thereby realizing batch and fast encoding and decoding of standard code characters corresponding to the second encoding method; the character string to be processed including non-standard code characters is processed by the scalar processing instruction, thereby realizing simple and convenient encoding and decoding of standard code characters and non-standard code characters corresponding to the second encoding method; and after each scalar processing instruction completes processing, the statistical data of the standard code characters meets the preset conditions, and the processing is switched from the scalar processing instruction to the vector read instruction in a timely and effective manner. Based on this, this method is more suitable for encoding and decoding character strings of mixed standard code characters and non-standard code characters, thereby improving the processing efficiency of encoding and decoding.

[0044] The character encoding and decoding method provided in the embodiment of the present application can be applied to devices that can be applied to character encoding and decoding with data processing capabilities, such as terminal devices, servers, etc. Among them, terminal devices include but are not limited to mobile phones, computers, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, etc.; servers can be independent physical servers, or server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide cloud computing services, but are not limited to these. Terminal devices and servers can be directly or indirectly connected through wired or wireless communication, and this application does not limit this. In addition, the character encoding and decoding method of the embodiment of the present invention can be applied to various scenarios, including but not limited to cloud technology, etc.

[0045] Big data, as used in big data business scenarios, refers to data sets that cannot be captured, managed, and processed within a specific timeframe using conventional software tools. These are massive, high-growth, and diverse information assets that require new processing models to enhance decision-making, insight discovery, and process optimization. With the advent of the cloud era, big data has attracted increasing attention, requiring specialized technologies to effectively process large amounts of time-sensitive data. Technologies suitable for big data include massively parallel processing databases, data mining, distributed file systems, distributed databases, cloud computing platforms, the internet, and scalable storage systems.

[0046] In order to facilitate understanding of the technical solution of the present application, the character encoding and decoding method provided in the embodiment of the present application is introduced below in combination with actual application scenarios.

[0047] See also Figure 2 , which is a schematic diagram of an application scenario of a character encoding and decoding method provided by an embodiment of the present application. Figure 1 The application scenario shown includes a server 200 as a character encoding and decoding device.

[0048] Server 200 reads a character string to be processed from a target character string in a first encoding method based on a vector read instruction. The target character string includes standard code characters and non-standard code characters, where standard code characters are single-byte characters that meet a preset encoding standard. For example, if the first encoding method is UTF-8 and the standard code characters are ASCII characters, then the UTF-8 target character string includes ASCII characters and non-ASCII characters. Server 200 can read the character string to be processed from the UTF-8 target character string using the vector read instruction.

[0049] If all the characters to be processed in the string to be processed are standard code characters, server 200 simultaneously performs standard code encoding and decoding on the characters to be processed in the string to be processed based on the vector encoding and decoding instructions, obtaining an encoding and decoding string corresponding to the second encoding method of the string to be processed. For example, if the second encoding method is UTF-16, then after the above steps, server 200 can determine that all the characters to be processed in the string to be processed are ASCII characters. If so, server 200 simultaneously performs ASCII encoding and decoding on the ASCII characters in the string to be processed based on the vector encoding and decoding instructions, obtaining an encoding and decoding string corresponding to the UTF-16 encoding and decoding of the string to be processed.

[0050] If the string to be processed includes non-standard coded characters, server 200 performs encoding and decoding processing on the characters to be processed in the string to be processed based on the scalar processing instructions, and obtains the encoded and decoded characters corresponding to the second encoding method of the characters to be processed. For example, after the above steps, server 200 can determine whether the string to be processed includes non-ASCII characters. If so, server 200 performs encoding and decoding processing on the characters to be processed in the string to be processed based on the scalar processing instructions, and obtains the encoded and decoded characters corresponding to the characters to be processed in UTF-16.

[0051] After obtaining the encoded and decoded characters of the second encoding method corresponding to the character to be processed each time, if the statistical data of the standard code characters corresponding to the character to be processed meets the preset conditions, the server 200 returns to continue executing the vector read instruction to read the character string to be processed in the target character string of the first encoding method until the encoded and decoded character string of the target character string corresponding to the second encoding method is obtained. For example, after the server 200 obtains the encoded and decoded characters of the UTF-16 corresponding to the character to be processed each time, the server 200 can determine whether the statistical data of the ASCII characters corresponding to the character to be processed meets the preset conditions. If so, the server 200 can return to continue executing the vector read instruction to read the character string to be processed in the UTF-8 target character string until the encoded and decoded character string of the target character string corresponding to the UTF-16 is obtained.

[0052] It can be seen that when the target string of the first encoding method has standard code characters and non-standard code characters, after the server 200 reads the to-be-processed string through the vector read instruction, the server 200 processes the to-be-processed string that only includes standard code characters through the vector encoding and decoding instruction, thereby realizing batch and fast encoding and decoding of standard code characters corresponding to the second encoding method; the server 200 processes the to-be-processed string including non-standard code characters through the scalar processing instruction, thereby realizing simple and convenient encoding and decoding of standard code characters and non-standard code characters corresponding to the second encoding method; and after each scalar processing instruction completes processing, the statistical data of the standard code characters meets the preset conditions, and the server 200 switches from the scalar processing instruction processing to the vector read instruction reading in a timely and effective manner. Based on this, this method is more suitable for encoding and decoding strings of mixed standard code characters and non-standard code characters, thereby improving the processing efficiency of encoding and decoding.

[0053] Next, the character encoding and decoding method provided in the embodiment of the present application is specifically introduced using the server as the character encoding and decoding device.

[0054] See also Figure 2 , which is a flow chart of a character encoding and decoding method provided by an embodiment of the present application. Figure 2 As shown, the character encoding and decoding method is executed by a terminal device and includes the following steps:

[0055] S301: Reading a character string to be processed in a target character string of a first encoding method based on a vector read instruction; the target character string includes standard code characters and non-standard code characters, and the standard code characters are single-byte characters that meet a preset encoding standard.

[0056] Since encoding and decoding strings through some vector instruction-based processing methods can improve the processing efficiency of encoding and decoding; and in some big data business scenarios, such as the data pattern shown in Table 1 below, many data are actually strings of mixed standard code characters and non-standard code characters; therefore, in an embodiment of the present application, for the target string of the first encoding method to be encoded and decoded, the target string includes standard code characters and non-standard code characters, and the standard code characters are single-byte characters that meet the preset encoding standards; first consider encoding and decoding the string based on the vector instruction processing method, and first need to read the character string to be processed in the target string through the vector read instruction.

[0057] Table 1 Data model

[0058] Server 1 Server 2 Number of standard code strings 261，638,189,160 173,902,229,090 Total length of characters 11,158,881,418,519 1,820,736,171,200 Number of non-standard code strings 12,523,473.392 8,377,405,394 Number of single-byte characters 18,301,513,388,867 196,963,363,879 2-byte character count 1,076,017,683 45,054,505 3-byte character count 438,180,072,848 84,196,510,232 4-byte character count 12,918,135 771,721

[0059] As an example, when the first encoding method is UTF-8 and the standard code characters are ASCII characters, that is, the UTF-8 target string includes ASCII characters and non-ASCII characters, a vector read instruction is used to read the string to be processed in the UTF-8 target string. Based on this, the string to be processed can be an ASCII string or a non-ASCII string. When the string to be processed is an ASCII string, it means that the characters to be processed in the string to be processed are all ASCII characters; when the string to be processed is a non-ASCII string, it means that the string to be processed includes non-ASCII characters.

[0060] S302: If all the characters to be processed in the character string to be processed are standard code characters, standard code encoding and decoding are performed on the characters to be processed in the character string to be processed simultaneously based on the vector encoding and decoding instruction to obtain an encoded and decoded character string corresponding to the second encoding method of the character string to be processed.

[0061] Because some common encoding and decoding algorithms based on vector instructions in the related art are highly complex or have excessive overhead when encoding and decoding non-standard code words in a string, and the vector instructions have poor scalability at the byte level; languages ​​such as Java based on vector instructions cannot encode and decode non-standard code words in a string; that is, the processing method based on vector instructions is not suitable for encoding and decoding the above-mentioned string, resulting in low encoding and decoding processing efficiency. Therefore, in the embodiment of the present application, after reading the string to be processed in S301, it is necessary to determine whether the characters to be processed in the string to be processed are all standard code characters. If so, it means that the string to be processed only includes standard code characters; then, the characters to be processed in the string to be processed can be simultaneously subjected to standard code encoding and decoding processing through vector encoding and decoding instructions to obtain an encoding and decoding string corresponding to the second encoding method of the string to be processed, and the encoding and decoding efficiency of the standard code characters can be accelerated in parallel to achieve batch and fast encoding and decoding of the standard code characters corresponding to the second encoding method.

[0062] Among them, on the basis that the standard code character is a single-byte character that meets the preset encoding standard and the highest bit of the standard code character is 0, considering that the standard code character and a single-byte character whose highest bit is 1 and the remaining bits are 0 are ANDed together, the result is 0; based on this, in order to determine whether the characters to be processed in the character string to be processed are all standard code characters, a character string can be pre-set as a preset character string, in which the highest bit of each byte in the preset character string is 1 and the remaining bits are 0, and the number of bytes of the preset character string and the preset character string is the same; then refer to Figure 4This figure is a flow chart of a method for determining that all characters to be processed in a string to be processed are standard code characters, provided by an embodiment of the present application. An AND operation is performed on the string to be processed and a preset string to obtain an operation result. The operation result is determined to be 0. If so, it is determined that all characters to be processed in the string to be processed are standard code characters. Therefore, the present application provides a possible implementation method, and the method for determining that all characters to be processed in a string to be processed are standard code characters includes the following steps S1-S2:

[0063] S1: Perform an AND operation on the character string to be processed and the preset character string to obtain an operation result; the highest bit of each byte in the preset character string is 1 and the remaining bits are 0, and the number of bytes in the character string to be processed and the preset character string is the same.

[0064] S2: If the operation result is 0, it is determined that all the characters to be processed in the character string to be processed are standard code characters.

[0065] As an example, based on the above example, the character string to be processed is subjected to an AND operation with a preset character string having the same number of bytes, with the highest bit of each byte being 1 and the remaining bits being 0, to obtain an operation result; when the operation result is 0, it is determined that the characters to be processed in the character string to be processed are all ASCII characters, that is, the character string to be processed is an ASCII string. Figure 5 , this figure is a schematic diagram of a preset string and a string to be processed provided in an embodiment of the present application; wherein, if the result of the AND operation between the string to be processed A and the preset string is 0, then the characters to be processed in the string to be processed A are all ASCII characters, that is, the string to be processed A is an ASCII string.

[0066] Among them, when S302 is specifically implemented, the vector encoding and decoding instructions may include, for example, vector unpacking instructions and vector packing instructions. Then, the vector unpacking instructions and vector packing instructions can be used to simultaneously perform standard code encoding and decoding processing on the characters to be processed in the character string to be processed to obtain the encoded and decoded character string corresponding to the second encoding method of the character string to be processed.

[0067] As an example, when the second encoding method is UTF-16, the ASCII characters in the string to be processed are simultaneously ASCII encoded and decoded through vector unpacking instructions and vector packing instructions. Combined with the conversion of ASCII characters between UTF-8 and UTF-16 as shown in Table 2 below, before using the ASCII character encoded by 1 byte, add 1 byte with all bits being 0, expand the ASCII character encoded by 1 byte to ASCII character encoded by 2 bytes, and obtain the UTF-16 encoded and decoded string corresponding to the string to be processed.

[0068] Table 2 Conversion of ASCII characters between UTF-8 and UTF-16

[0069] UTF-8 UTF-16 ASCII characters 0aaaaaaa 00000000 0aaaaaaa

[0070] S303: If the character string to be processed includes non-standard code characters, encoding and decoding the characters to be processed in the character string to be processed are performed in sequence based on the scalar processing instruction to obtain encoded and decoded characters corresponding to the second encoding method of the characters to be processed.

[0071] In addition to the situation in which all the characters to be processed in the above-mentioned S302 character string to be processed are standard code characters, there is also a situation in which the character string to be processed includes non-standard code characters. At this time, the processing method based on vector instructions is not suitable for encoding and decoding the character string to be processed; it is necessary to replace the vector encoding and decoding instructions with scalar processing instructions, and encode and decode the characters to be processed in the character string to be processed in turn to obtain the encoded and decoded characters corresponding to the second encoding method of the characters to be processed, simplifying the encoding and decoding difficulty of non-standard code characters, so as to realize simple and convenient encoding and decoding of standard code characters and non-standard code characters corresponding to the second encoding method.

[0072] Among them, corresponding to Figure 4 , see Figure 6 This figure is a schematic diagram of a process for determining whether a character string to be processed includes non-standard code characters, provided in an embodiment of the present application. An AND operation is performed on the character string to be processed and a preset character string to obtain an operation result. A determination is made as to whether the operation result is 0. If not, the character string to be processed is determined to include non-standard code characters. Therefore, the present application provides a possible implementation method, wherein the method for determining whether a character string to be processed includes non-standard code characters includes S3: If the operation result is not 0, determining that the character string to be processed includes non-standard code characters.

[0073] As an example, corresponding to the above example, the string to be processed is subjected to an AND operation with a preset string having the same number of bytes, with the highest bit of each byte being 1 and the remaining bits being 0, to obtain an operation result; if the operation result is not 0, it is determined that the string to be processed includes non-ASCII characters, that is, the string to be processed is a non-ASCII string. Figure 5 In the example, if the result of the AND operation between the character string to be processed B and the preset character string is not 0, the character string to be processed B includes non-ASCII characters, that is, the character string to be processed B is a non-ASCII character string.

[0074] In the specific implementation of step 303, in order to implement encoding and decoding of the character string to be processed, the scalar processing instruction needs to include a scalar read instruction and a scalar encoding and decoding instruction, wherein the scalar read instruction is used to sequentially read the characters to be processed in the character string to be processed, and the scalar encoding and decoding instruction is used to sequentially encode and decode the read characters to be processed to obtain the encoded and decoded characters corresponding to the second encoding method of the characters to be processed. Therefore, the present application provides a possible implementation method, in which the scalar processing instruction includes a scalar read instruction and a scalar encoding and decoding instruction. S303 may, for example, include the following S3031-S3032:

[0075] S3031: Read the characters to be processed in the character string to be processed in sequence based on the scalar read instruction.

[0076] S3032: Perform encoding and decoding processing on the characters to be processed in sequence based on the scalar encoding and decoding instructions to obtain encoded and decoded characters corresponding to the second encoding method of the characters to be processed.

[0077] Among them, when S3032 is specifically implemented, on the basis that the character string to be processed includes non-standard code characters, the characters to be processed in the read character string to be processed can be standard code characters or non-standard code characters. The encoding and decoding processing methods of standard code characters and non-standard code characters are different. When the characters to be processed are standard code characters, it is necessary to perform standard code encoding and decoding processing on the characters to be processed through scalar encoding and decoding instructions to obtain the encoding and decoding characters of the second encoding method corresponding to the characters to be processed; and when the characters to be processed are non-standard code characters, it is necessary to perform non-standard code encoding and decoding processing on the characters to be processed through scalar encoding and decoding instructions to obtain the encoding and decoding characters of the second encoding method corresponding to the characters to be processed. Therefore, the present application provides a possible implementation method, and S3032 may, for example, include the following S4-S5:

[0078] S4: If the character to be processed is a standard code character, perform standard code encoding and decoding processing on the character to be processed based on the scalar encoding and decoding instruction to obtain a codec character corresponding to the second encoding method of the character to be processed.

[0079] S5: If the character to be processed is a non-standard code character, perform non-standard code encoding and decoding processing on the character to be processed based on the scalar encoding and decoding instruction to obtain an encoded and decoded character corresponding to the second encoding method of the character to be processed.

[0080] Among them, when S5 is specifically implemented, if the character to be processed is a non-standard code character, first, it is necessary to determine whether the character to be processed is a non-standard code character that meets the first encoding method. If so, it means that the character to be processed read by S3031 is not an abnormal encoding, and the non-standard code encoding and decoding processing can be performed on the character to be processed through the scalar encoding and decoding instruction to obtain the encoding and decoding result corresponding to the character to be processed; then, it is also necessary to determine whether the encoding and decoding result meets the second encoding method. If so, it means that the non-standard code character that meets the first encoding method is successfully encoded and decoded into a codec character that meets the second encoding method, and the encoding and decoding result is. Therefore, the present application provides a possible implementation method, and S5 can, for example, include the following S51-S52:

[0081] S51: If the character to be processed is a non-standard code character that satisfies the first encoding method, perform non-standard code encoding and decoding processing on the character to be processed based on the scalar encoding and decoding instruction to obtain an encoding and decoding result corresponding to the character to be processed.

[0082] Among them, determining whether the character to be processed is a non-standard code character that meets the first encoding method can, for example, include determining whether the character to be processed can read the corresponding byte character, and whether the highest bit of each byte character meets the setting of the first encoding method.

[0083] S52: If the encoding and decoding result satisfies the second encoding method, the encoding and decoding result is determined as the encoding and decoding character of the character to be processed corresponding to the second encoding method.

[0084] As an example, based on the above example, see Figure 7 , this figure is a flow chart of encoding and decoding the characters to be processed in the character string to be processed in sequence based on scalar processing instructions provided by an embodiment of the present application; when the second encoding method is UTF-16, the characters to be processed in the character string to be processed are read in sequence by scalar reading instructions to determine whether the characters to be processed are ASCII characters. If so, ASCII encoding and decoding are performed on the characters to be processed based on the scalar encoding and decoding instructions to obtain the UTF-16 encoding and decoding characters corresponding to the characters to be processed; if not, when the characters to be processed are non-ASCII characters, non-ASCII encoding and decoding are performed on the characters to be processed through scalar encoding and decoding instructions to obtain the UTF-16 encoding and decoding characters corresponding to the characters to be processed.

[0085] Among them, when the character to be processed is a non-ASCII character, the non-ASCII encoding and decoding processing is performed on the character to be processed through the scalar encoding and decoding instruction to obtain the UTF-16 encoding and decoding character corresponding to the character to be processed. Specifically, it can be: determine whether the character to be processed is a 2-byte character. If so, perform 2-byte encoding and decoding processing on the character to be processed through the scalar encoding and decoding instruction to obtain the UTF-16 encoding and decoding character corresponding to the character to be processed; if not, determine whether the character to be processed is a 3-byte character. If so, perform 3-byte encoding and decoding processing on the character to be processed through the scalar encoding and decoding instruction to obtain the UTF-16 encoding and decoding character corresponding to the character to be processed; if not, determine whether the character to be processed is a 4-byte character. If so, perform 4-byte encoding and decoding processing on the character to be processed through the scalar encoding and decoding instruction to obtain the UTF-16 encoding and decoding character corresponding to the character to be processed; if not, determine that the encoding and decoding is abnormal.

[0086] In practical applications, the scalar encoding and decoding instructions are used to perform 2-byte encoding and decoding on the characters to be processed to obtain the UTF-16 encoding and decoding characters corresponding to the characters to be processed. Specifically, it can be as follows: Combine the conversion between UTF-8 and UTF-16 of multiple byte characters shown in Table 3 below; If the character to be processed is a 2-byte character that meets UTF-8, perform 2-byte encoding and decoding on the character to be processed based on the scalar encoding and decoding instructions to obtain the encoding and decoding result corresponding to the character to be processed; If the encoding and decoding result meets UTF-16, the character to be processed corresponds to the UTF-16 encoding and decoding character; If the character to be processed is a 2-byte character that meets UTF-8 ... If the character to be processed is a 3-byte character that satisfies UTF-8, 3-byte encoding and decoding processing is performed on the character to be processed based on the scalar encoding and decoding instruction to obtain the encoding and decoding result corresponding to the character to be processed; if the encoding and decoding result satisfies UTF-16, the character to be processed corresponds to the encoding and decoding character of UTF-16; if the character to be processed is a 4-byte character that satisfies UTF-8, 4-byte encoding and decoding processing is performed on the character to be processed based on the scalar encoding and decoding instruction to obtain the encoding and decoding result corresponding to the character to be processed; if the encoding and decoding result satisfies UTF-16, the character to be processed corresponds to the encoding and decoding character of UTF-16.

[0087] Table 3 Conversion between UTF-8 and UTF-16 for multiple byte characters

[0088]

[0089] S304: After obtaining the encoded and decoded characters of the second encoding method corresponding to the character to be processed each time, if the statistical data of the standard code characters corresponding to the character to be processed meets the preset conditions, return to continue executing the vector-based read instruction to read the character string to be processed in the target character string of the first encoding method until the encoded and decoded character string of the second encoding method corresponding to the target character string is obtained.

[0090] In the embodiment of the present application, although it is relatively simple and convenient to encode and decode non-standard code characters through scalar processing instructions, it is still faster to encode and decode standard code characters through vector reading instructions and vector encoding and decoding instructions; therefore, each time after obtaining the encoding and decoding characters of the second encoding method corresponding to the character to be processed in S303, it can be determined whether the statistical data of the standard code characters corresponding to the character to be processed meet the preset conditions. If so, it indicates that it is necessary to switch from scalar processing instructions to vector reading instructions in a timely and effective manner, and then return to continue executing the target character string to be processed in the first encoding method based on the vector reading instructions until the encoding and decoding string of the target character string corresponding to the second encoding method is obtained. This method is more suitable for encoding and decoding a character string that is a mixture of standard code characters and non-standard code characters, thereby improving the processing efficiency of encoding and decoding.

[0091] Among them, it is taken into account that before the last non-standard code character in the string to be processed is encoded and decoded by the scalar processing instruction and the encoded and decoded character corresponding to the second encoding method of the last non-standard code character is obtained, it is necessary to avoid performance loss caused by switching from the scalar processing instruction processing to the vector reading instruction reading; it is also taken into account that the standard code characters are continuously encoded and decoded by the scalar processing instruction and the encoded and decoded characters corresponding to the second encoding method of the standard code characters are continuously obtained, it is necessary to switch from the scalar processing instruction processing to the vector reading instruction reading in a timely and effective manner; the position of the character to be processed is after the position of the last non-standard code character in the string to be processed, and the number of consecutive standard code characters corresponding to the character to be processed matches the preset number as a preset condition, and the statistical data of the standard code characters corresponding to the character to be processed meets the above preset condition, and returns to continue executing S301. Therefore, the present application provides a possible implementation method, and the preset condition includes that the position of the character to be processed is after the position of the last non-standard code character in the string to be processed, and the number of consecutive standard code characters corresponding to the character to be processed matches the preset number.

[0092] As an example, see Figure 8 , which is a flow chart of switching from scalar processing instruction processing to vector reading instruction reading provided by an embodiment of the present application. Figure 7 After obtaining the UTF-16 encoding and decoding character corresponding to the character to be processed each time, if the ASCII character statistics corresponding to the character to be processed satisfy that the position of the character to be processed is after the position of the last non-standard code character in the character string to be processed, and the number of consecutive ASCII characters corresponding to the character to be processed matches the preset number, return to continue executing the vector-based read instruction to read the character string to be processed in the UTF-8 target string until the UTF-16 encoding and decoding string corresponding to the target string is obtained.

[0093] Among them, see Figure 9 This figure is a schematic diagram of the position of the last non-standard code character in a string to be processed, provided in an embodiment of the present application. The string to be processed, C, includes both ASCII and non-ASCII characters. The starting position of the string to be processed, C, is n, and the position of the last non-ASCII character in the string to be processed is n+8.

[0094] Among them, when the preset number is 2, the continuous number of ASCII characters corresponding to the character to be processed matches 2. It can be that each time the UTF-16 encoding and decoding character corresponding to the character to be processed is obtained, the character to be processed is an ASCII character, and the continuous number of ASCII characters corresponding to the character to be processed is increased by 1. The character to be processed is a non-ASCII character, and the continuous number of ASCII characters corresponding to the character to be processed is reset to 0, and so on, until the continuous number of ASCII characters corresponding to the character to be processed is 2.

[0095] Alternatively, the number of consecutive ASCII characters corresponding to the character to be processed matches 2. This can be because each time the UTF-16 encoding and decoding character corresponding to the character to be processed is obtained, if the character to be processed is an ASCII character, the number of consecutive ASCII characters corresponding to the character to be processed is reduced by 1; if the character to be processed is a non-ASCII character, the number of consecutive ASCII characters corresponding to the character to be processed is reset to 2, and so on, until the number of consecutive ASCII characters corresponding to the character to be processed is 0.

[0096] In addition, in an embodiment of the present application, after S304 returns to continue executing S301, it is also possible to continue to determine whether the characters to be processed in the character string to be processed that is read again are all standard code characters. If so, it means that the character string to be processed that is read again only includes standard code characters. At this time, the preset number can also be updated to the difference between the preset number and 1, so that it can be switched from scalar processing instruction processing to vector read instruction reading more promptly and effectively. It is also possible to continue to determine whether the character string to be processed that is read again includes non-standard code characters. If so, it means that the character string to be processed that is read again includes non-standard code characters. At this time, the preset number can also be updated to the sum of the preset number and 1, so as to avoid the subsequent frequent switching from scalar processing instruction processing to vector read instruction reading performance loss. Among them, the updated preset number is greater than or equal to the initial value of the preset number, for example, the initial value of the preset number is greater than or equal to 2. Therefore, the present application provides a possible implementation method. After S304 returns to continue executing S301, the method can also include the following S6-S7, for example:

[0097] S6: If the characters to be processed in the re-read character string to be processed are all standard code characters, the preset number is updated to the difference between the preset number and 1; wherein the updated preset number is greater than or equal to the initial value of the preset number.

[0098] S7: If the character string to be processed read again includes non-standard code characters, the preset number is updated to the sum of the preset number and 1.

[0099] As an example, if the characters to be processed in the string to be processed that is read again are all ASCII characters, indicating that the processing match with the subsequent vector encoding and decoding instructions is successful, the preset number is updated to the difference between the preset number and 1; if the string to be processed that is read again includes non-ASCII characters, indicating that the processing match with the subsequent vector encoding and decoding instructions fails, the preset number is updated to the sum of the preset number and 1.

[0100] See also Figure 10 , this figure is a processing method based on scalar instructions, a processing method based on vector instructions, and an effect diagram of character encoding and decoding implemented by the embodiment of the present application. Tests were conducted on character strings of different lengths, including ASCII strings, MixASCII strings (evenly mixed ASCII characters and non-ASCII characters), and MajorASCII strings (mixed 99% ASCII characters and 1% non-ASCII characters). It can be seen that for ASCII strings, the implementation method of the present application is basically on par with the processing method based on vector instructions; for MixASCII strings, the implementation method of the present application is basically better than the processing method based on vector instructions. For MajorASCII strings, the implementation method of the present application is significantly better than the processing method based on vector instructions and the processing method based on scalar instructions.

[0101] The character encoding and decoding method provided in the above embodiment reads the character string to be processed in the target character string of the first encoding method through a vector reading instruction, and the target character string has standard code characters and non-standard code characters, and the standard code characters are single-byte characters that meet the preset encoding standard; when the characters to be processed in the character string to be processed are all standard code characters, the vector encoding and decoding instructions are used to simultaneously perform standard code encoding and decoding processing on the characters to be processed in the character string to be processed to obtain the encoded and decoded character string corresponding to the second encoding method of the character string to be processed; when the character string to be processed includes non-standard code characters, the scalar processing instructions are used to perform encoding and decoding processing on the characters to be processed in sequence to obtain the encoded and decoded character string corresponding to the second encoding method of the character to be processed; after each scalar processing instruction completes processing, when it is detected that the statistical data of the standard code characters corresponding to the character to be processed meet the preset conditions, it returns to continue executing the vector reading instruction reading step until the encoded and decoded character string corresponding to the second encoding method of the target character string is obtained.

[0102] It can be seen that in the case where the target string of the first encoding method has standard code characters and non-standard code characters, after the to-be-processed string is read by the vector read instruction, the to-be-processed string that only includes standard code characters is processed by the vector encoding and decoding instruction, thereby realizing batch and fast encoding and decoding of standard code characters corresponding to the second encoding method; the to-be-processed string including non-standard code characters is processed by the scalar processing instruction, thereby realizing simple and convenient encoding and decoding of standard code characters and non-standard code characters corresponding to the second encoding method; and after each scalar processing instruction completes processing, the statistical data of the standard code characters meets the preset conditions, and the processing is switched from the scalar processing instruction to the vector read instruction in a timely and effective manner. Based on this, this method is more suitable for encoding and decoding strings of mixed standard code characters and non-standard code characters, thereby improving the processing efficiency of encoding and decoding.

[0103] In accordance with the character encoding and decoding method provided in the above embodiment, an embodiment of the present application further provides a character encoding and decoding device.

[0104] See also Figure 11 , Figure 11 Schematic diagram of a character encoding and decoding device provided in an embodiment of the present application. Figure 11 As shown, the character encoding and decoding device 1100 includes: a reading unit 1101, a first encoding and decoding unit 1102, a second encoding and decoding unit 1103 and a returning unit 1104;

[0105] A reading unit 1101 is configured to read a character string to be processed in a target character string in a first encoding mode based on a vector read instruction; the target character string includes standard code characters and non-standard code characters, and the standard code characters are single-byte characters that meet a preset encoding standard;

[0106] The first encoding and decoding unit 1102 is configured to, if all characters to be processed in the character string to be processed are standard code characters, simultaneously perform standard code encoding and decoding on the characters to be processed in the character string to be processed based on the vector encoding and decoding instruction to obtain a codec character string corresponding to the second encoding method of the character string to be processed;

[0107] The second encoding and decoding unit 1103 is configured to, if the character string to be processed includes non-standard coded characters, perform encoding and decoding processing on the characters to be processed in sequence based on the scalar processing instruction to obtain encoded and decoded characters corresponding to the characters to be processed in the second encoding mode;

[0108] The return unit 1104 is used to return to continue executing the vector read instruction to read the target character string in the first encoding method after obtaining the encoding and decoding character corresponding to the second encoding method of the character to be processed each time, if the standard code character statistics corresponding to the character to be processed meet the preset conditions, until the encoding and decoding character corresponding to the target character string in the second encoding method is obtained.

[0109] As a possible implementation, the preset conditions include that the position of the character to be processed is after the position of the last non-standard code character in the character string to be processed, and the number of consecutive standard code characters corresponding to the character to be processed matches a preset number.

[0110] As a possible implementation, the apparatus further includes: a first updating unit and a second updating unit;

[0111] A first updating unit is configured to update the preset number to the difference between the preset number and 1 if all characters to be processed in the character string to be processed read again are standard code characters;

[0112] The second updating unit is further configured to update the preset number to the sum of the preset number and 1 if the character string to be processed read again includes non-standard code characters;

[0113] The updated preset number is greater than or equal to the initial value of the preset number.

[0114] As a possible implementation, the scalar processing instruction includes a scalar read instruction and a scalar encoding and decoding instruction, and the second encoding and decoding unit 1103 includes: a reading subunit and an encoding and decoding subunit;

[0115] A reading subunit, configured to sequentially read characters to be processed in a character string to be processed based on a scalar read instruction;

[0116] The encoding and decoding subunit is used to perform encoding and decoding processing on the characters to be processed in sequence based on the scalar encoding and decoding instructions, and obtain the encoded and decoded characters corresponding to the second encoding method of the characters to be processed.

[0117] As a possible implementation, the encoding and decoding subunit includes: a first encoding and decoding module and a second encoding and decoding module;

[0118] A first encoding and decoding module is configured to, if the character to be processed is a standard code character, perform standard code encoding and decoding processing on the character to be processed based on the scalar encoding and decoding instruction to obtain an encoded and decoded character corresponding to a second encoding method of the character to be processed;

[0119] The second encoding and decoding module is used to perform non-standard code encoding and decoding processing on the character to be processed based on the scalar encoding and decoding instruction if the character to be processed is a non-standard code character, and obtain the encoded and decoded character of the second encoding method corresponding to the character to be processed.

[0120] As a possible implementation, the second encoding and decoding processing module includes: an encoding and decoding submodule and a determination submodule;

[0121] The encoding and decoding submodule is configured to, if the character to be processed is a non-standard code character that satisfies the first encoding method, perform non-standard code encoding and decoding processing on the character to be processed based on the scalar encoding and decoding instruction to obtain an encoding and decoding result corresponding to the character to be processed;

[0122] The determination submodule is configured to determine the encoding and decoding result as an encoded and decoded character corresponding to the second encoding method of the character to be processed if the encoding and decoding result satisfies the second encoding method.

[0123] As a possible implementation, the apparatus further includes: an operating unit, a first determining unit, and a second determining unit;

[0124] An AND operation unit, configured to perform an AND operation on the character string to be processed and a preset character string to obtain an operation result; the highest bit of each byte in the preset character string is 1, and the remaining bits are 0, and the number of bytes in the character string to be processed and the preset character string is the same;

[0125] A first determining unit is configured to determine, if the operation result is 0, that all characters to be processed in the character string to be processed are standard code characters;

[0126] The second determining unit is configured to determine, if the operation result is not 0, that the character string to be processed includes non-standard code characters.

[0127] The device for character encoding and decoding provided in the above embodiment reads a character string to be processed in a target character string of a first encoding method through a vector reading instruction, wherein the target character string has standard code characters and non-standard code characters, and the standard code characters are single-byte characters that meet the preset encoding standard; when the characters to be processed in the character string to be processed are all standard code characters, the vector encoding and decoding instructions are used to simultaneously perform standard code encoding and decoding processing on the characters to be processed in the character string to be processed to obtain an encoded and decoded character string corresponding to the second encoding method for the character string to be processed; when the character string to be processed includes non-standard code characters, the scalar processing instructions are used to perform encoding and decoding processing on the characters to be processed in sequence to obtain an encoded and decoded character string corresponding to the second encoding method for the character to be processed; after each scalar processing instruction is completed, when it is detected that the statistical data of the standard code characters corresponding to the character to be processed meet the preset conditions, the vector reading instruction reading step is returned to continue until the encoded and decoded character string corresponding to the second encoding method for the target character string is obtained.

[0128] It can be seen that in the case where the target string of the first encoding method has standard code characters and non-standard code characters, after the to-be-processed string is read by the vector read instruction, the to-be-processed string that only includes standard code characters is processed by the vector encoding and decoding instruction, thereby realizing batch and fast encoding and decoding of standard code characters corresponding to the second encoding method; the to-be-processed string including non-standard code characters is processed by the scalar processing instruction, thereby realizing simple and convenient encoding and decoding of standard code characters and non-standard code characters corresponding to the second encoding method; and after each scalar processing instruction completes processing, the statistical data of the standard code characters meets the preset conditions, and the processing is switched from the scalar processing instruction to the vector read instruction in a timely and effective manner. Based on this, this method is more suitable for encoding and decoding strings of mixed standard code characters and non-standard code characters, thereby improving the processing efficiency of encoding and decoding.

[0129] The embodiment of the present application also provides a device for character encoding and decoding. The computer device provided by the embodiment of the present application will be introduced from the perspective of hardware implementation.

[0130] See also Figure 12 , Figure 12 This is a schematic diagram of a server structure provided by an embodiment of the present application. The server 1200 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPUs) 1222 (for example, one or more processors) and memories 1232, and one or more storage media 1230 (for example, one or more massive storage devices) for storing application programs 1242 or data 1244. Among them, the memories 1232 and the storage media 1230 may be temporary storage or permanent storage. The program stored in the storage medium 1230 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1222 may be configured to communicate with the storage medium 1230 to execute a series of instruction operations in the storage medium 1230 on the server 1200.

[0131] The server 1200 may also include one or more power supplies 1226, one or more wired or wireless network interfaces 1250, one or more input and output interfaces 1258, and / or one or more operating systems 1241, such as Windows Server 200. NM , Mac OS X NM , Unix NM , Linux NM , FreeBSD NM etc.

[0132] The steps performed by the server in the above embodiment can be based on the Figure 12 The server structure shown.

[0133] For example, the CPU 1222 is configured to execute the following steps:

[0134] Reading a character string to be processed in a target character string in a first encoding mode based on a vector read instruction; the target character string includes standard code characters and non-standard code characters, and the standard code characters are single-byte characters that meet a preset encoding standard;

[0135] If all the characters to be processed in the character string to be processed are standard code characters, performing standard code encoding and decoding processing on the characters to be processed in the character string to be processed simultaneously based on the vector encoding and decoding instruction to obtain an encoded and decoded character string corresponding to the second encoding method of the character string to be processed;

[0136] If the character string to be processed includes non-standard code characters, encoding and decoding the characters to be processed in the character string to be processed in sequence based on the scalar processing instruction to obtain encoded and decoded characters corresponding to the second encoding method of the characters to be processed;

[0137] After obtaining the encoded and decoded characters of the second encoding method corresponding to the character to be processed each time, if the statistical data of the standard code characters corresponding to the character to be processed meets the preset conditions, return to continue executing the vector read instruction to read the character string to be processed in the target string of the first encoding method until the encoded and decoded character string of the second encoding method corresponding to the target string is obtained.

[0138] In response to the character encoding and decoding method described above, an embodiment of the present application further provides a terminal device, which is applied to the above-mentioned character encoding and decoding method.

[0139] See also Figure 13 , Figure 13 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method section of the embodiment of the present application. The terminal device can be any terminal device including a mobile phone, tablet computer, etc. Taking the terminal device as a mobile phone as an example:

[0140] Figure 13 The block diagram shows a partial structure of a mobile phone related to the terminal device provided in the embodiment of the present application. Figure 13 The mobile phone includes components such as a radio frequency (RF) circuit 1310, a memory 1320, an input unit 1330, a display unit 1340, a sensor 1350, an audio circuit 1360, a wireless fidelity (WiFi) module 1370, a processor 1380, and a power supply 1390. It will be understood by those skilled in the art that Figure 13The mobile phone structure shown in the figure does not constitute a limitation to the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0141] The following combination Figure 13 A detailed introduction to the various components of a mobile phone:

[0142] The RF circuit 1310 can be used to receive and send signals during information transmission or calls. In particular, it receives downlink information from the base station and sends it to the processor 1380 for processing. In addition, it sends the designed uplink data to the base station. Generally, the RF circuit 1310 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit 1310 can also communicate with the network and other devices via wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LNE), email, Short Messaging Service (SMS), etc.

[0143] Memory 1320 can be used to store software programs and modules. Processor 1380 implements various functional applications and data processing of the mobile phone by running the software programs and modules stored in memory 1320. Memory 1320 may mainly include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area may store data created based on the use of the mobile phone (such as audio data, a phone book, etc.). In addition, memory 1320 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0144] The input unit 1330 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 1330 may include a touch panel 1331 and other input devices 1332. The touch panel 1331, also known as a touch screen, can collect user touch operations on or near it (such as operations performed by the user using a finger, stylus, or any other suitable object or accessory on or near the touch panel 1331) and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 1331 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch direction and detects the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device and converts it into touch point coordinates, which are then sent to the processor 1380. It can also receive commands sent by the processor 1380 and execute them. In addition, the touch panel 1331 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1331, the input unit 1330 may further include other input devices 1332. Specifically, the other input devices 1332 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, power keys, etc.), a trackball, a mouse, and a joystick.

[0145] The display unit 1340 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 1340 may include a display panel 1341. Optionally, the display panel 1341 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 1331 may cover the display panel 1341. When the touch panel 1331 detects a touch operation on or near it, it is transmitted to the processor 1380 to determine the type of touch event. Subsequently, the processor 1380 provides corresponding visual output on the display panel 1341 according to the type of touch event. Although in Figure 13 In the embodiment, the touch panel 1331 and the display panel 1341 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 1331 and the display panel 1341 can be integrated to realize the input and output functions of the mobile phone.

[0146] The mobile phone may also include at least one sensor 1350, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 1341 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 1341 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that the mobile phone can also be configured with, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be described here.

[0147] Audio circuit 1360, speaker 1361, and microphone 1362 provide an audio interface between the user and the phone. Audio circuit 1360 converts received audio data into electrical signals and transmits them to speaker 1361, which then converts them into sound signals for output. Microphone 1362, on the other hand, converts collected sound signals into electrical signals, which are then received by audio circuit 1360 and converted into audio data. The audio data is then processed by processor 1380 and transmitted to, for example, another phone via RF circuit 1310, or stored in memory 1320 for further processing.

[0148] WiFi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse the web and access streaming media through the WiFi module 1370. It provides users with wireless broadband Internet access. Figure 13 A WiFi module 1370 is shown, but it is understandable that it is not an essential component of the mobile phone and can be omitted as needed without changing the essence of the invention.

[0149] Processor 1380 is the control center of the phone, connecting all parts of the phone using various interfaces and circuits. By running or executing software programs and / or modules stored in memory 1320 and accessing data stored in memory 1320, it executes various phone functions and processes data, thereby providing overall control of the phone. Optionally, processor 1380 may include one or more processing units; preferably, processor 1380 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 1380.

[0150] The mobile phone also includes a power supply 1390 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 1380 through a power management system, thereby managing charging, discharging, and power consumption through the power management system.

[0151] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0152] In the embodiment of the present application, the memory 1320 included in the mobile phone can store program codes and transmit the program codes to the processor.

[0153] The processor 1380 included in the mobile phone can execute the following steps according to the instructions in the program code:

[0154] Reading a character string to be processed in a target character string in a first encoding mode based on a vector read instruction; the target character string includes standard code characters and non-standard code characters, and the standard code characters are single-byte characters that meet a preset encoding standard;

[0155] If all the characters to be processed in the character string to be processed are standard code characters, performing standard code encoding and decoding processing on the characters to be processed in the character string to be processed simultaneously based on the vector encoding and decoding instruction to obtain an encoded and decoded character string corresponding to the second encoding method of the character string to be processed;

[0156] If the character string to be processed includes non-standard code characters, encoding and decoding the characters to be processed in the character string to be processed in sequence based on the scalar processing instruction to obtain encoded and decoded characters corresponding to the second encoding method of the characters to be processed;

[0157] After obtaining the encoded and decoded characters of the second encoding method corresponding to the character to be processed each time, if the statistical data of the standard code characters corresponding to the character to be processed meets the preset conditions, return to continue executing the vector read instruction to read the character string to be processed in the target string of the first encoding method until the encoded and decoded character string of the second encoding method corresponding to the target string is obtained.

[0158] An embodiment of the present application further provides a computer-readable storage medium for storing a computer program, which is used to execute the character encoding and decoding method provided in the above embodiment.

[0159] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a device for character encoding and decoding reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the device for character encoding and decoding to perform the character encoding and decoding methods provided in various optional implementations of the above aspects.

[0160] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the above-mentioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the above-mentioned storage medium can be at least one of the following media: read-only memory (English: Read-Only Memory, abbreviated: ROM), RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0161] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0162] The above is only one specific implementation of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for character encoding and decoding, characterized in that: The method comprises: Reading a character string to be processed in a target character string of a first encoding method based on a vector read instruction; the target character string includes standard code characters and non-standard code characters, and the standard code characters are single-byte characters that meet a preset encoding standard; If all the characters to be processed in the character string to be processed are the standard code characters, performing standard code encoding and decoding processing on the characters to be processed in the character string to be processed simultaneously based on the vector encoding and decoding instruction to obtain an encoded and decoded character string corresponding to the second encoding mode of the character string to be processed; If the character string to be processed includes the non-standard code character, encoding and decoding the characters to be processed in the character string to be processed in sequence based on the scalar processing instruction to obtain the encoded and decoded characters corresponding to the second encoding method of the characters to be processed; After obtaining the encoded and decoded character of the second encoding method corresponding to the character to be processed each time, if the standard code character statistics corresponding to the character to be processed meet the preset conditions, return to continue executing the vector-based read instruction to read the character string to be processed in the target string of the first encoding method until the encoded and decoded character string of the second encoding method corresponding to the target character string is obtained.

2. The method according to claim 1, characterized in that The preset conditions include that the position of the character to be processed is after the position of the last non-standard code character in the character string to be processed, and the number of consecutive standard code characters corresponding to the character to be processed matches a preset number.

3. The method according to claim 2, characterized in that After returning to continue executing the vector-based read instruction to read the character string to be processed in the target character string in the first encoding mode, the method further includes: If all characters to be processed in the character string to be processed read again are the standard code characters, the preset number is updated to the difference between the preset number and 1; If the character string to be processed read again includes the non-standard code character, updating the preset number to the sum of the preset number and 1; The updated preset number is greater than or equal to the initial value of the preset number.

4. The method according to claim 1, characterized in that The scalar processing instruction includes a scalar read instruction and a scalar encoding and decoding instruction. The encoding and decoding processing is performed on the characters to be processed in the character string to be processed in sequence based on the scalar processing instruction to obtain the encoded and decoded characters corresponding to the second encoding method of the characters to be processed, including: Reading the characters to be processed in the character string to be processed in sequence based on the scalar read instruction; The characters to be processed are encoded and decoded in sequence based on the scalar encoding and decoding instructions to obtain encoded and decoded characters corresponding to the second encoding method of the characters to be processed.

5. The method according to claim 4, characterized in that: The encoding and decoding processing of the characters to be processed is performed in sequence based on the scalar encoding and decoding instructions to obtain the encoded and decoded characters corresponding to the second encoding mode of the characters to be processed, including: If the character to be processed is the standard code character, performing standard code encoding and decoding processing on the character to be processed based on the scalar encoding and decoding instruction to obtain a codec character corresponding to the second encoding method of the character to be processed; If the character to be processed is the non-standard code character, non-standard code encoding and decoding processing is performed on the character to be processed based on the scalar encoding and decoding instruction to obtain the encoded and decoded character of the character to be processed corresponding to the second encoding method.

6. The method according to claim 5, characterized in that The performing non-standard code encoding and decoding processing on the character to be processed based on the scalar encoding and decoding instruction to obtain the encoded and decoded character corresponding to the second encoding mode of the character to be processed includes: If the character to be processed is a non-standard code character that satisfies the first encoding method, performing non-standard code encoding and decoding processing on the character to be processed based on the scalar encoding and decoding instruction to obtain an encoding and decoding result corresponding to the character to be processed; If the encoding and decoding result satisfies the second encoding method, the encoding and decoding result is determined as the encoding and decoding character of the corresponding character to be processed in the second encoding method.

7. The method according to any one of claims 1 to 6, characterized in that: The method for determining that all characters to be processed in the character string to be processed are characters of the standard code includes: Performing an AND operation on the character string to be processed and a preset character string to obtain an operation result; the highest bit of each byte in the preset character string is 1 and the remaining bits are 0, and the number of bytes in the character string to be processed and the preset character string is the same; If the operation result is 0, it is determined that the characters to be processed in the character string to be processed are all the standard code characters; The method for determining that the character string to be processed includes the non-standard code character includes: If the operation result is not 0, it is determined that the character string to be processed includes the non-standard code character.

8. A device for character encoding and decoding, characterized in that: The device comprises: a reading unit, a first encoding and decoding unit, a second encoding and decoding unit and a returning unit; The reading unit is configured to read a character string to be processed in a target character string in a first encoding mode based on a vector read instruction; the target character string includes standard code characters and non-standard code characters, and the standard code characters are single-byte characters that meet a preset encoding standard; The first encoding and decoding unit is configured to, if all the characters to be processed in the character string to be processed are the standard code characters, simultaneously perform standard code encoding and decoding processing on the characters to be processed in the character string to be processed based on the vector encoding and decoding instruction, to obtain a codec character string corresponding to the second encoding mode for the character string to be processed; The second encoding and decoding unit is configured to, if the character string to be processed includes the non-standard code character, sequentially encode and decode the characters to be processed in the character string to be processed based on the scalar processing instruction to obtain the encoded and decoded characters corresponding to the second encoding method of the characters to be processed; The return unit is used to return to continue executing the vector-based read instruction to read the character string to be processed in the target character string of the first encoding method after obtaining the encoding and decoding character corresponding to the second encoding method of the character to be processed each time, if the standard code character statistics corresponding to the character to be processed meet the preset conditions, until the encoding and decoding character string corresponding to the target character string of the second encoding method is obtained.

9. A computer device, characterized in that: The computer device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the character encoding and decoding method according to any one of claims 1 to 7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the character encoding and decoding method according to any one of claims 1 to 7.

Images