A method, electronic device and medium for Java object serialization and deserialization

Abstract

The application provides a Java object serialization and deserialization method, which can work in a heterogeneous environment, can select different transmission strategies according to different transmission scenes, supports reasonable data type conversion when binary code is deserialized into objects, and has the advantages of fast data transmission and persistent storage efficiency and small space occupation of serialized data. Through the application, users can realize the serialization and deserialization of native types, common basic types, basic type arrays, basic collections and class objects in Java, and can also realize the conversion of reasonable data types.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a method for serializing Java objects into binary codes and deserializing binary codes into Java objects, an electronic device and a medium. BACKGROUND

[0002] Serialization refers to the process of storing Java object data in the heap memory into a disk file or passing to other network nodes through a certain way, which is usually the process of converting data structures or objects into binary. Deserialization refers to the process of restoring object data in a disk file or object data on a network node into a Java object, that is, the process of converting binary strings generated in the serialization process into data structures or objects. When two processes are communicating remotely, they can send various types of data to each other, regardless of the type of data, which will be transmitted in the form of binary sequence on the network. The sender needs to convert the object into a byte sequence to transmit it on the network; the receiver needs to restore the byte sequence to the object. Therefore, serialization and deserialization of Java objects have broad application prospects in the fields of big data, electronic communication and other fields that require frequent data exchange and storage.

[0003] In the field of big data analysis, serialization and deserialization of objects are a basic operation and a basic feature of efficient communication between distributed computing nodes and potentially non-uniform execution environments. Serialization and deserialization operations are widely used in big data analysis frameworks for remote process calls and shuffling of massive data transmission. Massive data operations in today's big data analysis frameworks, such as map / reduce, shufflfle, use serialization and deserialization operations a lot. In large web applications, to solve the intermittent failures that may occur in these applications, data serialization in the application is an important link, and serialization and deserialization are almost things that engineers face every day. A proper serialization protocol can not only improve the versatility, robustness, security and optimization of system performance, but also make the system easy to debug and easy to expand. In the field of electronic communication, serialization technology has always been an important part of data exchange. Since electronic communication requires frequent data transmission, the sender needs to serialize the object into binary code during data transmission, and the receiver needs to deserialize the binary code into an object. Therefore, improving the speed of serialization and deserialization can greatly improve the efficiency of data transmission.

[0004] Currently, the serialization and deserialization methods for Java objects have poor flexibility, lack of serialization stream sending strategies corresponding to the corresponding scenarios, and low efficiency of data transmission and processing. SUMMARY

[0005] In view of the problems in the prior art, the present application provides a Java object serialization and deserialization method, electronic equipment and medium, which can adapt to various transmission conditions in a heterogeneous environment, including serialization and deserialization of native types (boolean, short, int, etc.), commonly used basic types (String, Date, BigInteger, etc.), basic type arrays (boolean[], short[], int[], etc.), basic collections and class objects in Java.

[0006] The technical scheme of the present application is:

[0007] The Java object serialization and deserialization method comprises the following steps of serialization:

[0008] Step 1: initialize an object schema for storing enumeration types and class object structures in serialization and an object dataBuffer of a byte array encapsulation class; the byte array buffer in the object dataBuffer is used for storing the result of this serialization;

[0009] Step 2: generate serialization data

[0010] First, the object to be serialized is checked to check whether it is AsType, and if it is AsType, the value is converted into the type corresponding to AsType; the AsType is a set of interfaces defined for transmission optimization, which can realize processing of one data type as another data type;

[0011] Then, different serialization strategies are adopted according to the object type to be serialized to obtain a serialization result array buffer; the object type is: direct conversion type, basic collection, file or enumeration type, and class object;

[0012] Step 3: generate result data by using the obtained serialization result array buffer;

[0013] Deserialization comprises the following steps:

[0014] Step 4: read out the byte length occupied by the content of the schema, the total byte length of the serialization result, the content of the schema and the serialization result in the final result array of serialization in sequence;

[0015] Step 5: adopt different deserialization strategies according to the data type to be deserialized to realize the deserialization process.

[0016] Further, in step 1, the object schema is divided into two types, EnumSchema for storing the structure of the enumeration type and ObjectSchema for storing the structure of the class object.

[0017] Further, in step 2, the direct conversion type includes the native type of Java, the commonly used basic type of Java and the array of the basic type of Java; if the object type is the direct conversion type, the type code of the object is stored in the byte array buffer in the dataBuffer, and then the object is serialized and the serialized result is stored in the byte array.

[0018] Further, in step 2, the basic collection, file or enumeration type includes the list type, array type, Map type, set type, file type or enumeration type.

[0019] If the object type is the list type, array type, Map type or set type, the byte code corresponding to the current object type is stored in the byte array buffer, then the number of elements of the object is stored, and finally each element in the object is serialized.

[0020] If the object type is the file type, the byte code corresponding to the current object type is stored in the byte array buffer, then the length of the file name and the serialized file name are stored in the byte array buffer, and finally the byte length of the file content and the serialized file content are stored.

[0021] If the object type is the enumeration type, it is first determined whether the enumSchema corresponding to the enumeration type exists, if not, the name of the enumeration type and the corresponding enumSchema are stored in the hash table schemas in the form of key:value, wherein the enumSchema stores each name in the enumeration type definition and the order number of the name in the definition; if the enumSchema corresponding to the enumeration type exists, the byte code corresponding to the enumeration type is first stored in the byte array buffer, then the order number of the enumSchema corresponding to the enumeration type in the hash table schemas is stored, and finally the order number of the value of the enumeration variable in the enumeration type definition is stored.

[0022] Further, in step 2, when serializing the class object, firstly, it is judged whether the objectSchema corresponding to the class exists. If not, the class name corresponding to the class object and the corresponding objectSchema are stored in the hash table schemas in the form of key:value, wherein the objectSchema stores the names of all available attributes of the class and the serial number of the attribute; if the objectSchema corresponding to the class exists, then:

[0023] First, a byte array objectContent is newly created;

[0024] All available attributes in the class are traversed. For each attribute, firstly, the serial number of the attribute in the class is stored in the byte array objectContent, and then each attribute is serialized. After the traversal, all attributes of the entire class are serialized, and the result is stored in the byte array objectContent;

[0025] In the byte array buffer, firstly, the type code of the class object is stored, then the serial number of the objectSchema corresponding to the class in the hash table schemas is stored, then the size of the byte array objectContent is stored, and finally the content of the entire byte array objectContent is stored. The serialization process of the entire class object ends.

[0026] Further, in step 3, the process of generating the result data of the entire serialization process is as follows:

[0027] Step 3.1: a byte array object rst encapsulating the class object is newly created, the byte array in the object is used to store the result data of this serialization, a byte array schemaBytes is newly created and used to store the content of all schemas, and a byte array dataBytes is newly created and used to store the content of the serialization result array buffer;

[0028] Step 3.2: all schemas in this serialization process are traversed. For each schema, firstly, the serial number of the schema is stored in the byte array schemaBytes, then the type of the schema is stored, then the number of bytes occupied by the content of the schema is stored, and finally the content of the schema is stored. After the traversal, the result data of all schemas is stored in the byte array schemaBytes;

[0029] Step 3.3: the content of the serialization result array buffer is copied to the byte array dataBytes;

[0030] Step 3.4: In the byte array of the object rst, first store the length of the byte array schemaBytes, then store the length of the byte array dataBytes, then store the content of the byte array schemaBytes, and finally store the content of the byte array dataBytes, and the result data of the entire serialization process is stored.

[0031] Further, in step 4, if the content part of the schema contains multiple schemas, separate the order number and content of each schema from the schema content part in the byte array, and store the order number of the schema and the corresponding enumSchema or objectSchema in the hash table schemas in the form of key:value.

[0032] Further, in step 5, the deserialization process includes:

[0033] Step 5.1: Read the first byte of the byte array to obtain the data type of the object, and start from the second byte of the byte array to obtain the data value that needs to be deserialized;

[0034] Step 5.2: Check if the object type that needs to be converted exists, if it exists, select the corresponding data structure to store the deserialized object according to different types;

[0035] Step 5.3: Check if the object type that needs to be deserialized is a direct conversion type, if it is, obtain the corresponding serialization and deserialization processor of the type, the processor first reads the data stored in the offset position, and then deserializes the object to obtain the deserialized object;

[0036] Step 5.4: Determine whether the object type that needs to be deserialized is a linked list List type, a Map type, a Set type, a File type, or an Enum type;

[0037] If the object type that needs to be deserialized is a linked list List type, a Map type, or a Set type, first read the number of elements from the serialization result array and convert it to an int type, then read and deserialize the content of each element, and then store the obtained elements in the corresponding type of object; In the Map type, each element is in the form of key:value, and when deserializing the Map, the key and the value are deserialized respectively;

[0038] If the object type that needs to be deserialized is a file type, read the byte length occupied by the file name length from the serialization result array, then read the file name according to the length, then read the byte length occupied by the file content, and then read the file content according to the length;

[0039] If the object type to be deserialized is an enumeration type, the schema order number corresponding to the enumeration type is read from the serialization result array, then the order number corresponding to the value of the enumeration variable in the enumeration type definition is read, the enumSchema corresponding to the enumeration type is obtained from the hash table schemas according to the schema order number, and finally the name of the enumeration value is obtained from the enumSchema according to the order number corresponding to the value in the enumeration type definition;

[0040] Step 5.5: Determine whether the object type to be deserialized is a class object. If yes, first separate the schema order number and the byte length occupied by the serialization content of the object from the serialization result array in order; then obtain the objectSchema corresponding to the class from the hash table schemas according to the schema order number; then obtain the class name from the objectSchema corresponding to the class, so as to create the object of the class through the forName function of Java; traverse and read the order number of each class attribute from the serialization result array, obtain the name of the attribute through the order number, then deserialize to obtain the value of the attribute, and set the value of the attribute in the created class object; after the traversal of the class attributes is completed, the deserialized class object is obtained.

[0041] An electronic device, comprising a processor and a memory storing a program, the program comprising instructions that, when executed by the processor, cause the processor to perform the above method.

[0042] A computer-readable storage medium storing a program, the program comprising instructions that, when executed by a processor of an electronic device, cause the electronic device to perform the above method.

[0043] Through the application, the user can select corresponding serialization flow sending strategies in different heterogeneous data transmission scenarios, greatly improving the flexibility in the serialization process. At the same time, the application stores and transmits data types using byte encoding, which, compared with traditional string encoding, has faster efficiency and smaller space occupation of the generated result data after serialization. The user can combine the application with data transmission and data operation of big data to greatly shorten the data transmission and processing efficiency in the case of large data volume; the application can also be applied to large web development to improve the performance of large web applications; the user can also combine the application with the electronic communication field to transmit data using the serialization and deserialization method of the application to improve the communication efficiency. In summary, the application provides a solid technical foundation for network communication and data persistence in terms of serialization and deserialization data types, efficiency, space occupation, etc. The application can be applied to fields that need to frequently use serialization and deserialization technologies, such as big data field, web development field, electronic communication field, etc.

[0044] Advantages

[0045] The application realizes a serialization method that can work in a heterogeneous environment and can select different transmission strategies according to different transmission scenarios, and supports reasonable data type conversion when binary code is deserialized into objects (such as converting a linked list into an array), has faster data transmission and persistent storage efficiency, and smaller space occupation of serialized data. Through the application, the user can realize serialization and deserialization of native types (boolean, short, int, etc.), commonly used basic types (String, Date, BigInteger, etc.), basic type arrays (boolean[], short[], int[], etc.), basic collections, and class objects in Java, and can also realize reasonable data type conversion.

[0046] The user can combine the application with big data, web development, electronic communication, etc. For example, the application can be applied to portable and lossless communication between distributed computing nodes. When transmitting data between nodes, the sender node first serializes a large number of objects into a byte stream; then, the receiver node reconstructs the objects from the serialized byte stream. The serialization and deserialization method of the application can greatly improve the efficiency of data transmission. The application can also be combined with large web development. Since large web applications need to frequently transmit data and store data locally during operation, using the application can significantly improve the performance of large web applications. The application can also be combined with the electronic communication field to save time spent on serialization and deserialization and effectively improve the efficiency of data transmission in electronic communication.

[0047] Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following and the attached drawings. BRIEF DESCRIPTION OF DRAWINGS

[0048] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, including the accompanying drawings.

[0049] Figure 1 is a serialized stream structure diagram of an embodiment.

[0050] Figure 2 is a serialized stream sending strategy diagram in different scenarios of an embodiment. DETAILED DESCRIPTION

[0051] The present application is a serialization method that can work in a heterogeneous environment and can select different transmission strategies according to different transmission scenarios. The serialized stream formed by the present application is pure binary, and therefore can be used in a heterogeneous scenario.

[0052] The present application uses schema to store the structure of enumeration types and class objects in serialization, and is divided into two types: EnumSchema for storing enumeration type structure and ObjectSchema for storing class object structure.

[0053] The serialized stream formed by the present application is divided into two relatively independent parts: all schemas form the header of the serialized stream, and the data value after serialization forms the data part of the serialized stream. The header of the serialized stream is detachable, which maximally strips the data type and data value inside the object, and therefore supports selecting different transmission strategies according to different transmission scenarios, greatly improving the flexibility in the serialization and deserialization process. In the data transmission process, the class structure that has been transmitted can not be transmitted again, or the class structure needed for this transmission can be stored in a remote service in advance, reducing the size of the serialized stream. After the data transmission process is completed, the receiving node can deserializes the header and data part of the serialized stream in parallel, improving the efficiency of the deserialization process.

[0054] When serializing a class object of a nested call, the present application takes Figure 1 (a) as an example, the object of ClassA contains the objects of ClassB and ClassC, and contains an enumeration type, and the object of ClassB contains the object of ClassD. For such a class object of a nested call, the code snippet is first converted into a corresponding object graph, as shown in Figure 1 (b). The byte stream structure after serialization of the present application is as shown in Figure 1(c) shown, in the serialization process, for the class object type, the objectschema corresponding to the class is stored in the header of the serialization stream, each class is stored only once, and the id corresponding to the class type is generated according to the storage order. Then the class object is recursively split, and the basic type in the class object is directly serialized into a byte stream, and the class object type, collection type and the like which cannot be directly serialized is also recursively split until all are disassembled into basic types. For each enumeration type, the corresponding enumschema storage structure is stored, and the objectschema is also stored in the header of the serialization stream.

[0055] In the deserialization process, the serialized byte stream is first split into a header and a data part, the header is deserialized into an objectschema representing a class object and an enumschema representing an enumeration type structure, and the data part is deserialized according to the schema. Similar to the serialization process, the deserialization process also takes a recursive way to deserialize the class object, for the basic type, directly deserializes into the corresponding type, for the class object type, collection type and the like which cannot be directly deserialized, recursively split these types until all are disassembled into basic types, and the basic types obtained after disassembly are deserialized.

[0056] The application can select different data transmission strategies according to different transmission scenarios. For the case of small amount of data transmission, at this time the influence of the header on the whole data transmission efficiency is not big, the header and the data part can be sent in series, such as Figure 2 (a) shown; when a large amount of data transmission is needed, such as in the data transmission process in the field of big data, electronic information and the like, since most of the cases are multiple transmissions of objects of the same batch of classes, therefore the two parts can be transmitted separately, such as Figure 2 (b) shown. For the sending node, only the schema needed in the transmission process is sent as the header of the serialization stream in the first transmission, or the header is sent in advance to the remote service, such as Figure 2 (c) shown, in the whole data transmission process, the sending node only needs to send the serialized header once, correspondingly, the receiving node also only needs to receive once, and the rest of the time is used for transmitting data values. Since the application divides the serialization stream into detachable header and data part as the transmission main body, after the receiving node receives all the serialization streams, the header and the data part can be deserialized separately, and the data structure recovery and data value filling are synchronized.

[0057] The application also uses a self-defined length coding method to effectively reduce the size of the serialized stream. The serialization process of an object requires a large amount of storage length, such as storing the length of a basic collection type such as an array, a linked list, etc. when serializing the basic collection type; the length of a schema of a class object also needs to be stored first when storing the schema (i.e. the number of occupied bytes in the byte array after serialization). The conventional serialization process stores the length in the original long type. In a real data transmission scenario, the length value is often not very large, and the use of long undoubtedly causes a waste of space in most cases. In order to solve this problem, the application uses a variable-length integer to store all the lengths used in the serialization process. For a long type representing a length, the application first checks the actual size and constructs a byte array according to the number of valid value bits of the long, such as when the long is 257, the valid value is 9 bits, and a two-byte size byte array is constructed. In this case, the length coding method of the application saves two bytes of space compared with the conventional serialization method. Considering the universality of the length attribute in the serialization process, the length coding method of the application can greatly reduce the size of the serialized stream.

[0058] The specific steps of serialization and deserialization are as follows:

[0059] (I) Serialization

[0060] In order to support the serialization of various data types in Java objects, a schema for storing the enumeration type and class object structure in serialization and a byte array encapsulation class object dataBuffer are first initialized, and the byte array buffer in the object dataBuffer is used to store the results of this serialization; then different serialization strategies are adopted according to the data types to be serialized; finally, the result data of this serialization is generated. The steps of serialization in the application are described as follows:

[0061] Step 1: Initialization

[0062] An object schema of a MotpBuilderSchema class and an object dataBuffer of a byte array encapsulation class MByteBuffer are initialized. The schema is used to store the enumeration type and class object structure in serialization, which is divided into two types: EnumSchema for storing the enumeration type structure and ObjectSchema for storing the class object structure; the byte array buffer in the object dataBuffer is used to store the results of this serialization.

[0063] Step 2: Generate serialized data

[0064] 1) Object Simplification

[0065] Motp first checks the object to be serialized, and checks if it is an AsType. If it is an AsType, the value is converted to the type corresponding to the AsType. AsType is a set of interfaces defined by Motp for transmission optimization, which can treat one data type as another data type. For example, when transmitting the coordinates (x, y) of a point, the conventional transmission method needs to transmit the x field and the y field respectively. Motp uses a long type to store the coordinates, with the first 32 bits storing the x coordinate and the last 32 bits storing the y coordinate. When serializing the long type, it needs to be restored to the original x field and y field form before the subsequent serialization step.

[0066] 2) Determine if it is a direct conversion type

[0067] Motp checks if the type of the object to be serialized is a direct conversion type, which includes Java's native types (boolean, short, int, etc.), Java's commonly used basic types (String, Date, BigInteger, etc.), and Java's basic type arrays (boolean[], short[], int[], etc.).

[0068] For direct conversion types:

[0069] (1) Store the type code of the object

[0070] Motp encodes each data type by one byte (such as int type encoded by 0x03), and Motp stores the type code of the object in the byte array buffer in dataBuffer after obtaining the corresponding encoding of the type.

[0071] (2) Serialize the object

[0072] Motp serializes the object of this type and stores the serialized result in the byte array. For example, when the appendData function detects that the int type is a direct conversion type, it obtains the int type processor MotpInt32Processer and calls the appendInt function to serialize the int type object. Since the int type occupies 4 bytes, the serialized byte array occupies 4 indexes, and the first index of the byte array stores the high 8 bits of the int type, so the int is right shifted by 24 bits and ANDed with 0xFF to get the high 8 bits of the int type, and the other bytes are calculated and stored in a similar manner.

[0073] 3) Determine whether it is a basic collection, file or enumeration type

[0074] Motp determines whether the object to be serialized is a linked list List type, an array Array type, a Map type, a set Set type, a file File type or an enumeration Enum type.

[0075] (1) Basic collection type

[0076] If the object is a linked list, an array, a Map or a set type, first store the byte code corresponding to the current object type in the byte array buffer, then store the number of elements of the object, and finally traverse each element in the object, call the appendData function to serialize each element in the object, and the serialization process of the object ends. In particular, since each element in the Map is in the form of key:value, when traversing the Map, the key and the value need to be serialized by calling the appendData function respectively.

[0077] (2) File type

[0078] If the object is a file type, first store the byte code corresponding to the current object type in the byte array buffer, then store the file name length and the serialized file name in the byte array buffer, and finally store the byte length of the file content and the serialized file content.

[0079] (3) Enumeration type

[0080] If the object is an enumeration type, first determine whether the enumSchema corresponding to the enumeration type already exists. If it does not exist, store the name of the enumeration type and the corresponding enumSchema in the form of key:value in the hash table schemas (store all enumSchema and objectSchema), where enumSchema stores each name in the enumeration type definition and the order number of the name in the definition (starting from 0, increasing by 1). If the enumSchema corresponding to the enumeration type already exists, first store the byte code corresponding to the enumeration type in the byte array buffer, then store the order number of the enumSchema corresponding to the enumeration type in the hash table schemas (the value is the index value of the storage location plus 1, which is equivalent to the id of the enumSchema in the hash table schemas), and finally store the order number of the value of the enumeration variable in the enumeration type definition.

[0081] 4) Serialization of class objects

[0082] When serializing a class object, first determine whether the objectSchema corresponding to the class exists. If not, store the class name and the corresponding objectSchema in the form of key:value in the hash table schemas (store all enumSchema and objectSchema). The objectSchema stores the names of all available properties of the class (including inherited from the parent class, but not including the parent class properties that have been overridden) and the order number of the property (starting from 1, incrementing by 1). If the objectSchema corresponding to the class already exists, then:

[0083] (1) Create a byte array objectContent.

[0084] (2) Traverse all available properties in the class. For each property, first store the order number of the property in the class in the byte array objectContent, and then call the appendData function to serialize each property. After the traversal is complete, all properties of the entire class are serialized, and the results are stored in the byte array objectContent.

[0085] (3) In the byte array buffer, first store the type code of the class object, then store the order number of the objectSchema corresponding to the class in the hash table schemas (the value is the index value of the storage location plus 1, which is equivalent to the id of the objectSchema in the hash table schemas), then store the size of the byte array objectContent, and finally store the entire byte array objectContent. The serialization process of the entire class object is complete.

[0086] Step 3: Generate result data

[0087] After obtaining the serialized result array buffer, Motp generates the result data of the entire serialization process. When generating the result data:

[0088] 1) Data structure initialization

[0089] Motp creates a byte array encapsulation class MByteBuffer object rst, which is used to store the result data of this serialization. A byte array schemaBytes is created to store the contents of all schemas, and a byte array dataBytes is created to store the contents of the serialized result array buffer.

[0090] 2) Fill the byte array schemaBytes

[0091] All schemas in the serialization process are traversed, and for each schema, the serial number of the schema is first stored in the byte array schemaBytes, then the type of the schema (enumSchema or objectSchema) is stored, then the number of bytes occupied by the schema content is stored, and finally the content of the schema is stored. After the traversal, the result data of all schemas is stored in the byte array schemaBytes.

[0092] 3) Fill the byte array dataBytes

[0093] The content of the serialization result array buffer obtained in steps 1 and 2 is copied to the byte array dataBytes.

[0094] 4) Store the result data in the byte array of the object rst

[0095] In the byte array of the object rst, the length of the byte array schemaBytes is first stored, then the length of the byte array dataBytes is stored, then the content of the byte array schemaBytes is stored, and finally the content of the byte array dataBytes is stored, and the result data of the entire serialization process is stored.

[0096] (II) Deserialization

[0097] To support the deserialization of multiple data types in Java objects, the byte length occupied by the schema content, the total byte length of the serialization result, the schema content, and the serialization result are first read in order from the final result array of serialization; then different deserialization strategies are adopted according to the data type to be deserialized.

[0098] The steps of deserialization in the present application are described as follows:

[0099] Step 4: Initialization

[0100] Since deserialization is the reverse process of serialization, deserialization needs to read the byte length occupied by the schema content, the total byte length occupied by the serialization result, the schema content, and the serialization result in order from the final result array of serialization. The schema content part may contain multiple schemas (enumSchema or objectSchema), so it is necessary to separate the serial number and content of each schema from the schema content part in the byte array, and store the serial number and corresponding enumSchema or objectSchema of the schema in the hash table schemas in the form of key:value.

[0101] Step 5: Generating the deserialized object

[0102] In the process of generating the deserialized object, the readData function is called according to the length of the schema content, the total length of the serialized result, the schema content and the serialized result, to realize the deserialization process.

[0103] 1) Separating the data type and the data value

[0104] In the process of serializing an object, the data type code of the object needs to be stored in the byte array first, and then the serialized result of the object is stored, so in the process of deserialization, the data type code and the data value need to be separated first. Since Motp encodes each data type in one byte, when reading the data type, only the first byte needs to be read, and the data type of the object can be obtained. The bytes in the byte array from the second byte are the data value that needs to be deserialized.

[0105] 2) Checking whether there is an object type that needs to be converted

[0106] In the process of deserialization, there may be an object type that needs to be converted, such as a serialized linked list object that can be converted into an array or a set object after deserialization; a serialized set object that can be converted into an array or a linked list object after deserialization. For the case where there is an object type that needs to be converted, only the corresponding data structure needs to be selected to store the deserialized object for different types, so the remaining steps of deserialization are described in the case where there is no object type that needs to be converted.

[0107] 3) Judging whether it is a direct conversion type

[0108] Motp checks whether the object type code that needs to be deserialized is a direct conversion type, which includes Java's native types (boolean, short, int, etc.), Java's commonly used basic types (String, Date, BigInteger, etc.), and Java's basic type arrays (boolean[], short[], int[], etc.). If the object type that needs to be deserialized is a direct conversion type, Motp obtains the corresponding serialization and deserialization processor for the type (such as MotpInt32Processer for int type), the processor first reads the data stored in the offset position, and then deserializes the object to obtain the deserialized object.

[0109] Take int type as an example, Motp checks that int type is a direct conversion type, then gets the processor MotpInt32Processer of int type, and calls readInt function to deserialize the object of int type.

[0110] readInt function firstly calls assetOffset function to ensure that the offset does not exceed the size of the buffer itself, and since int type occupies 4 bytes, the parameter of assetOffset is 4. Then the first byte in the byte array is obtained, and after shifting left 8 bits, the next byte is or operated, and so on. After shifting left 4 times and or operating 3 times, the value of int is obtained.

[0111] 4) Determine whether it is a basic set, file or enumeration type

[0112] Motp determines whether the object type code to be deserialized is a linked list List type, a Map type, a set Set type, a file File type or an enumeration Enum type.

[0113] (1) Basic set

[0114] If the object type code to be deserialized is a linked list List type, a Map type or a set Set type, firstly read the number of elements from the serialization result array and convert it to int type, then traverse (control the number of traversals by reading the number of elements) to read the content of each element and call readData function to deserialize, and then store the obtained set elements in the corresponding type of basic set. In particular, since each element in Map is in the form of key:value, when deserializing Map, the key and value need to be deserialized by calling readData function respectively.

[0115] (2) File type

[0116] If the object type code to be deserialized is a file type, firstly read the byte length occupied by the file name length from the serialization result array, then read the file name according to the length, then read the byte length occupied by the file content, and read the file content according to the length.

[0117] (3) Enumeration type

[0118] If the object type to be deserialized is an enumeration type, first read the schema order number corresponding to the enumeration type from the serialization result array, then read the order number corresponding to the value of the enumeration variable in the enumeration type definition, then obtain the enumSchema corresponding to the enumeration type from the hash table schemas according to the schema order number, and finally obtain the name of the enumeration value from the enumSchema according to the order number corresponding to the value in the enumeration type definition.

[0119] 5) Determine whether it is a class object

[0120] If the object to be deserialized is a class object, then:

[0121] (1) Separate the schema order number and the byte length occupied by the serialization content of the object from the serialization result array in order.

[0122] (2) Obtain the objectSchema corresponding to the class from the hash table schemas according to the schema order number.

[0123] (3) Obtain the class name from the objectSchema corresponding to the class, so as to create an object of the class through the forName function of Java.

[0124] (4) Traverse and read the order number of each class attribute from the serialization result array, obtain the name of the attribute through the order number, then call the readData function to deserialize the value of the attribute, and set the value of the attribute in the created class object. After the traversal of the class attributes is completed, the deserialized class object is obtained.

[0125] The specific implementation code is as follows:

[0126] For the serialization part:

[0127] 1) The code of the entire serialization flow is as follows:

[0128]

[0129] 2) The code of the function appendData for implementing the serialization function is as follows:

[0130]

[0131]

[0132]

[0133] 3) In the step 2 of serialization, it is needed to judge whether the serialized object is a direct conversion type, and according to the object type, the corresponding Motp serialization deserialization processor is selected, and the corresponding relationship between the data type and the serialization deserialization processor is as follows:

[0134]

[0135]

[0136] 4) The serialization code for int type is as follows:

[0137]

[0138] 5) The serialization code for List type is as follows:

[0139]

[0140] 6) The serialization code for class object is as follows:

[0141]

[0142]

[0143] For the deserialization part:

[0144] 1) The whole deserialization flow code is as follows:

[0145]

[0146] 2) The function readData code for realizing the deserialization function is as follows:

[0147]

[0148]

[0149]

[0150] 3) The deserialization code for int type is as follows:

[0151]

[0152] 4) The deserialization code for List type is as follows:

[0153]

[0154]

[0155]

[0156] 6) Deserialization code for the class object is as follows:

[0157]

[0158]

[0159]

[0160] Although the embodiments of the present application have been shown and described above, it should be understood that the above-described embodiments are exemplary and should not be construed as limiting the present application, and those ordinarily skilled in the art can make changes, modifications, replacements and variations to the above-described embodiments without departing from the principles and spirit of the present application within the scope of the present application.

Claims

1. A method for Java object serialization and deserialization, characterized in that: the serialization comprises the following steps: Step 1: initializing an object schema for storing enumeration type and class object structure in serialization and an object dataBuffer for encapsulating a byte array class; a byte array buffer in the object dataBuffer is used for storing the result of the present serialization; Step 2: generating serialization data firstly, checking the object to be serialized, checking whether it is AsType, if it is AsType, converting the value into the type corresponding to AsType; the AsType is a set of interfaces defined for transmission optimization, which can realize processing one data type as another data type; then, taking different serialization strategies according to the object type to be serialized to obtain a serialization result array buffer; the object type is: direct conversion type, basic collection, file or enumeration type, class object; Step 3: generating result data by using the obtained serialization result array buffer; the deserialization comprises the following steps: Step 4: reading out the byte length occupied by the content of the schema, the total byte length of the serialization result, the content of the schema and the serialization result in the final result array of the serialization in order; Step 5: taking different deserialization strategies according to the data type to be deserialized to realize the deserialization process.

2. The method for Java object serialization and deserialization according to claim 1, characterized in that: In step 1, the object schema is divided into two types: EnumSchema for storing enumeration type structure and ObjectSchema for storing class object structure.

3. The method for Java object serialization and deserialization according to claim 1, characterized in that: In step 2, the direct conversion type includes Java native type, Java common basic type and Java basic type array; if the object type is direct conversion type, the type code of the object is stored in the byte array buffer in the dataBuffer first, then the object is serialized, and the serialized result is stored in the byte array.

4. The method for Java object serialization and deserialization according to claim 2, characterized in that: In step 2, the basic collection, file or enumeration type includes linked list List type, array Array type, Map type, set Set type, file File type or enumeration Enum type; if the object type is linked list type, array type, Map type or set type, the byte code corresponding to the current object type is stored in the byte array buffer first, then the element number of the object is stored, and finally each element in the object is serialized; if the object type is file type, the byte code corresponding to the current object type is stored in the byte array buffer first, then the file name length and the serialized file name are stored in the byte array buffer, and finally the byte length occupied by the file content and the serialized file content are stored. If the object type is an enumeration type, first determine whether the enumSchema corresponding to the enumeration type already exists, if not, then the name of the enumeration type and the corresponding enumSchema are stored in the hash table schemas in the form of key:value, wherein the enumSchema stores each name in the enumeration type definition and the order number of the name in the definition; if the enumSchema corresponding to the enumeration type already exists, first store the byte code corresponding to the enumeration type in the byte array buffer, then store the order number of the enumSchema corresponding to the enumeration type in the hash table schemas, and finally store the order number of the value of the enumeration variable in the enumeration type definition.

5. The method for Java object serialization and deserialization according to claim 2, wherein: In step 2, when serializing the class object, first determine whether the objectSchema corresponding to the class already exists, if not, then the class name and the corresponding objectSchema are stored in the hash table schemas in the form of key:value, wherein the objectSchema stores the name of all available properties of the class and the order number of the property; if the objectSchema corresponding to the class already exists, then: First, a byte array objectContent is created; All available properties in the class are traversed, for each property, first store the order number of the property in the class in the byte array objectContent, then serialize each property; after the traversal, all properties of the entire class are serialized and the result is stored in the byte array objectContent; In the byte array buffer, first store the type code of the class object, then store the order number of the objectSchema corresponding to the class in the hash table schemas, then store the size of the byte array objectContent, and finally store the entire byte array objectContent, the serialization process of the entire class object ends.

6. The method for Java object serialization and deserialization according to claim 1, wherein: In step 3, the process of generating the result data of the entire serialization process is: Step 3.1: create a byte array object rst to encapsulate the class object, the byte array in the object is used to store the result data of this serialization, create a byte array schemaBytes to store the contents of all schemas, and create a byte array dataBytes to store the contents of the serialized result array buffer; Step 3.2: traverse all schemas in this serialization process, for each schema, first store the order number of the schema in the byte array schemaBytes, then store the type of the schema, then store the number of bytes occupied by the contents of the schema, and finally store the contents of the schema; after the traversal, the result data of all schemas is stored in the byte array schemaBytes; Step 3.3: Copy the content of the serialization result array buffer to the byte array dataBytes; Step 3.4: In the byte array of the object rst, first store the length of the byte array schemaBytes, then store the length of the byte array dataBytes, then store the content of the byte array schemaBytes, and finally store the content of the byte array dataBytes. The result data of the entire serialization process is stored.

7. The method for Java object serialization and deserialization according to claim 1, wherein: In step 4, if the content part of the schema contains multiple schemas, separate the sequence number and content of each schema from the schema content part in the byte array, and store the sequence number and corresponding enumSchema or objectSchema of the schema in the hash table schemas in the form of key:value.

8. The method for Java object serialization and deserialization according to claim 1, wherein: In step 5, the deserialization process includes: Step 5.1: Read the first byte of the byte array to obtain the data type of the object, and start from the second byte of the byte array to obtain the data value that needs to be deserialized; Step 5.2: Check if the object type that needs to be converted exists, if it exists, select the corresponding data structure to store the deserialized object according to the different types; Step 5.3: Check if the object type that needs to be deserialized is a direct conversion type, if it is, get the corresponding serialization and deserialization processor of the type, the processor first reads the data stored in the offset position, and then deserializes the object to obtain the deserialized object; Step 5.4: Determine whether the object type that needs to be deserialized is a linked list List type, a Map type, a Set type, a File type, or an Enum type; If the object type that needs to be deserialized is a linked list List type, a Map type, or a Set type, first read the number of elements from the serialization result array and convert it to an int type, then read and deserialize the content of each element, and then store the obtained elements in the corresponding type of object; In the Map type, each element is in the form of key:value, and when deserializing the Map, the key and value are deserialized respectively; If the object type that needs to be deserialized is a file type, read the byte length occupied by the file name length from the serialization result array, then read the file name according to the length, then read the byte length occupied by the file content, and then read the file content according to the length; If the object type that needs to be deserialized is an enumeration type, read the schema sequence number corresponding to the enumeration type from the serialization result array, then read the value of the enumeration variable in the enumeration type definition corresponding to the sequence number, then get the enumSchema corresponding to the enumeration type from the hash table schemas according to the schema sequence number, and finally get the name of the enumeration value from the enumSchema according to the sequence number corresponding to the value in the enumeration type definition; Step 5.5: judging whether the object type needing deserialization is a class object, if yes, firstly separating the schema sequence number and the byte length occupied by the serialization content of the object from the serialization result array in order; then obtaining the objectSchema corresponding to the class from the hash table schemas according to the schema sequence number; then obtaining the class name from the objectSchema corresponding to the class, thereby creating the object of the class through the forName function of Java; traversing and reading the sequence number of each class attribute from the serialization result array, obtaining the name of the attribute through the sequence number, then deserializing to obtain the value of the attribute, and setting the value of the attribute in the created class object; after the traversal of the class attribute ends, the deserialized class object is obtained.

9. An electronic device, comprising a processor and a memory storing a program, the program comprising instructions to: The instructions, when executed by the processor, cause the processor to perform the method of any one of claims 1-8.

10. A computer readable storage medium storing a program, the program comprising instructions to: The instructions, when executed by the processor of the electronic device, cause the electronic device to perform the method of any one of claims 1-8.

Images