Method, device and equipment for processing object record data based on backend service

Abstract

The present invention provides a method for processing object record data based on backend services, which can be applied to the field of data processing technology. The method for processing object record data based on backend services includes: processing the object record data in blocks to obtain multiple fields; when it is determined that the type of the field is the first type, obtaining the first column storage space corresponding to the field from the column storage space; when it is determined that the field matches the first reference value, updating the quantity identifier stored in the first storage block; when it is determined that the field does not match the first reference value, determining the target first storage block from multiple first storage blocks according to the storage block arrangement order, and storing the difference between the field and the reference value in the target first storage block. The present invention also provides a device and apparatus for processing object record data based on backend services.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device and equipment for processing object record data based on backend services. Background Art

[0002] With the rapid development of science and technology, the number of Internet users continues to increase, and the object record data generated by the Internet backend services is gradually increasing, which increases the storage cost.

[0003] Traditional methods to reduce storage costs include data compression, hot and cold separation, disk storage, etc. For the Internet industry, the storage of object record data in Internet backend services is difficult to reduce storage costs using traditional methods because the amount of object record data in Internet backend services far exceeds the amount of traditional object record data. Summary of the invention

[0004] In view of the above problems, the present invention provides a method, apparatus and device for processing object record data based on backend services.

[0005] According to a first aspect of the present invention, there is provided a method for processing object record data based on a backend service, comprising: performing block processing on the object record data to obtain a plurality of fields; when it is determined that the type of the field is a first type, obtaining a first column storage space corresponding to the field from the column storage space; when it is determined that the field matches a first reference value, updating a quantity identifier stored in a first storage block; when it is determined that the field does not match the first reference value, determining a target first storage block from a plurality of first storage blocks according to an arrangement order of the storage blocks, and storing a difference between the field and the reference value in the target first storage block.

[0006] The first column storage space includes a plurality of first storage blocks and a first first storage block. The first first storage block includes a first reference value and a quantity identifier. The quantity identifier is used to represent the quantity of fields that are the same as the reference value.

[0007] The first column storage space also includes a storage type identifier, and the number of the first column storage space is multiple.

[0008] According to an embodiment of the present invention, acquiring a first column storage space corresponding to a field from a column storage space includes: determining a first column storage space corresponding to the type of the field according to storage type identifiers of respective first column storage spaces.

[0009] A field includes a field value.

[0010] According to an embodiment of the present invention, updating the quantity identifier stored in the first storage block includes: determining a first storage difference based on a field value of a field and a first reference value of the first storage block; and accumulating the quantity identifier of the first storage block based on the first storage difference to complete the operation of updating the quantity identifier.

[0011] The column storage space also includes a second column storage space, the second column storage space includes a plurality of second storage blocks and a second first storage block, and the second first storage block includes a second reference value and a reference position identifier.

[0012] According to an embodiment of the present invention, a method for processing object record data based on a backend service also includes: when it is determined that the type of a field is a second type, obtaining a second column storage space corresponding to the field from the column storage space; obtaining a second storage difference value based on the field value of the field and a second reference value of a second first storage block; based on the second storage difference value, determining a target location identifier that matches the second storage difference value from a plurality of location identifiers; and storing the second storage difference value in a target second storage block that matches the target location identifier.

[0013] The position identifier is determined based on the storage block arrangement sequence and the reference position identifier, and the position identifier corresponds to the second storage block.

[0014] According to an embodiment of the present invention, the method for processing object record data based on the backend service further includes: when it is determined that the type of the field is the third type, storing the field value of the field in a third column storage space of the column storage space.

[0015] According to an embodiment of the present invention, the method for processing object record data based on the back-end service also includes: when a first storage difference between the field value of a field and a first reference value is greater than the number of rows in the first column storage space, setting the field value of the field to a new first reference value and setting the new quantity identifier to a preset quantity identifier; based on the new first reference value and the new quantity identifier, creating a new first storage block and multiple new first storage blocks in the first column storage space.

[0016] According to an embodiment of the present invention, a method for processing object record data based on a back-end service includes: in response to a user's data query request, determining the object record data to be queried from the data query request; when the type of the query field is the first type, determining a first read difference between the field value of the query field and a first reference value of the first storage block; determining a target number of storage rows based on a quantity identifier of the first storage block and the first read difference; and obtaining, based on the target number of storage rows, field values ​​of other fields of the object record data to be queried that belong to the same as the query field.

[0017] The object record data to be queried includes a query field. The number of the second storage block includes multiple.

[0018] According to an embodiment of the present invention, based on the target number of storage rows, field values ​​of other fields of the object record data to be queried that belong to the same object as the query field are obtained, including: when the type of the other fields is the second type, based on the target number of storage rows and the respective reference position identifiers of the multiple second storage blocks, a target second storage block is screened from the multiple second storage blocks in the second column storage space; based on the target second storage block, the field value corresponding to the target number of storage rows is determined as the field value of the other fields.

[0019] A second aspect of the present invention provides a device for processing object record data based on a backend service, comprising:

[0020] The block module is used to process the object record data in blocks to obtain multiple fields.

[0021] The acquisition module is used to acquire a first column storage space corresponding to the field from the column storage space when it is determined that the type of the field is the first type.

[0022] According to an embodiment of the present invention, the first column storage space includes multiple first storage blocks and a first first storage block, the first first storage block includes a first reference value and a quantity identifier, and the quantity identifier is used to represent the number of fields that are the same as the reference value.

[0023] The updating module is used to update the quantity identifier stored in the first storage block when it is determined that the field matches the first reference value.

[0024] The storage module is used to determine a target first storage block from a plurality of first storage blocks according to the storage block arrangement sequence when it is determined that the field does not match the first reference value, and store the field in the target first storage block.

[0025] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0026] The fourth aspect of the present invention further provides a computer-readable storage medium on which a computer program or instruction is stored, and the steps of the above method are implemented when the above computer program or instruction is executed by a processor.

[0027] The fifth aspect of the present invention also provides a computer program product, including a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0028] According to an embodiment of the present invention, by updating the quantity identifier in the first storage block, fields matching the first reference value can be stored. When the number of matching fields is large, a large number of repeated identical fields can be stored by updating the quantity identifier, saving a large amount of storage space. When a field does not match the first reference value, since the difference between the field and the first reference value is smaller than the field, storage space can be further saved and storage costs can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0030] Figure 1 An application scenario diagram is shown of a method, apparatus and device for processing object record data based on backend services according to an embodiment of the present invention.

[0031] Figure 2 A flow chart of a method for processing object record data based on backend services according to an embodiment of the present invention is shown.

[0032] Figure 3 A flowchart of a method for processing object record data based on backend services according to another embodiment of the present invention is shown.

[0033] Figure 4 A flowchart of a method for processing object record data based on backend services according to yet another embodiment of the present invention is shown.

[0034] Figure 5 A structural block diagram of a device for processing object record data based on backend services according to an embodiment of the present invention is shown.

[0035] Figure 6 A block diagram of an electronic device suitable for implementing a method for processing object record data based on a backend service according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0036] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0037] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0038] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0039] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0040] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, invention and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0041] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present invention provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0042] In the process of implementing the present invention, it is found that the existing database for storing object record data of Internet back-end services includes a column storage database. Although the existing column storage database is stored in columns, its block division method is only by column, and no fine block processing is performed, resulting in poor storage cost reduction effect.

[0043] An embodiment of the present invention provides a method for processing object record data based on a backend service, which processes the object record data in blocks to obtain multiple fields; when it is determined that the type of the field is a first type, a first column storage space corresponding to the field is obtained from the column storage space; when it is determined that the field matches a first reference value, the quantity identifier stored in the first storage block is updated; when it is determined that the field does not match the first reference value, a target first storage block is determined from multiple first storage blocks according to the storage block arrangement order, and the difference between the field and the reference value is stored in the target first storage block.

[0044] According to an embodiment of the present invention, the first column storage space includes multiple first storage blocks and a first first storage block, the first first storage block includes a first reference value and a quantity identifier, and the quantity identifier is used to represent the number of fields that are the same as the reference value.

[0045] Figure 1 An application scenario diagram is shown of a method, apparatus and device for processing object record data based on backend services according to an embodiment of the present invention.

[0046] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables.

[0047] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0048] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0049] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0050] It should be noted that the method for processing object record data based on backend services provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the device for processing object record data based on backend services provided in the embodiment of the present invention can generally be set in the server 105. The method for processing object record data based on backend services provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the device for processing object record data based on backend services provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0051] It should be understood that Figure 1 The number of the first terminal device, the second terminal device, the third terminal device, the network and the server in the embodiment is only for illustration. According to the implementation requirements, there may be any number of the first terminal device, the second terminal device, the third terminal device, the network and the server.

[0052] The following will be based on Figure 1 The scene described by Figure 2~Figure 4 A method for processing object record data based on backend services in an embodiment of the invention is described in detail.

[0053] Figure 2 A flow chart of a method for processing object record data based on backend services according to an embodiment of the present invention is shown.

[0054] like Figure 2 As shown, the method for processing object record data based on backend service in this embodiment includes operations S210 to S240.

[0055] In operation S210, the object record data is divided into blocks to obtain a plurality of fields.

[0056] According to an embodiment of the present invention, the object record data may be the object record data obtained through user authorization. The object record data may be the Internet user behavior record data within a preset period of time, and the fields may include record ID, object ID, behavior type, behavior number, behavior content, and behavior time.

[0057] According to an embodiment of the present invention, the object record data may be the record data when a user uses the Internet. The number of object record data may include multiple, and the multiple object record data may be the record data when multiple users use the Internet, or the record data when one user uses the Internet.

[0058] In operation S220 , when it is determined that the type of the field is the first type, a first column storage space corresponding to the field is acquired from the column storage space.

[0059] According to an embodiment of the present invention, the first column storage space includes multiple first storage blocks and a first first storage block, the first first storage block includes a first reference value and a quantity identifier, and the quantity identifier is used to represent the number of fields that are the same as the reference value.

[0060] According to an embodiment of the present invention, the column storage space may correspond to the preset time period one by one. For example, the object record data of each day may be stored in the same column storage space.

[0061] According to an embodiment of the present invention, the first type may be any one of a behavior type, a behavior number, an object ID, and a time, and the behavior type, the behavior number, the object ID, and the time are all digital types. When the type of the field is a behavior type or a behavior number, the values ​​of the fields of the plurality of object record data are highly repetitive. When the type of the field is a time or an object ID, the values ​​of the fields of the plurality of object record data have certain regularities, such as a linear growth regularity.

[0062] According to an embodiment of the present invention, the behavior type may be obtained according to the actual behavior classification. For example, the actual behavior includes clicking a dialog box and sending a message, and the corresponding behavior types may be 001 and 002. When it is recognized that the user clicks the dialog box, the obtained behavior type is 002.

[0063] According to an embodiment of the present invention, the column storage space refers to a space in a storage system that stores object record data in the form of column storage.

[0064] According to an embodiment of the present invention, a query statement may be used to simultaneously query the first column storage space matching each field from the column storage space.

[0065] In operation S230, when it is determined that the field matches the first reference value, the quantity identifier stored in the first storage block is updated.

[0066] According to an embodiment of the present invention, a value associated with a field may be compared with a first reference value, and when the comparison result indicates that the two are the same, an update statement may be used to update the quantity identifier.

[0067] In operation S240, when it is determined that the field does not match the first reference value, a target first storage block is determined from the plurality of first storage blocks according to the storage block arrangement order, and a difference between the field and the first reference value is stored in the target first storage block.

[0068] According to an embodiment of the present invention, the storage block arrangement order can be set according to demand.

[0069] According to an embodiment of the present invention, the value associated with the field can be compared with the first reference value. When the comparison result indicates that the two are different, a first storage block with empty storage is selected from a plurality of first storage blocks according to the storage block arrangement order and used as the target first storage block. After the target first storage block is determined, the content of the first storage block is replaced with the value associated with the field to complete the storage of the field.

[0070] According to an embodiment of the present invention, by updating the quantity identifier in the first storage block, fields matching the first reference value can be stored. When the number of matching fields is large, a large number of repeated identical fields can be stored by updating the quantity identifier, saving a large amount of storage space. When a field does not match the first reference value, since the difference between the field and the first reference value is smaller than the field, storage space can be further saved and storage costs can be reduced.

[0071] According to an embodiment of the present invention, the first column storage space further includes a storage type identifier, and the number of the first column storage space is multiple.

[0072] According to an embodiment of the present invention, obtaining a first column storage space corresponding to a field from a column storage space may include the following operations: determining a first column storage space corresponding to the type of the field according to storage type identifiers of respective first column storage spaces.

[0073] According to an embodiment of the present invention, the first column storage space corresponding to the storage type identifier and the field may be screened from the column storage space according to a preset arrangement order of the plurality of first column storage spaces.

[0074] According to an embodiment of the present invention, the first column storage space corresponding to each field may be determined in sequence according to a preset field sorting order, and the number of the first column storage space corresponding to each field may be single.

[0075] According to an embodiment of the present invention, a field includes a field value.

[0076] According to an embodiment of the present invention, updating the quantity identifier stored in the first storage block may include the following operations: determining a first storage difference based on the field value of the field and a first reference value of the first storage block; accumulating the quantity identifier of the first storage block based on the first storage difference to complete the operation of updating the quantity identifier.

[0077] According to an embodiment of the present invention, the number of the first storage block may be single, and the first reference value may be obtained according to the first object record data within a preset period. After determining the first reference value, the quantity identifier is set to a preset value, such as 1.

[0078] According to an embodiment of the present invention, the field value of the field can be subtracted from the first reference value of the first storage block to obtain a first storage difference value. When the first storage difference value meets the storage requirement, the quantity identifier of the first storage block is accumulated to complete the operation of updating the quantity identifier.

[0079] According to the embodiment of the present invention, the storage of the field is completed by updating the quantity identifier instead of storing the field value of the field, which can save storage space.

[0080] According to an embodiment of the present invention, the column storage space further includes a second column storage space, the second column storage space includes a plurality of second storage blocks and a second first storage block, and the second first storage block includes a second reference value and a reference position identifier.

[0081] According to an embodiment of the present invention, the position identifier is determined based on the storage block arrangement sequence and the reference position identifier, and the position identifier corresponds to the second storage block.

[0082] According to an embodiment of the present invention, the method for processing object record data based on the back-end service may also include the following operations: when a first storage difference between the field value of a field and a first reference value is greater than the number of rows in the first column storage space, the field value of the field is set to a new first reference value, and the new quantity identifier is set to a preset quantity identifier; based on the new first reference value and the new quantity identifier, a new first storage block and multiple new first storage blocks are created in the first column storage space.

[0083] According to an embodiment of the present invention, the preset quantity identifier can be set according to actual needs, for example, it can be 1. The new reference value and the new quantity identifier can be written into the first storage block, and the new multiple first storage blocks are all set to the identifier to be stored, for example, empty, to complete the creation.

[0084] Figure 3 A flowchart of a method for processing object record data based on backend services according to another embodiment of the present invention is shown.

[0085] like Figure 3As shown, the method for processing object record data based on backend service in this embodiment includes operations S310 to S340.

[0086] In operation S310 , when it is determined that the type of the field is the second type, a second column storage space corresponding to the field is acquired from the column storage space.

[0087] In operation S320, a second storage difference value is obtained according to the field value of the field and a second reference value of the second first storage block.

[0088] In operation S330, based on the second stored difference value, a target location identifier matching the second stored difference value is determined from a plurality of location identifiers.

[0089] In operation S340, the second stored difference value is stored in a target second storage block that matches the target location identifier.

[0090] According to an embodiment of the present invention, the second type may be any one of a behavior type, a behavior number, an object ID, and a time in a high-concurrency environment.

[0091] According to an embodiment of the present invention, a query statement may be used to query a second column storage space that matches the field from the column storage space. The second column storage space may also include a second storage type identifier.

[0092] According to an embodiment of the present invention, the second column storage space may include multiple column storage subspaces, each of which may include a second first storage block and multiple second storage blocks. In the case where there is no second storage block to be stored in the column storage subspace, a new column storage subspace may be created, and the field value of the field to be stored may be used as the second reference value of the second first storage block included in the new column storage subspace.

[0093] According to an embodiment of the present invention, a data structure of a column storage subspace may be set, which may specifically include the number of second storage blocks, the range of second reference values, and the range of second storage difference values. A data structure of a second column storage space may also be set, which may specifically include the number of column storage subspaces. The number of column storage subspaces may also be configured by adding a new field. For example, the new field is set to 4, indicating that the second column storage space includes at most 4 column storage subspaces, and the number of second reference values ​​is at most 4. When the second storage space can store 40,000 field values, every 10,000 field values ​​correspond to a second reference value. When the second storage space can store 38,000 field values, the first 30,000 field values ​​correspond to three second reference values, and 8,000 field values ​​correspond to one second reference value. By setting the data structure of the column storage subspace, the data structure of the second column storage space, and the new field, the subsequent storage and reading of the field value are completed.

[0094] According to an embodiment of the present invention, a second column storage space matching the type of the field can be determined from the second storage type identifiers of each of the plurality of second column storage spaces. A target column storage subspace is determined from the plurality of column storage subspaces of the second column storage space in the following manner: when a second reference value exists in the second first storage block and a second storage block to be stored exists in the column storage subspace, the column storage subspace is determined as the target column storage subspace.

[0095] According to an embodiment of the present invention, the field value of the field may be subtracted from the second reference value of the second first storage block of the target column storage subspace to obtain a second storage difference value.

[0096] According to an embodiment of the present invention, multiple position identifiers can be obtained by sorting in a preset position order based on the reference position identifier. For example, the reference position identifier is 1000 rows, and there are five position identifiers, namely 1, 2, 3, 4 and 5, which respectively represent the distance rows from the reference position identifier.

[0097] According to an embodiment of the present invention, multiple location identifiers can be accessed in sequence according to a preset location order. When the second storage block corresponding to the location identifier does not store a field value, the location identifier is determined as the target location identifier, and the second storage block is determined as the target second storage block.

[0098] For example, the number of object record data is 1,000,000, and the first column storage space can store 62,500 field values. Each object record data can be processed in blocks, and for the time type field, 1,000,000 field values ​​can be obtained. The accuracy of the field value is in seconds, for example, the field value can be represented by the Greenwich time stamp. The 1,000,000 time type field values ​​can be divided into 16 groups, each group including 62,500 field values ​​arranged in chronological order. The quantity identifier of the first storage block included in the first column storage space can be 1, and the first reference value of the first storage block can be the first field value in each group. The remaining multiple field values ​​in each group can be stored in multiple first storage blocks in chronological order.

[0099] For the traditional storage method, each traditional first storage block stores a field value. Table 1-a shows the traditional first column storage space and the occupied space of the traditional first storage block, and the occupied space is expressed in bytes. Table 1-b shows the first column storage space included in the column storage space. The Greenwich time stamp of 2024-04-17 23:59:59 is 1713369599, and 1713369599 as the field value requires 4 bytes. Through the traditional storage method, 1000000 consecutive Greenwich time stamps after 2024-04-17 23:59:59 require 4000000 bytes, that is, 3.81 megabytes (M). Through the column storage space for storage, the first base value included in the first storage block of the first column storage space is 1713369600, and the quantity is identified as 1.

[0100] For each of the 62499 field values ​​after 1713369600, a target first storage block is determined from the 62499 first storage blocks according to the storage block arrangement order, and the difference between the field value and the first reference value is stored in the target first storage block. Each first storage block occupies 2 bytes, and each first column storage space occupies 4+62500*2 bytes, that is, 122 kilobytes (KB). It can be seen that using the first column storage space can reduce the occupied space.

[0101] When the first reference value is 1713369600 and the field values ​​of the three fields are all 1713369600, it is determined that the field matches the first reference value, and the three first stored difference values ​​are obtained, all of which are 0. Based on the first stored difference value, the quantity identifier is accumulated, and the updated quantity identifier is 4.

[0102] When the first reference value is 1713369600 and the field value of the field is 1713369603, it is determined that the field does not match the first reference value, the target first storage block is determined from the multiple first storage blocks according to the storage block arrangement order, and the difference 3 between the field and the first reference value is stored in the target first storage block.

[0103] Under high concurrency conditions, hundreds of object record data can be generated per second. Table 1-c is the second column storage space. The second column storage space may include four column storage subspaces. The column storage subspace may include 1 second first storage block and 155 second storage blocks. Each second first storage block occupies 4 bytes, the reference position identifier occupies 2 bytes, and the second storage block occupies 1 byte. When 62,500 field values ​​are stored in the second column storage space, the second reference value of the first second first storage block is 1713369600. When the field value of the field is 1713369605, the second storage difference is 5 according to the field value of the field and the second reference value of the second first storage block. Based on the second storage difference, the target position identifier 5 that matches the second storage difference is determined from multiple position identifiers, and the second storage difference 5 is stored in the target second storage block that matches the target position identifier. In the case that the second storage block to be stored does not exist in the first column storage subspace, a second column storage subspace can be created, and the corresponding second reference value of the second first storage block is 1713369756, and the reference position identifier is 157. When the field value of the field is 1713369757, the second storage difference value is obtained as 1 according to the field value of the field and the second reference value of the second first storage block, and the second storage difference value 5 is stored in the target second storage block that matches the target position identifier. The value range of the second storage difference value stored in the second storage block is 1-155, and the second column storage space occupies 400*4+400*2+62100 bytes, that is, 64.5KB.

[0104] Table 1-a

[0105]

[0106] Table 1-b

[0107]

[0108] Table 1-c

[0109]

[0110] According to an embodiment of the present invention, the method for processing object record data based on the backend service may further include the following operation: when it is determined that the type of the field is the third type, storing the field value of the field in the third column storage space of the column storage space.

[0111] According to an embodiment of the present invention, a query statement may be used to query a third column storage space that matches the field from the column storage space.

[0112] According to an embodiment of the present invention, the third type may be behavior content, and the behavior content is a character type. The third column storage space includes a plurality of third storage blocks that are sequentially sorted. When a plurality of fields are received, the field values ​​may be sequentially stored in the third storage block of the third column storage space.

[0113] Figure 4 A flowchart of a method for processing object record data based on backend services according to yet another embodiment of the present invention is shown.

[0114] like Figure 4 As shown, the method for processing object record data based on backend service in this embodiment includes operations S410 to S440.

[0115] In operation S410 , in response to a data query request from a user, object record data to be queried is determined from the data query request.

[0116] In operation S420, in a case where the type of the query field is the first type, a first read difference value between a field value of the query field and a first reference value of a first head storage block is determined.

[0117] In operation S430, the target storage row number is determined according to the number identifier of the first storage block and the first read difference.

[0118] In operation S440, according to the target storage row number, the field values ​​of other fields belonging to the object record data to be queried and the query field are obtained.

[0119] According to an embodiment of the present invention, the object record data to be queried includes a query field.

[0120] According to an embodiment of the present invention, the data query request may further include information related to the object record data to be queried, for example, location information of the first column storage space.

[0121] According to an embodiment of the present invention, a data query request may be parsed according to a preset data format to obtain object record data to be queried.

[0122] According to an embodiment of the present invention, a geometric operation such as a subtraction operation may be performed on the field value of the query field and the first reference value of the first storage block to obtain a first read difference value.

[0123] According to an embodiment of the present invention, the number identifier of the first storage block and the first read difference value may be added together to obtain the target number of storage rows.

[0124] According to an embodiment of the present invention, a preset time period may include multiple data query requests, which may be multiple data query requests from the same user or multiple data query requests from multiple users. The data query requests of the users may be responded to in sequence according to the time sequence of the data query requests.

[0125] According to an embodiment of the present invention, other storage spaces corresponding to other fields can be obtained from the column storage space according to the types of other fields. In the case where the type of other fields is the third type, the third column storage space corresponding to other fields can be obtained, and the target third storage block corresponding to the target storage row number can be found from the third column storage space, and the field value stored in the target third storage block can be used as the field value of other fields. In the case where the type of other fields is the second type, the second column storage space corresponding to other fields can be obtained, and the target storage block can be determined according to the target storage row number and the reference position identifier of the second first storage block included in the second column storage space, and the position identifier of the second storage block, and the field value of other fields can be obtained based on the field value stored in the target storage block. In the case where the type of other fields is the first type, the first column storage space corresponding to other fields can be obtained, and the target storage block can be obtained according to the difference H between the target storage row number and the quantity identifier. In the case of 0, the first reference value can be used as the field value of other fields. When H is not 0, the Hth second storage block can be obtained from multiple first storage blocks according to the arrangement order of the first storage blocks as the target storage block, and the sum of the field value stored in the target storage block and the first reference value is used as the field value of other fields.

[0126] According to an embodiment of the present invention, the field values ​​of other fields are determined by the target number of storage rows. Since there is no need to sequentially access other storage spaces related to other fields, it is more efficient to obtain the field values ​​of other fields.

[0127] According to an embodiment of the present invention, the number of the second first storage blocks includes multiple.

[0128] According to an embodiment of the present invention, obtaining the field values ​​of other fields of the object record data to be queried that are the same as the query field based on the target storage row number can include the following operations: when the type of the other fields is the second type, screening the target second storage block from the multiple second storage blocks in the second column storage space based on the target storage row number and the respective reference position identifiers of the multiple second storage blocks; based on the target second storage block, determining the field value corresponding to the target storage row number as the field value of the other fields.

[0129] According to an embodiment of the present invention, the target storage row number can be subtracted from the respective reference position identifiers of the plurality of second first storage blocks to obtain a plurality of comparison difference values. The comparison difference value with the smallest value is selected from the plurality of comparison difference values ​​as the target comparison difference value. The second first storage block corresponding to the target comparison difference value is used as the target second first storage block.

[0130] According to an embodiment of the present invention, when the target comparison difference is 0, the second reference value of the target second first storage block is used as the field value of other fields. When the target comparison difference is not 0, the target second storage block corresponding to the target comparison interpolation can be determined from multiple second storage blocks in sequence according to the reference position identifier of the target second first storage block. The second storage difference stored in the target second storage block is added to the second reference value of the target second first storage block to obtain the field value of other fields.

[0131] According to an embodiment of the present invention, the target second storage block is determined by multiple reference position identifiers. Since the number of multiple reference position identifiers is smaller than the storage quantity of the second column storage space, it is more efficient to determine the field values ​​of other fields based on the target second storage block.

[0132] Based on the above-mentioned method for processing object record data based on backend service, the present invention also provides a device for processing object record data based on backend service. Figure 5 The device is described in detail.

[0133] Figure 5 A structural block diagram of a device for processing object record data based on backend services according to an embodiment of the present invention is shown.

[0134] like Figure 5 As shown, the device for processing object record data based on backend service in this embodiment includes a segmentation module 510 , an acquisition module 520 , an update module 530 and a storage module 540 .

[0135] The block module 510 is used to perform block processing on the object record data to obtain multiple fields.

[0136] The acquisition module 520 is configured to acquire a first column storage space corresponding to the field from the column storage space when it is determined that the type of the field is the first type.

[0137] According to an embodiment of the present invention, the first column storage space includes multiple first storage blocks and a first first storage block, the first first storage block includes a first reference value and a quantity identifier, and the quantity identifier is used to represent the number of fields that are the same as the reference value.

[0138] The updating module 530 is used to update the quantity identifier stored in the first storage block when it is determined that the field matches the first reference value.

[0139] The storage module 540 is used to determine a target first storage block from a plurality of first storage blocks according to the storage block arrangement order when it is determined that the field does not match the first reference value, and store the field in the target first storage block.

[0140] According to an embodiment of the present invention, the first column storage space further includes a storage type identifier, and the number of the first column storage space is multiple.

[0141] According to an embodiment of the present invention, the acquisition module 520 includes: a first determination unit.

[0142] The first determining unit is configured to determine a first column storage space corresponding to a type of the field according to storage type identifiers of each of the plurality of first column storage spaces.

[0143] According to an embodiment of the present invention, a field includes a field value.

[0144] According to an embodiment of the present invention, the updating module 530 includes: an acquiring unit and an accumulating unit.

[0145] The acquisition unit is used to determine a first storage difference value according to a field value of the field and a first reference value of a first storage block.

[0146] The accumulation unit is used to accumulate the quantity identifier of the first storage block based on the first storage difference to complete the operation of updating the quantity identifier.

[0147] According to an embodiment of the present invention, the column storage space further includes a second column storage space, the second column storage space includes a plurality of second storage blocks and a second first storage block, and the second first storage block includes a second reference value and a reference position identifier.

[0148] According to an embodiment of the present invention, the device for processing object record data based on the backend service further includes: a space acquisition module, a difference acquisition module, a first determination module and a difference storage module.

[0149] The space acquisition module is used to acquire a second column storage space corresponding to the field from the column storage space when it is determined that the type of the field is the second type.

[0150] The difference acquisition module is used to acquire a second storage difference according to the field value of the field and the second reference value of the second first storage block.

[0151] The determination module is used to determine, based on the second stored difference, a target location identifier that matches the second stored difference from a plurality of location identifiers.

[0152] The difference storage module is used to store the second stored difference in a target second storage block that matches the target position identifier.

[0153] According to an embodiment of the present invention, the position identifier is determined based on the storage block arrangement sequence and the reference position identifier, and the position identifier corresponds to the second storage block.

[0154] According to an embodiment of the present invention, the device for processing object record data based on the backend service further includes: a field storage module.

[0155] The field storage module is used to store the field value of the field into the third column storage space of the column storage space when it is determined that the type of the field is the third type.

[0156] According to an embodiment of the present invention, the device for processing object record data based on the backend service further includes: a setting module and a creation module.

[0157] A setting module is used to set the field value of the field to a new first reference value and set the new quantity identifier to a preset quantity identifier when a first storage difference between the field value of the field and the first reference value is greater than the number of rows of the first column storage space.

[0158] The creation module is used to create a new first storage block and a plurality of new first storage blocks in the first column storage space according to the new first reference value and the new quantity identifier.

[0159] According to an embodiment of the present invention, a device for processing object record data based on a backend service includes: a second determination module, a third determination module, a fourth determination module and a field value acquisition module.

[0160] The second determining module is used to respond to a data query request from a user and determine the object record data to be queried from the data query request.

[0161] According to an embodiment of the present invention, the object record data to be queried includes a query field.

[0162] The third determination module is used to determine a first read difference between the field value of the query field and a first reference value of the first storage block when the type of the query field is the first type.

[0163] The fourth determination module is used to determine the target storage row number according to the quantity identifier of the first storage block and the first read difference.

[0164] The field value acquisition module is used to obtain the field values ​​of other fields of the object record data to be queried that are the same as the query field according to the target storage row number.

[0165] According to an embodiment of the present invention, the number of the second first storage blocks includes multiple.

[0166] According to an embodiment of the present invention, the field value acquisition module includes: a screening unit and a second determination unit.

[0167] The screening unit is used to screen the target second first storage block from the multiple second first storage blocks in the second column storage space according to the target storage row number and the respective reference position identifiers of the multiple second first storage blocks when the type of other fields is the second type.

[0168] The second determining unit is used to determine the field value corresponding to the target storage row number as the field value of other fields based on the target second first storage block.

[0169] According to an embodiment of the present invention, any multiple modules of the block module 510, the acquisition module 520, the update module 530 and the storage module 540 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the block module 510, the acquisition module 520, the update module 530 and the storage module 540 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the block module 510, the acquisition module 520, the update module 530 and the storage module 540 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0170] Figure 6 A block diagram of an electronic device suitable for implementing a method for processing object record data based on a backend service according to an embodiment of the present invention is shown.

[0171] like Figure 6As shown, the electronic device according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 to a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0172] In RAM 603, various programs and data required for the operation of the electronic device are stored. The processor 601, ROM 602 and RAM 603 are connected to each other via a bus 604. The processor 601 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. The processor 601 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in the one or more memories.

[0173] According to an embodiment of the present invention, the electronic device may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device may further include one or more of the following components connected to the input / output (I / O) interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0174] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0175] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM602 and / or RAM603 described above and / or one or more memories other than ROM602 and RAM603.

[0176] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the above method provided by the embodiment of the present invention.

[0177] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 601. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0178] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0179] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0180] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0181] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0182] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0183] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for processing object record data based on backend services, characterized in that: The method comprises: The object record data is processed in blocks to obtain multiple fields; In the case where it is determined that the type of the field is the first type, a first column storage space corresponding to the field is acquired from the column storage space, wherein the first column storage space includes a first storage block and a plurality of first storage blocks, the first storage block includes a first reference value and a quantity identifier, and the quantity identifier is used to represent the number of fields having the same reference value; In the case where it is determined that the field matches the first reference value, updating the quantity identifier stored in the first storage block; When it is determined that the field does not match the first reference value, a target first storage block is determined from the plurality of first storage blocks according to the storage block arrangement order, and a difference between the field and the first reference value is stored in the target first storage block.

2. The method according to claim 1, characterized in that The first column storage space further includes a storage type identifier, and the number of the first column storage space is multiple; The acquiring, from the column storage space, a first column storage space corresponding to the field includes: A first column storage space corresponding to the type of the field is determined according to the storage type identifiers of each of the plurality of first column storage spaces.

3. The method according to claim 1, characterized in that The field includes a field value; Wherein, updating the quantity identifier stored in the first storage block includes: Determining a first storage difference value according to the field value of the field and a first reference value of the first storage block; Based on the first stored difference, the quantity identifier of the first storage block is accumulated to complete the operation of updating the quantity identifier.

4. The method according to claim 1, characterized in that: The column storage space further includes a second column storage space, the second column storage space includes a second first storage block and a plurality of second storage blocks, the second first storage block includes a second reference value and a reference position identifier; The method further comprises: When it is determined that the type of the field is the second type, acquiring a second column storage space corresponding to the field from the column storage space; determining a second storage difference value according to the field value of the field and a second reference value of the second first storage block; Based on the second storage difference, determining a target position identifier that matches the second storage difference from a plurality of position identifiers, wherein the position identifier is determined based on the storage block arrangement sequence and a reference position identifier, and the position identifier corresponds to the second storage block; The second stored difference is stored in a target second storage block that matches the target position identifier.

5. The method according to claim 3, characterized in that: The method further comprises: When it is determined that the type of the field is the third type, the field value of the field is stored in a third column storage space of the column storage space.

6. The method according to claim 3, characterized in that The method further comprises: When a first storage difference between the field value of the field and the first reference value is greater than the number of rows of the first column storage space, the field value of the field is set to a new first reference value, and the new quantity identifier is set to a preset quantity identifier; According to the new first reference value and the new quantity identifier, a new first storage block and a plurality of new first storage blocks are created in the first column storage space.

7. The method according to claim 4, characterized in that The method comprises: In response to a data query request from a user, determining object record data to be queried from the data query request, wherein the object record data to be queried includes a query field; In a case where the type of the query field is the first type, determining a first read difference between a field value of the query field and a first reference value of the first first storage block; Determining a target number of storage rows according to the quantity identifier of the first storage block and the first read difference; According to the target storage row number, the field values ​​of other fields of the object record data to be queried that are related to the query field are obtained.

8. The method according to claim 7, characterized in that The number of the second storage blocks includes a plurality; Wherein, obtaining the field values ​​of other fields of the object record data to be queried that are the same as the query field according to the target storage row number includes: When the type of the other field is the second type, selecting a target second first storage block from the plurality of second first storage blocks in the second column storage space according to the target storage row number and the respective reference position identifiers of the plurality of second first storage blocks; Based on the target second first storage block, the field value corresponding to the target storage row number is determined as the field value of the other field.

9. A device for processing object record data based on backend services, characterized in that: The device comprises: The block module is used to process the object record data in blocks to obtain multiple fields; an acquisition module, configured to acquire, when determining that the type of the field is the first type, a first column storage space corresponding to the field from the column storage space, wherein the first column storage space includes a plurality of first storage blocks and a first first storage block, the first first storage block includes a first reference value and a quantity identifier, and the quantity identifier is used to represent the number of fields having the same reference value; an updating module, configured to update the quantity identifier stored in the first storage block when it is determined that the field matches the first reference value; The storage module is used to determine a target first storage block from the plurality of first storage blocks according to the storage block arrangement order when it is determined that the field does not match the first reference value, and store the field in the target first storage block.

10. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

Images