Data splicing method, device, electronic device and readable storage medium
By pre-configuring service interface field data, obtaining and concatenating data concatenation field values, the problem of large code volume and high complexity in existing technologies is solved, and efficient data concatenation is achieved.
Patent Information
- Application Number
- CN202310147024.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-15
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2043-02-15
Smart Images

Figure CN117724859B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a data splicing method, device, electronic device and readable storage medium. Background Art
[0002] At present, the streaming data link of most commercial advertising real-time data first obtains detailed data at event granularity from the distributed publish-subscribe messaging system (Kafka), then extracts and converts the obtained data, performs pre-aggregation services on the processed data, and then performs dimensional splicing, that is, splicing some attribute data into the detailed data, and finally aggregates the spliced data according to business needs to obtain the data required by the business line.
[0003] In the prior art, there is a lack of flexibility when splicing data dimensions. Different codes need to be developed and deployed for attribute values of different dimensions. The development cycle for splicing attribute values of different dimensions is long and the efficiency is low. In addition, as the splicing dimensions increase, the amount of code, code complexity and code maintenance difficulty also increase. Summary of the invention
[0004] Based on the above research, the present invention provides a data splicing method, device, electronic device and readable storage medium, which can reduce the amount of code, reduce the complexity and difficulty of code maintenance, and improve work efficiency.
[0005] The embodiments of the present invention can be implemented in the following ways:
[0006] In a first aspect, an embodiment of the present invention provides a data splicing method, comprising:
[0007] Acquire a target splicing field corresponding to the data to be spliced; the data to be spliced contains a basic field corresponding to the target splicing field and a basic field value of the basic field;
[0008] Determine the target service interface corresponding to the target splicing field and the target interface method of the target service interface according to the preconfigured service interface field data; the service interface field data includes each splicing field and the service interface corresponding to the splicing field and the interface method of the service interface;
[0009] According to the target interface method and the basic field value of the basic field, the target service interface is called to obtain the splicing field value of the target splicing field, and the target splicing field and the splicing field value of the target splicing field are spliced into the data to be spliced.
[0010] In an optional implementation, the calling the target service interface to obtain the spliced field value of the target spliced field according to the target interface method and the base field value of the base field includes:
[0011] Obtaining an input parameter template of the target interface method; the input parameter template is configured with dynamic query parameters of the target interface method;
[0012] Based on the basic field value of the basic field, the dynamic query parameter of the input parameter template is replaced to obtain the input parameter of the target interface method;
[0013] Based on the input parameters of the target interface method, the target interface method is executed, and the target service interface is called to obtain the splicing field value of the target splicing field.
[0014] In an optional implementation, the step of executing the target interface method based on the input parameter of the target interface method and calling the target service interface to obtain the splicing field value of the target splicing field includes:
[0015] Based on the input parameters of the target interface method, the target interface method is executed, and the target service interface is called to obtain query result data; the query result data includes at least one return field and a return field value corresponding to the return field;
[0016] Acquire the return field data of the target interface method, and determine the first return field in the query result data corresponding to the target splicing field according to the return field data; the return field data includes the corresponding relationship data between the target splicing field and the return field of the target interface method;
[0017] The return field value corresponding to the first return field in the query result data is determined as the splicing field value of the target splicing field.
[0018] In an optional implementation manner, the step of splicing the target splicing field and the splicing field value of the target splicing field into the data to be spliced includes:
[0019] According to the return field data of the target interface method, determining a second return field in the query result data corresponding to the basic field; the return field data includes the corresponding relationship data between the basic field and the return field of the target interface method, and the basic field value of the basic field matches the return field value of the corresponding return field;
[0020] Determining target data to be spliced corresponding to the query result data according to the return field value of the second return field;
[0021] The splicing field and the splicing field value of the splicing field are spliced into the target data to be spliced.
[0022] In an optional implementation, before calling the target service interface to obtain the concatenated field value of the target concatenated field according to the target interface method and the base field value of the base field, the method further includes:
[0023] Detect whether there is a splicing field with the same corresponding basic field, target service interface and target interface method in the target splicing field corresponding to the data to be spliced;
[0024] Merge the splicing fields that have the same basic fields, target service interfaces, and target interface methods to obtain a splicing field group;
[0025] The step of calling the target service interface to obtain the spliced field value of the target spliced field according to the target interface method and the basic field value of the basic field includes:
[0026] According to the target interface method corresponding to the splicing field group and the basic field value of the basic field in the to-be-spliced data corresponding to each target splicing field in the splicing field group, the splicing field value of each target splicing field in the splicing field group is obtained.
[0027] In an optional implementation, before calling the target service interface to obtain the concatenated field value of the target concatenated field according to the target interface method and the base field value of the base field, the method further includes:
[0028] Determine a query threshold of the target service interface and a data volume of the to-be-joined data corresponding to the target joint field;
[0029] If the data volume is greater than the query threshold of the target service interface, the to-be-joined data corresponding to the target joint field is split.
[0030] In an optional implementation manner, before obtaining the target splicing field corresponding to the data to be spliced, the method further includes:
[0031] Obtain a file package of the service interface, parse the file package, and obtain the interface method of the service interface, as well as the input parameter structure information and method return structure information of the interface method;
[0032] Based on the interface method of the service interface, the input parameter structure information of the interface method and the method return structure information, a first correspondence between the splicing field and the service interface and the interface method of the service interface is established, a second correspondence between the splicing field and the return field of the corresponding interface method is established, and a third correspondence between the basic field corresponding to the splicing field and the return field of the corresponding interface method is established, and an input parameter template of the splicing field corresponding to the interface method is established;
[0033] The service interface field data is obtained based on the first corresponding relationship, and the return field data of the interface method corresponding to the splicing field is obtained based on the second corresponding relationship and the third corresponding relationship.
[0034] In a second aspect, an embodiment of the present invention provides a data splicing device, the data splicing device comprising:
[0035] A data acquisition module is used to acquire a target splicing field corresponding to the data to be spliced; the data to be spliced contains a basic field corresponding to the target splicing field and a basic field value of the basic field;
[0036] A method determination module, used to determine the target service interface corresponding to the target splicing field and the target interface method of the target service interface according to pre-configured service interface field data; the service interface field data includes each splicing field and the service interface corresponding to the splicing field and the interface method of the service interface;
[0037] A field determination module is used to call the target service interface to obtain the splicing field value of the target splicing field according to the target interface method and the basic field value of the basic field, and splice the target splicing field and the splicing field value of the target splicing field into the data to be spliced.
[0038] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the data splicing method described in any one of the above-mentioned embodiments when executing the computer program.
[0039] In a fourth aspect, an embodiment of the present invention provides a readable storage medium, wherein the readable storage medium includes a computer program, and when the computer program is executed, the electronic device where the readable storage medium is located is controlled to execute the data splicing method described in any of the above embodiments.
[0040] The data splicing method, device, electronic device and readable storage medium provided by the present invention pre-configure service interface field data including each splicing field and the service interface corresponding to the splicing field and the interface method of the service interface. After obtaining the target splicing field corresponding to the data to be spliced, the target service interface corresponding to the target splicing field and the target interface method of the target service interface can be determined according to the pre-configured service interface field data, and then according to the target interface method of the target service interface and the basic field value of the basic field, the target service interface can be called to obtain the splicing field value of the target splicing field, and the target splicing field and the splicing field value of the target splicing field are spliced into the data to be spliced. In this way, for different splicing fields, when it is necessary to splice data, it is only necessary to obtain the corresponding interface method through the service interface field data, and the data value to be spliced can be obtained through the interface method, without the need to write additional code and deploy it, which reduces the amount of code, reduces the complexity and maintenance difficulty, and improves work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] The technical solution and other beneficial effects of the present application will be made apparent by describing in detail the specific implementation methods of the present application in conjunction with the accompanying drawings.
[0042] Figure 1 A schematic diagram of an application scenario of the data splicing method provided in an embodiment of the present invention.
[0043] Figure 2 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention.
[0044] Figure 3 A flowchart of a data splicing method provided by an embodiment of the present invention.
[0045] Figure 4 A schematic diagram of another flow chart of the data splicing method provided in an embodiment of the present invention.
[0046] Figure 5 A schematic diagram of another flow chart of the data splicing method provided in an embodiment of the present invention.
[0047] Figure 6 A block diagram of a data splicing device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0048] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.
[0049] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, "plurality" means two or more, unless otherwise clearly and specifically defined.
[0050] The disclosure below provides many different embodiments or examples to realize the different structures of the present application. In order to simplify the disclosure of the present application, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeat reference numbers and / or reference letters in different examples, and this repetition is for the purpose of simplification and clarity, which itself does not indicate the relationship between the various embodiments and / or settings discussed. In addition, the various specific processes and material examples provided by the present application, but those of ordinary skill in the art can appreciate the application of other processes and / or the use of other materials.
[0051] As described in the background technology, the streaming data link of most commercial advertising real-time data is that the server and the client first upload the embedded data records to Kafka, thereby obtaining detailed data at the event granularity from Kafka, and then perform data extraction, conversion and other cleaning processes on the obtained data, and then perform pre-aggregation services on the processed data to pre-aggregate the advertising unit granularity data within a period of time, and then perform dimensional splicing, that is, splicing some attribute data into the detailed data, and finally aggregate the spliced data according to business needs to obtain the data required by the business line.
[0052] The splicing of data dimensions is currently done through customized development in the market. Different codes need to be developed and deployed for attribute values of different dimensions. When a new type of embedding information is added, that is, a new splicing dimension is added, it is necessary to customize the development of the access interface to splice the corresponding dimensional data. The development cycle is long, the efficiency is low, and the code is difficult to maintain. As the dimensions increase, the amount of code and complexity are also very high.
[0053] Based on the above research, the present invention provides a data splicing method, device, electronic device and readable storage medium, by pre-configuring service interface field data including each splicing field and the service interface corresponding to the splicing field and the interface method of the service interface, after obtaining the target splicing field corresponding to the data to be spliced, the target service interface corresponding to the target splicing field and the target interface method of the target service interface can be determined according to the pre-configured service interface field data, and then according to the target interface method of the target service interface and the basic field value of the basic field, the splicing field value of the target splicing field can be determined, and the target splicing field and the splicing field value of the target splicing field are spliced into the data to be spliced. In this way, for different splicing fields, when it is necessary to splice data, it is only necessary to obtain the corresponding interface method through the service interface field data, and the data value to be spliced can be obtained through the interface method, without the need to write additional code and deploy it, which reduces the amount of code, reduces the complexity and maintenance difficulty, and improves work efficiency.
[0054] See also Figure 1 , Figure 1 The following is a schematic diagram of a scenario of the data splicing method provided in this embodiment. Figure 1 As shown, the scenario diagram of the data splicing method provided in this embodiment includes a service provider, a service caller, a registration platform and an upstream end.
[0055] The upstream end refers to the party that stores and provides the data to be spliced to the service caller, for example, Kafka. The service provider is the party that provides the service and can provide the data to be spliced to the service caller through the service interface. The registration platform refers to the platform that stores the address information (IP address and port number) of the service provider. When deploying the service, the service provider will report the address information of the device where the service is deployed to the registration center. The registration center will then associate the service identifier (such as the service name, the full class path of the service, etc.) with the address information and store it.
[0056] The service caller refers to the party that executes the data splicing service. The service caller first obtains the data to be spliced from the upstream end, and then obtains the address information of the service from the registration center based on the service identifier of the service, and establishes a connection with the service interface of the service provider based on the address information. After that, the service caller can call the service interface of the service provider to query the splicing value and splice the splicing value into the data to be spliced.
[0057] The data splicing method provided in this embodiment is applied to the service caller and is executed by the electronic device configured in the service caller. Figure 2 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 2As shown, the electronic device 100 includes a data splicing device 10, a memory 20, a processor 30, and a communication unit 40. The memory 20, the processor 30, and the communication unit 40 are electrically connected to each other directly or indirectly to achieve signal transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.
[0058] In this embodiment, the data splicing device 10 includes at least one software function module that can be stored in the memory 20 in the form of software or firmware. The processor 30 is used to execute the software function module stored in the memory 20. When the electronic device 100 is running, the processor 30 communicates with the memory 20 through a bus, and the processor 30 executes the software function module to implement the data splicing method described in this embodiment.
[0059] Among them, the memory 20 can be, but is not limited to, random access memory (Random Access Memory, RAM), read only memory (Read Only Memory, ROM), programmable read-only memory (Programmable Read-Only Memory, PROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), electrically erasable read-only memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.
[0060] The processor 30 is used to perform one or more functions described in this embodiment. In some embodiments, the processor 30 may include one or more processing cores (e.g., a single-core processor (S) or a multi-core processor (S)). By way of example only, the processor 30 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), an application specific instruction set processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a controller, a microcontroller unit, a reduced instruction set computer (RISC) or a microprocessor, or any combination thereof.
[0061] For ease of description, only one processor is described in the electronic device 100. However, it should be noted that the electronic device 100 in this embodiment may also include multiple processors, so the steps performed by one processor described in this embodiment may also be performed jointly or individually by multiple processors. For example, if the processor of the electronic device performs step A and step B, it should be understood that step A and step B may also be performed jointly by two different processors or individually in one processor. For example, the processor performs step A and the second processor performs step B, or the processor and the second processor perform steps A and B together.
[0062] In this embodiment, the process definition method disclosed in any implementation manner may be applied to the processor 30 or implemented by the processor 30 .
[0063] The communication unit 40 is used to establish a communication connection between the electronic device 100 and other devices through a network, and to send and receive data through the network.
[0064] In some embodiments, the network may be any type of wired or wireless network, or a combination thereof. By way of example only, the network may include a wired network, a wireless network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a wide area network (WAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, or a near field communication (NFC) network, or any combination thereof.
[0065] In this embodiment, the electronic device can be a terminal, a notebook, a tablet, an ultra-mobile personal computer (UMPC), a personal digital assistant (PDA), a physical server and other devices, or it can be a service cluster composed of multiple physical servers. Each node in the service cluster can perform different functions. Specifically, this embodiment does not impose any restrictions on the type of electronic device.
[0066] Understandably, Figure 1 The structure shown is for illustration only. The electronic device may also have Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown. Figure 1 The components shown may be implemented in hardware, software or a combination thereof.
[0067] based on Figure 2 The implementation architecture of the present invention provides a data splicing method, which is applied to Figure 2 The electronic device shown consists of Figure 2 The electronic device shown in the figure performs. The process of the data splicing method provided by the embodiment of the present invention is described in detail below. Figure 3 As shown, the data splicing method provided by the embodiment of the present invention includes steps S101 to S103.
[0068] Step S101: Acquire the target splicing field corresponding to the data to be spliced.
[0069] The data to be spliced refers to data with other dimensions that need to be spliced, for example, the embedded data collected by the server and the client. This type of data is generally user behavior event data, such as user click behavior data and browsing behavior data. Usually, this type of data only has ID attributes, and does not have the user's basic attribute information, such as city, address, age, gender, longitude and latitude, account type, operator, network, device and other attribute information. Therefore, it is necessary to splice the user's basic attribute information.
[0070] In this embodiment, the target splicing field corresponding to the data to be spliced is a field that does not exist in the data to be spliced and is a splicing field that needs to be spliced. The target splicing field corresponding to the data to be spliced can be set based on business needs, and different businesses can have different splicing fields.
[0071] In this embodiment, the basic field is an existing field in the data to be spliced, and the basic field value of the basic field is an existing value in the data to be spliced, which can be used as an input parameter value of the request to obtain the spliced field value of the spliced field, that is, the spliced field value of the spliced field is the result value returned by the request based on the basic field value.
[0072] It should be noted that since the field value of the splicing field is obtained based on the basic field value request, for each splicing field, there is a corresponding basic field, and for the target splicing field corresponding to the data to be spliced, there is a basic field corresponding to the target splicing field and the basic field value of the basic field in the data to be spliced.
[0073] In this embodiment, after obtaining the data to be spliced from the upstream end (such as Kafka) and determining the target splicing field to be spliced, the fields in the data to be spliced are obtained by performing field analysis on the data to be spliced, and then based on the correspondence between the basic fields and the splicing fields, the basic fields corresponding to the target splicing fields and the basic field values of the basic fields are obtained from the fields of the data to be spliced.
[0074] For example, the data to be spliced obtained from the upstream end is {"placement":"explore_feed","noteId":"636c6f090000000023019355","trackId":"ad scpcn_85001486-6d1a-11ed-b85d-525400dcf445#tracking@2az37noaknmwzy8e5pf ab","userId":"5ccb9582000000001200b412","ip":"113.89.238.214","userAgent":"","action":"impression","target_type":"note","creativeId":"24520092","is_spam":0,"collectTs":1670219901073}, after field analysis, the fields that can be obtained include placement, noteId, trackId, userId, userAgent, action, creativeId, etc. , the field value of the placement field is explore_feed, the field value of the noteId field is 636c6f090000000023019355, the field value of the trackId field is adscpcn_85001486-6d1a-11ed-b85d-525400dcf445#tracking@2az37noaknmwzy8e5pfab, the field value of the userId field is 5ccb9582000000001200b412, the field value of the userAgent field is empty, the field value of the action field is impression, and the field value of the creativeId field is 24520092. Assuming that it is necessary to splice the noteType, noteTitle, creativeName and other fields to the data to be spliced, the noteType, noteTitle, creativeName and other fields are the target splicing fields corresponding to the data to be spliced, wherein the field values of the target splicing fields noteType and noteTitle are obtained based on the field value request of the field noteId, and the field value of the target splicing field creativeName is obtained based on the field value request of the field creativeId, then the field noteId is the basic field corresponding to the target splicing fields noteType and noteTitle, and the field creativeId is the basic field creativeId corresponding to the target splicing field creativeName, as shown in Table 1.
[0075] Table 1
[0076] Field Name Field Type Field division noteId String Basic fields creativeId String Basic fields noteType Int Concatenate fields noteTitle String Concatenate fields creativeName String Concatenate fields
[0077] It should be noted that, in this embodiment, the acquired data to be spliced may be multiple or one, the target splicing fields corresponding to each data to be spliced may be the same or different, and each data to be spliced may correspond to one target splicing field or multiple target splicing fields.
[0078] Step S102: Determine the target service interface corresponding to the target splicing field and the target interface method of the target service interface according to the preconfigured service interface field data.
[0079] After obtaining the target splicing field corresponding to the data to be spliced, the basic field corresponding to the target splicing field and the basic field value of the basic field, you can request to obtain the splicing field value corresponding to the target splicing field based on the basic field value of the basic field.
[0080] Since the splicing field value of the splicing field is obtained by calling the service interface (such as the RPC interface) provided by the service provider, different codes need to be written to obtain the corresponding splicing field values for different splicing fields. The development cycle is long, the efficiency is low, and the code is difficult to maintain. As the dimension of the splicing field increases, the amount of code and complexity are also very high.
[0081] In order to improve efficiency and reduce the difficulty of code maintenance, this embodiment pre-configures the service interface field data of each splicing field and the service interface and the interface method of the service interface corresponding to the splicing field. After obtaining the target splicing field corresponding to the data to be spliced, the target service interface and the target interface method of the target service interface corresponding to the target splicing field can be determined according to the service interface field data. As shown in Table 2, Table 2 shows the splicing fields (noteType, noteTitle and creativeName) and the corresponding service interfaces and the interface methods of the service interfaces. Each service interface has a unique service identifier, such as a service name, a full class path, etc. Each splicing field corresponds to a service interface and an interface method.
[0082] Table 2
[0083] Basic fields Concatenate fields Service interface (service identifier) Interface Methods noteId noteType community_service getNoteInfoByIds noteId noteTitle community_service getNoteInfoByIds creativeId creativeName ads_central_service getCreativeInfoByIds
[0084] Optionally, after obtaining the target splicing field corresponding to the data to be spliced, the target splicing field can be matched with the splicing field in the service interface field data. If there is a matching splicing field in the service interface field data that matches the target splicing field, the service interface and interface method corresponding to the matching splicing field are used as the service interface and interface method corresponding to the target splicing field. If there is no matching splicing field in the service interface field data that matches the target splicing field corresponding to the data to be spliced, an alarm can be issued to prompt the developer to perform configuration detection and update operations.
[0085] Step S103: according to the target interface method and the basic field value of the basic field, the target service interface is called to obtain the splicing field value of the target splicing field, and the target splicing field and the splicing field value of the target splicing field are spliced into the data to be spliced.
[0086] In this embodiment, after obtaining the target service interface corresponding to the target splicing field and the interface method of the target service interface, the address information can be obtained on the registration platform based on the service identifier of the target service interface, and then a connection can be established with the target service interface based on the address information. After that, the target interface method of the target service interface is executed, that is, the target service interface can be called to obtain the splicing field value of the target splicing field.
[0087] Considering that the service provider may deploy the service on multiple devices when deploying the service, and may upload multiple address information to the registration platform, the service identifier based on the target service interface may obtain multiple address information on the registration platform. When there are multiple address information, this embodiment can filter one target address information from the multiple address information through polling or hash algorithm, and establish a connection with the target service interface based on the target address information, so as to ensure the load balancing of the service query.
[0088] Since the field value of the target splicing field is obtained based on the basic field value request, when the target interface method of the target service interface is executed, the basic field value of the corresponding basic field can be used as the input parameter of the target interface method, thereby calling the target service interface to obtain the splicing field value of the target splicing field.
[0089] For example, set the basic field to noteId, the target splicing field corresponding to the data to be spliced to noteType, the interface method to getNoteInfoByIds, the service interface to community_service, and obtain the splicing field value of the target splicing field noteType through the basic field noteId. If the basic field value of noteId in the data to be spliced is b8bb4c4d679e9b3, then b8bb4c4d679e9b3 is the input parameter of the method getNoteInfoByIds, that is, {"noteIds":["b8bb4c4d679e9b3"]}. After determining the input parameter of the interface method, execute the interface method to call the service interface community_service to return the splicing field value of the target splicing field noteType.
[0090] After obtaining the splicing field value of the target splicing field, the target splicing field and the splicing field value of the target splicing field can be spliced into the data to be spliced.
[0091] For example, assuming the target splicing field is creativeName, its splicing field value is XXX, and the data to be spliced is {"placement":"explore_feed","noteId":"636c6f090000000023019355","trackId":"adscpcn_85001486-6d1a-11ed-b85d-525400dcf445#tracking@2az37noaknmwzy8e5pf ab","userId":"5ccb9582000000001200b412","ip":"113.89.238.214","userAgent":"","action":"impression","target_type":"note","creativeId":"24520092","is_spam":0,"col lectTs":1670219901073}, after splicing the target splicing field and the splicing field value of the target splicing field into the data to be spliced, the spliced data can be obtained as {"placement":"explore_feed","noteId":"636c6f090000000023019355","trackId":"ad scpcn_85001486-6d1a-11ed-b85d-525400dcf445#tracking@2az37noaknmwzy8e5pf ab","userId":"5ccb9582000000001200b412","ip":"113.89.238.214","userAgent":"","action":"impression","tar get_type":"note","creativeId":"24520092","is_spam":0,"collectTs":"1670219901073","creativeName":"XXX"}.
[0092] The data splicing method provided in the present embodiment pre-configures service interface field data including each splicing field and the service interface corresponding to the splicing field and the interface method of the service interface. When performing field splicing, the target service interface and the target interface method corresponding to the target splicing field can be obtained directly based on the pre-configured service interface field data, and then the basic field value of the corresponding basic field is used as an input parameter to execute the target interface method, so as to call the target service interface and obtain the splicing field value corresponding to the target splicing field. Compared with the prior art, the present embodiment only needs to maintain the correspondence between the splicing field and the service interface and the interface method to realize data splicing. There is no need to write different codes and deploy them for different fields, which greatly reduces the amount of code, reduces the code complexity and maintenance difficulty, and improves work efficiency.
[0093] In order to realize automatic data splicing and improve work efficiency, this embodiment can pre-configure an input parameter template for the interface method corresponding to each splicing field. When executing the interface method, the input parameter template can be directly adjusted according to the basic field value of the basic field to obtain the input parameter of the interface method. Based on this, this embodiment can also include the step of calling the target service interface to obtain the splicing field value of the target splicing field according to the target interface method and the basic field value of the basic field:
[0094] (1) Get the input parameter template of the target interface method.
[0095] (2) Based on the basic field value of the basic field, the dynamic query parameters of the input parameter template are replaced to obtain the input parameters of the target interface method.
[0096] (3) Based on the input parameters of the target interface method, the target interface method is executed, and the target service interface is called to obtain the splicing field value of the target splicing field.
[0097] Among them, for each splicing field, the input parameter template of the interface method corresponding to the splicing field can be configured based on the basic field corresponding to the splicing field and the input parameter structure of the interface method. Since the splicing field value of the splicing field is obtained based on the basic field value of the basic field, there are basic fields in the input parameter structure of the interface method. When configuring the input parameter template of the interface method, the basic field part in the input parameter structure can be replaced with dynamic query parameters to obtain the input parameter template of the interface method. Therefore, the dynamic query parameters of the interface method are configured in the input parameter template.
[0098] For example, taking the basic field value of the basic field creativeId as the input parameter to obtain the concatenated field value of the concatenated field creativeName, assuming that the interface method is queryCreativeInfoByIds, the input parameter structure of parsing queryCreativeInfoByIds is as follows:
[0099]
[0100] The basic field is creativeId, so you can configure the following parameter template:
[0101] {
[0102] "creativesIds":[${creativeId}]
[0103] }
[0104] For the interface method corresponding to each splicing field, after configuring the input parameter template of the interface method corresponding to the splicing field based on the basic field corresponding to the splicing field and the input parameter structure of the interface method, the splicing field, the interface method corresponding to the splicing field, and the input parameter template of the interface method can be associated and stored. Then, after obtaining the target splicing field corresponding to the data to be spliced and the target interface method corresponding to the target splicing field, the input parameter template of the target interface method can be obtained according to the target splicing field and the target interface method corresponding to the target splicing field.
[0105] After obtaining the input parameter template of the target interface method, replace the dynamic query parameters with real values, that is, the basic field values of the basic fields, to obtain the input parameters of the target interface method. For example, the target splicing field corresponding to the data to be spliced is creativeName, the basic field corresponding to the target splicing field is creativeId, and the target interface method corresponding to the target splicing field is queryCreativeInfoByIds. Set the basic field value of the basic field creativeId in the data to be spliced to 53f20b8bb4c4d679e9b39058, and the parameter template of the interface method queryCreativeInfoById is {"creativesIds":[${creativeId}]}. Based on the basic field value of the basic field, replace the dynamic query parameters of the input parameter template, and obtain the input parameters of the target interface method as {"creativesIds":["53f20b8bb4c4d679e9b39058"]}.
[0106] After obtaining the input parameters of the target interface method, the target interface method is executed to call the target service interface to return the query result of the target splicing field, that is, the splicing field value.
[0107] Considering that the returned fields in the returned query result data may be inconsistent with the representation of the spliced fields, and at the same time, the field values of other fields may also exist in the returned result data, it may be impossible to accurately identify the spliced field values belonging to the spliced fields. In order to quickly and accurately obtain the spliced field value of the target spliced field, in this embodiment, a correspondence between the returned fields and the spliced fields in the query results can be established, and the spliced field value of the target spliced field can be quickly and accurately obtained through the correspondence between the returned fields and the spliced fields.
[0108] Based on this, please refer to Figure 4 In this embodiment, based on the input parameters of the target interface method, the step of executing the target interface method and obtaining the splicing field value of the target splicing field may also include:
[0109] Step S201: Based on the input parameters of the target interface method, the target interface method is executed, and the target service interface is called to obtain the query result data.
[0110] Step S202: Acquire the return field data of the target interface method, and determine the first return field in the query result data corresponding to the target concatenated field according to the return field data.
[0111] Step S203: Determine the return field value corresponding to the first return field in the query result data as the splicing field value of the target splicing field.
[0112] The query result data refers to the query result data returned from the service interface after executing the target interface method, and the query result data includes at least one return field and the return field value corresponding to the return field. It can be understood that the return field represents the field in the query result data. For example, if the target splicing field corresponding to the data to be spliced is creativeName, the basic field corresponding to the target splicing field is creativeId, and the target interface method corresponding to the target splicing field is queryCreativeInfoByIds, the query result data returned can be:
[0113]
[0114] Among them, the fields id and name are both returned fields in the returned query result data.
[0115] After the query result data is obtained, the first return field in the query result data corresponding to the target concatenated field can be determined according to the return field data of the preconfigured target interface method.
[0116] The returned field data includes the corresponding relationship data between the target splicing field and the return field of the target interface method.
[0117] In an optional implementation, the return field data of the interface method corresponding to each spliced field can configure the result word count data of the interface method according to the method return structure of the interface method. Based on the method return structure of the interface method, the field values of which fields can be returned by the interface method can be obtained, and then based on the returned fields, the corresponding relationship between the spliced field and the return field can be configured.
[0118] For example, taking the interface method corresponding to the concatenation field creativeName as an example, the method structure of the interface method is as follows:
[0119]
[0120] The method return structure of the interface method shows that the interface method can return the field values of the id field and the name field, and the field value of the name field is the splicing field value that needs to be spliced. Therefore, the correspondence between the splicing field creativeName and the returned field name can be established.
[0121] In an optional embodiment, based on the return field data of the preconfigured target interface method, the first return field in the query result data corresponding to the target splicing field is determined. The return field in the return field data can be matched with the return field in the query result data to obtain a matching return field in the query result data that matches the return field in the return field data. The matching return field is the first return field corresponding to the target splicing field.
[0122] After determining to obtain the first return field corresponding to the target splicing field in the query result data, the return field value corresponding to the first return field can be determined as the splicing field value of the target splicing field.
[0123] For example, the target splicing field corresponding to the data to be spliced is creativeName, the basic field corresponding to the target splicing field is creativeId, and the target interface method corresponding to the target splicing field is queryCreativeInfoByIds. Assuming that there is a corresponding relationship between the return field name and the splicing field creativeName (name->creativeName) in the returned field data, after obtaining the query result data, if the return field name exists in the query result data, then the return field name in the query result data is the first return field corresponding to the target splicing field creativeName. If the return field value of the return field name in the query result data is "Creative 1", then the return field value "Creative 1" is the splicing field value of the target splicing field creativeName.
[0124] After obtaining the splicing field value of the target splicing field, the target splicing field and the splicing field value of the target splicing field can be spliced into the data to be spliced.
[0125] Considering that in actual applications, the data to be spliced obtained from the upstream is generally multiple data, and batch query is usually performed when querying. For example, the target splicing field corresponding to the data to be spliced is creativeName, the basic field corresponding to the target splicing field is creativeId, and the target interface method corresponding to the target splicing field is queryCreativeInfoByIds. Assume that the upstream obtains 2 data to be spliced, and their corresponding creativeIds are 53f20b8bb4c4d679e9b39058 and 543384a02e1d934 respectively. a2c11dc72, based on the basic field value of the basic field, the dynamic query parameter of the input template is replaced, and the input parameter is {"creativesIds":["53f20b8bb4c4d679e9b39058","543384a02e1d934a2c11dc72"]}, that is, batch query is performed based on "53f20b8bb4c4d679e9b39058" and "543384a02e1d934a2c11dc72". In this case, the returned result data includes the query result data corresponding to each basic field value.
[0126] like
[0127]
[0128] Among them, {"id":"53f20b8b","name":"Creative 1"} is the query result data returned based on the basic field value 53f20b8bb4c4d679e9b39058, and {"id":"543384a0","name":"Creative 2"} is the query result data returned based on the basic field value 543384a02e1d934a2c11dc72.
[0129] When there are multiple query result data returned, the query result data and the data to be spliced may be spliced incorrectly. In order to accurately obtain the splicing field value of each piece of data to be spliced during batch query, in this embodiment, a corresponding relationship between the returned field and the basic field can also be established, with the basic field value of the basic field as the key value and the splicing field value of the splicing field as the value value, so as to accurately obtain the splicing field value of each piece of data to be spliced.
[0130] Based on this, please refer to Figure 5 In this embodiment, the step of splicing the target splicing field and the splicing field value of the target splicing field into the data to be spliced may also include:
[0131] Step S301: Determine a second return field in the query result data corresponding to the basic field according to the return field data of the target interface method.
[0132] Step S302: Determine the target data to be spliced corresponding to the query result data according to the return field value of the second return field.
[0133] Step S303: splicing the target splicing field and the splicing field value of the target splicing field into the target data to be spliced.
[0134] In this embodiment, the return field data of the target interface method corresponding to the target splicing field also includes the correspondence data between the basic field and the return field of the target interface method, that is, the return field data of the target interface method corresponding to the target splicing field includes the correspondence data between the target splicing field and the return field of the target interface method, as well as the correspondence data between the basic field corresponding to the target splicing field and the return field of the target interface method.
[0135] In an optional implementation, the return field data of the interface method corresponding to each splicing field can be configured according to the method return structure of the interface method. Based on the method return structure of the interface method, the field values of the fields that the interface method can return can be obtained, and then based on the returned fields, the corresponding relationship between the splicing field and the return field is configured, as well as the corresponding relationship between the basic field corresponding to the splicing field and the return field is configured, so as to obtain the return field data of the interface method.
[0136] For example, taking the interface method corresponding to the splicing field creativeName as an example, the splicing method is queryCreativeInfoByIds, the basic field is creativeId, and the configured return field data can be (id->creativeId, name->creativeName), where id and name are the return fields, id->creativeId represents the corresponding relationship data between the return field id and the basic field creativeId, and name->creativeName represents the corresponding relationship data between the return field name and the splicing field creativeName.
[0137] When performing batch queries on multiple data to be spliced, in order to accurately obtain the splicing field value of each data to be spliced, after executing the target interface method and obtaining the query result data, for each query result data, the second return field corresponding to the basic field in the query result data can be determined based on the return field data of the target interface method, and then the data to be spliced corresponding to the query result data can be obtained based on the return field value of the second return field.
[0138] Since the return field data includes the correspondence data between the basic field and the return field of the target interface method, for each query result data, the return field in the query result data is matched with the return field in the return field data to obtain the second return field in the query result data corresponding to the basic field.
[0139] In this embodiment, the return field value of the second return field matches the basic field value of the corresponding basic field. Therefore, after obtaining the second return field corresponding to the basic field in the query result data, the return field value of the second return field can be matched with the basic field value of the basic field in each data to be spliced. If there is a basic field value that matches the return field value of the second return field, the data to be spliced where the matched basic field value is located is used as the target data to be spliced corresponding to the query result data, and then the splicing field value and the target splicing field in the query result data are spliced into the target data to be spliced.
[0140] In this embodiment, the return field value of the second return field matches the basic field value of its corresponding basic field. The return field value of the second return field can be the same as the basic field value of its corresponding basic field, or the return field value of the second return field can be partially the same as the basic field value of its corresponding basic field. For example, it can be the same as the first N bits of the basic field value of its corresponding basic field, where N is a positive integer and can be set according to actual needs.
[0141] For example, taking the target splicing field corresponding to the data to be spliced as creativeName, the basic field corresponding to the target splicing field as creativeId, and the target interface method corresponding to the target splicing field as queryCreativeInfoByIds, the configured return field data can be (id->creativeId, name->creativeName). After batch query based on "53f20b8bb4c4d679e9b39058" and "543384a02e1d934a2c11dc72", the returned results are as follows, "data":[{"id":"53f20b8b","name":"Creative 1"},{"id":"543384a0","name":"Creative 2"}]}. For the query result data {"id":"53f20b8b", "name":"Creativity 1"}, based on the configured return field data, it can be seen that id is the second return field corresponding to the basic field creativeId, name is the first return field corresponding to the splicing field creativeName, and "Creativity 1" is the splicing field value. After matching the return field value "53f20b8b" corresponding to the second return field with the basic field value of the basic field creativeId in each data to be spliced, the matched basic field value can be obtained as 53f20b8bb4c4d679e9b39058, and then the data to be spliced with the basic field value of 53f20b8bb4c4d679e9b39058 can be used as the target data to be spliced corresponding to the query result data {"id":"53f20b8b", "name":"Creativity 1"}, and then the target splicing field creativeName and the splicing field value "Creativity 1" are spliced into the target data to be spliced. It can be understood that the query result data {"id":"543384a0", "name":"Creativity 2"} also follows a corresponding process, which will not be elaborated here.
[0142] It can be understood that for a single-value query with only one piece of data to be spliced, since only one piece of query result data is returned, it is necessary to only determine the first return field in the query result data corresponding to the target splicing field, and then splice the return field value corresponding to the first return field in the query result data and the target splicing field into the data to be spliced.
[0143] The data splicing method provided in this embodiment establishes a corresponding relationship between the return field and the basic field, and uses the basic field value of the basic field as the key value to perform data splicing on the data to be spliced, so as to accurately obtain the splicing field value of each data to be spliced, thereby avoiding data splicing errors.
[0144] Considering that in actual applications, some service interfaces do not support batch query, and some service interfaces support batch query, but the number of parameters that support batch query is limited, therefore, when performing batch query, it is necessary to consider the query volume that the service interface can bear, to avoid excessive query volume, the service interface cannot respond, and thus cannot query to obtain the field value to be spliced. Based on this, before obtaining the splicing field value of the target splicing field according to the target interface method and the basic field value of the basic field, the data splicing method provided in this embodiment also includes:
[0145] Determine the query threshold of the target service interface and the amount of data to be spliced corresponding to the target splicing field.
[0146] If the data volume is greater than the query threshold of the target service interface, the data to be spliced corresponding to the target splicing field is split.
[0147] The query threshold of the target service interface refers to the maximum number of parameters supported by the target service interface for batch query.
[0148] In this embodiment, a query threshold for each service interface is preconfigured, and the query threshold for each service interface is associated with the service identifier of the service interface. Therefore, for each target splicing field, after obtaining the target service interface corresponding to the target splicing field based on the service interface field data, the query threshold for the target service interface can be obtained based on the service identifier of the target service interface.
[0149] For each target splicing field, the number of data to be spliced corresponding to the target splicing field is counted to obtain the data volume of the data to be spliced corresponding to the target splicing field. In this embodiment, the data to be spliced corresponding to each target splicing field refers to the data to be spliced that are to be spliced with the target splicing field. For example, for target splicing field A, assuming that the data to be spliced a and the data to be spliced b are both to be spliced with the splicing field value of the target splicing field A, then the data to be spliced a and the data to be spliced b are the data to be spliced corresponding to the target splicing field A.
[0150] After obtaining the query threshold of the target service interface corresponding to each target splicing field and the corresponding data volume of the data to be spliced, for each target splicing field, compare the query threshold of the service interface corresponding to the target splicing field with the corresponding data volume of the data to be spliced. If the query threshold of the target service interface is greater than or equal to the data volume of the data to be spliced, it indicates that the target service interface supports a maximum number of batch query parameters that is greater than or equal to the data volume to be queried, and can support batch query of the data to be spliced corresponding to the target splicing field.
[0151] If the query threshold of the target service interface is smaller than the amount of data to be spliced, it indicates that the maximum number of parameters supported for batch query by the target service interface is smaller than the amount of data to be queried, and the target service interface cannot support batch query of the data to be spliced corresponding to the target splicing field. If a batch query is performed on the data to be spliced corresponding to the target splicing field, it may cause the target service interface to fail to respond in time, request crash, and other problems.
[0152] In order to avoid problems such as excessive query volume, unresponsive service interface, and request crash, when the query threshold of the service interface corresponding to the splicing field is less than the data volume of the corresponding data to be spliced, the corresponding data to be spliced needs to be split into multiple groups of data for multiple queries. For example, if the query threshold of the service interface corresponding to the A splicing field is set to 1000, and the data volume of the corresponding data to be spliced is 5000, the corresponding data to be spliced needs to be divided into 5000 / 1000=5 queries, that is, the number of parameters entered into the interface method of the service interface corresponding to the A splicing field each time is 1000.
[0153] In this embodiment, if the query threshold of the target service interface is less than the amount of data to be spliced, after splitting the data to be spliced corresponding to the target splicing field, multiple data groups to be spliced can be obtained. For each data group to be spliced, the basic field value of the basic field corresponding to the target splicing field in each data to be spliced in the data group to be spliced is filled into the input parameter template of the target interface method corresponding to the target splicing field, and the input parameter of the target interface method corresponding to the target splicing field is obtained. Then, based on the input parameter of the target interface method, the target interface method is executed to obtain the query result data returned from the target service interface. Since it is a batch query, there are multiple query result data returned from the target service interface. Therefore, after obtaining the query result data returned by the target service interface, the splicing field value of each data to be spliced in the data group to be spliced can be accurately obtained based on the process from step S301 to step S303, which will not be repeated here.
[0154] The data splicing method provided in this embodiment splits the query of the data to be spliced through the query threshold of the service interface and the data volume of the data to be spliced, so as to avoid the situation where the service interface cannot respond and the request crashes due to excessive query data volume, thereby ensuring the stability of the service request.
[0155] Considering that there are multiple splicing fields that need to query the same service interface in actual applications, in order to improve query efficiency, in this embodiment, the splicing fields that query the same service interface can be merged. Therefore, before obtaining the splicing field value of the target splicing field according to the target interface method and the basic field value of the basic field, the data splicing method provided in this embodiment also includes:
[0156] Check whether there is a target splicing field corresponding to the data to be spliced that has the same corresponding basic field, target service interface, and target interface method.
[0157] The target splicing fields that have the same basic fields, target service interfaces, and target interface methods are merged to obtain a splicing field group.
[0158] Accordingly, according to the target interface method and the basic field value of the basic field, the target service interface is called to obtain the spliced field value of the target spliced field, including:
[0159] According to the target interface method corresponding to the splicing field group and the basic field value of the basic field in the to-be-spliced data corresponding to each target splicing field in the splicing field group, the target service interface is called to obtain the splicing field value of each target splicing field in the splicing field group.
[0160] Among them, after obtaining the target service interface and the target interface method of the target service interface corresponding to each target splicing field according to the service interface field data, the basic fields, target service interfaces and the target interface methods of the target service interfaces corresponding to each target splicing field are compared to determine whether there are splicing fields with the same basic fields, target service interfaces and target interface methods in each target splicing field. If there are splicing fields with the same basic fields, target service interfaces and target interface methods, the splicing fields with the same basic fields, target service interfaces and target interface methods are merged to obtain a splicing field group.
[0161] For example, the target splicing fields noteType and noteTitle are both based on the basic field noteId query service interface community_service, and the interface method uses the getNoteInfoByIds method. Therefore, the target splicing fields noteType and noteTitle can be merged to obtain the splicing field group<noteType,noteTitle> .
[0162] It can be understood that the splicing field group includes at least two target splicing fields, and the basic fields, service interfaces and interface methods corresponding to the target splicing fields are all the same.
[0163] It should be noted that when the basic fields, target service interfaces, and target interface methods corresponding to any two target splicing fields are the same, it means that when the target interface method is executed, the splicing field values required by the two target splicing fields can be returned at the same time. Therefore, for the splicing field group, when the target interface method corresponding to the splicing field group is executed and the target service interface is called for data query, the query result data returned from the target service interface includes the splicing field values of each target splicing field in the splicing field group.
[0164] For the splicing field group, when executing the interface method corresponding to the splicing field group, you can first fill in the basic field value of the basic field in the to-be-spliced data corresponding to each target splicing field in the splicing field group into the input parameter template of the target interface method corresponding to the splicing field group, obtain the input parameter of the target interface method corresponding to the splicing field group, and then execute the target interface method to obtain the query result data returned from the target service interface.
[0165] For example, the target splicing field for the data a to be spliced is note_type, and the target splicing field for the data b to be spliced is note_title. The target splicing fields noteType and noteTitle are both based on the basic field noteId query service interface community_service, and the interface method is the getNoteInfoByIds method. Among them, the basic field noteIds in the data a to be spliced is b8bb4c4d679e9b3, and the basic field noteIds in the data b to be spliced is a02e1d934a2c11d. The input parameter of the interface method corresponding to the splicing field group is {"noteIds":["b8bb4c4d679e9b3","a02e1d934a2c11d"]}. After executing the interface method getNoteInfoByIds, the service interface CommunityService is called, and the query result data returned is:
[0166]
[0167]
[0168] Each query result data includes the target splicing field 'note_type' and the splicing field values corresponding to 'note_title', wherein the target splicing field note_type corresponds to the returned field type, and the target splicing field note_title corresponds to the returned field name.
[0169] The returned query result data includes the splicing field values of multiple target splicing fields. For each query result data, first determine the second return field corresponding to the basic field in the query result data according to the return field data of the target splicing method, determine the target data to be spliced corresponding to the query result data according to the return field value of the second return field, and then determine the target splicing field to be spliced with the target data to be spliced in the splicing field group, and then determine the first return field corresponding to the target splicing field to be spliced with the target data to be spliced in the query result data based on the return field data, determine the return field value corresponding to the first return field in the query result data as the splicing field value, and finally splice the splicing field to be spliced with the target data to be spliced and the splicing field value into the target data to be spliced.
[0170] The data splicing method provided in this embodiment can reduce the number of queries by merging the splicing fields requesting the same service interface, thereby improving the query efficiency.
[0171] In order to better implement the present method, before obtaining the splicing fields corresponding to the data to be spliced, the data splicing method provided in the present embodiment may further include the following steps:
[0172] (a) Obtain a file package of the service interface, parse the file package, and obtain the interface method of the service interface, as well as the input parameter structure information and method return structure information of the interface method.
[0173] (b) Based on the interface method of the service interface, the input parameter structure information of the interface method and the method return structure information, a first correspondence between the splicing field and the service interface and the interface method of the service interface is established, a second correspondence between the splicing field and the return field of the corresponding interface method is established, and a third correspondence between the basic field corresponding to the splicing field and the return field of the corresponding interface method is established, and an input parameter template for the splicing field corresponding to the interface method is established.
[0174] (c) Based on the first corresponding relationship, the service interface field data is obtained, and based on the second corresponding relationship and the third corresponding relationship, the return field data of the interface method corresponding to the splicing field is obtained.
[0175] The file package of the service interface may be a jar package compiled by the service provider from the interface description language (IDL) file of its service interface, or may be a file package compiled in other languages, without specific limitation.
[0176] To obtain the file package of the service interface, a connection may be directly established with the service provider, and the file package of the service interface may be directly obtained from the service provider.
[0177] In an optional implementation, in order to facilitate the management of file packages, this embodiment provides a storage warehouse (such as a Maven warehouse). The service provider can upload the file package of the service interface to the storage warehouse, and send the file package's storage address in the storage warehouse, the full class path of the service interface, the transmission protocol, etc. to the electronic device.
[0178] The storage address is the storage location (storage coordinates) of the file package in the storage warehouse. The storage address of the Maven warehouse can be represented by groupId and artifactId. The groupId corresponds to the service name provided by the service provider and is unique. The artifactId corresponds to the specific code module name, that is, the package name of the file package, and is unique.
[0179] The full class path is the full path of the class corresponding to the service provided by the service provider, which is used to locate the path of the service class. For example: com.xiaohongshu.fls.rpc.ads_data.AdsStatsCentralService.
[0180] The transmission protocol is the protocol used to transmit data between the service provider and the electronic device. When the service interface provided by the service provider is non-blocking (such as Java's TNonblockingServer), the Framed transmission protocol can be used for processing.
[0181] After the electronic device receives the storage address of the storage warehouse, the full class path of the service interface, and the transmission protocol, the developer of the electronic device can perform basic configuration of the service information based on the storage address of the storage warehouse, the full class path of the service interface, the transmission protocol and other information. In order to facilitate the establishment of the corresponding relationship, it is also necessary to set a service identifier for the service interface to serve as a business identifier. The service identifier can be a service name, such as ads_central_service, or it can be the full class path of the service interface. There is no specific restriction, it only needs to be unique.
[0182] After completing the relevant configuration, you can obtain the interface method of the service interface based on the configured service information.
[0183] Optionally, the address can be stored as a request parameter, and a file package can be obtained from the storage repository. The file package is then parsed, and based on the configured class full path, the interface method file of the service interface (such as the class file in Java) is obtained from the file package through reflection. The interface method of the service interface, the input parameter structure information of the interface method, and the method return structure information are obtained from the interface method file.
[0184] In an optional implementation, taking into account that the service provider will continuously update the version of the service interface, multiple versions of file packages may be stored in the storage warehouse. If different versions of file packages are stored in the storage warehouse, after obtaining all currently available versions of the file packages based on the storage address, the latest version of the file package can be used as the file package to be parsed, or the file package to be parsed can be selected according to the version requirements.
[0185] In an optional implementation, when multiple versions of file packages are stored in the storage warehouse, a version identifier can be set for each version of the file package. When the file package needs to be obtained, the storage address and version identifier can be used as request parameters to directly obtain the file package corresponding to the version identifier from the storage warehouse.
[0186] After obtaining the interface method of the service interface, the input parameter structure information of the interface method and the method return structure information, a first correspondence between the splicing field and the service interface and the interface method of the service interface can be established based on the interface method of the service interface, the input parameter structure information of the interface method and the method return structure information.
[0187] In this embodiment, for any service interface and any splicing field, if the interface method of the service interface can be obtained based on the method return structure information of the interface method of the service interface, and the type of the basic field corresponding to the splicing field matches the input parameter structure of the interface method of the service interface, then the correspondence between the splicing field and the service interface and the interface method of the service interface can be established.
[0188] Optionally, in this embodiment, the interface methods of the service interface, the input parameter structure information of the interface methods, and the method return structure information can be displayed to facilitate developers to obtain the functions implemented by the interface methods of the service interface, the input parameter structure information and the method return structure information required by the interface methods, and then establish a correspondence between the splicing fields and the service interface and the interface methods of the service interface by responding to configuration instructions based on the interface methods of the service interface, the input parameter structure information of the interface methods, and the method return structure information feedback.
[0189] After obtaining the correspondence between each splicing field and the service interface and the interface method of the service interface, the service interface field data can be obtained based on the correspondence between each splicing field and the service interface and the interface method of the service interface.
[0190] In this embodiment, based on the correspondence between each splicing field and the service interface and the interface method of the service interface, the interface method corresponding to each splicing field can be obtained. For the interface method corresponding to each splicing field, the return field that can be returned by the interface method corresponding to the splicing field can be obtained according to the method return structure information of the interface method corresponding to the splicing field, and then based on the return field that can be returned by the interface method corresponding to the splicing field, a second correspondence between the splicing field and the return field of the corresponding interface method and a third correspondence between the basic field corresponding to the splicing field and the return field of the corresponding interface method are established.
[0191] Optionally, in this embodiment, for each splicing field, the method return structure information of the interface method corresponding to the splicing field can be displayed to facilitate developers to obtain the return field that the interface method corresponding to the splicing field can return, and then by responding to the configuration instructions based on the method return structure information feedback, the second correspondence between the splicing field and the return field of the corresponding interface method and the third correspondence between the basic field corresponding to the splicing field and the return field of the corresponding interface method are displayed.
[0192] After obtaining the second correspondence between the splicing field and the return field of the corresponding interface method and the third correspondence between the basic field corresponding to the splicing field and the return field of the corresponding interface method, the return field data of the interface method corresponding to the splicing field is obtained according to the second correspondence and the third correspondence.
[0193] In this embodiment, for the interface method corresponding to each splicing field, the input parameter template of the interface method corresponding to the splicing field can be configured based on the basic field corresponding to the splicing field and the input parameter structure of the interface method, that is, the basic field part in the input parameter structure is replaced with dynamic query parameters, and the input parameter template of the interface method corresponding to the input parameter template of the interface method can be obtained.
[0194] In an optional implementation, the input parameter template of the interface method can also be configured in a response manner. For each splicing field, the input parameter structure information of the interface method corresponding to the splicing field can be displayed to facilitate developers to obtain the input parameter structure of the interface method corresponding to the splicing field, and then by responding to configuration instructions based on the input parameter structure information feedback, the basic field part in the input parameter structure is replaced with dynamic query parameters to obtain the input parameter template of the interface method corresponding to the input parameter template of the interface method.
[0195] After completing the configuration of service interface field data, interface method return field data, and input parameter template information, you can splice data based on the service interface field data, interface method return field data, and input parameter template. The processing process does not require additional code writing, and data splicing can be completed based on the relevant configuration information.
[0196] Moreover, for existing customization, when the version of the file package changes, or the field to be spliced changes, the code needs to be rewritten and redeployed. However, the data splicing method provided in this embodiment is based on configuration information, and the entire process is effective immediately. When the version of the file package changes, or the field to be spliced changes, only the configuration information needs to be changed, such as modifying the corresponding relationship or adding a new corresponding relationship, etc., without the need to redeploy or restart the service. Compared with the existing technology, it is efficient and convenient, and can take into account system stability.
[0197] Based on the same invention concept, please refer to Figure 6 The embodiment of the present invention provides a data splicing device 10, which is applied to Figure 2 The electronic equipment shown, such as Figure 6 As shown, the data splicing device 10 provided in the embodiment of the present invention includes a data acquisition module 11, a method determination module 12 and a field determination module 13.
[0198] The data acquisition module 11 is used to acquire the target splicing field corresponding to the data to be spliced; the data to be spliced contains a basic field corresponding to the target splicing field and a basic field value of the basic field.
[0199] The method determination module 12 is used to determine the target service interface corresponding to the target splicing field and the target interface method of the target service interface according to the preconfigured service interface field data; the service interface field data includes each splicing field and the service interface corresponding to the splicing field and the interface method of the service interface.
[0200] The field determination module 13 is used to call the target service interface to obtain the splicing field value of the target splicing field according to the target interface method and the basic field value of the basic field, and splice the target splicing field and the splicing field value of the target splicing field into the data to be spliced.
[0201] In an optional implementation manner, the field determination module 13 is used to:
[0202] Get the input parameter template of the target interface method; the input parameter template is configured with the dynamic query parameters of the target interface method.
[0203] Based on the basic field value of the basic field, the dynamic query parameters of the input parameter template are replaced to obtain the input parameters of the target interface method.
[0204] Based on the input parameters of the target interface method, the target interface method is executed, and the target service interface is called to obtain the splicing field value of the target splicing field.
[0205] In an optional implementation manner, the field determination module 13 is used to:
[0206] Based on the input parameters of the target interface method, the target interface method is executed, and the target service interface is called to obtain the query result data; the query result data includes at least one return field and the return field value corresponding to the return field.
[0207] The return field data of the target interface method is obtained, and the first return field in the query result data corresponding to the target splicing field is determined according to the return field data; the return field data includes the corresponding relationship data between the target splicing field and the return field of the target interface method.
[0208] The return field value corresponding to the first return field in the query result data is determined as the splicing field value of the target splicing field.
[0209] In an optional implementation manner, the field determination module 13 is used to:
[0210] According to the return field data of the target interface method, determine the second return field in the query result data corresponding to the basic field; the return field data includes the correspondence data between the basic field and the return field of the target interface method, and the basic field value of the basic field matches the return field value of the corresponding return field.
[0211] According to the return field value of the second return field, the target data to be spliced corresponding to the query result data is determined.
[0212] The target splicing field and the splicing field value of the target splicing field are spliced into the target data to be spliced.
[0213] In an optional implementation, the data splicing device 10 further includes a merging module, which is used to:
[0214] Check whether there is a splicing field with the same corresponding basic field, target service interface and target interface method in the target splicing field corresponding to the data to be spliced.
[0215] The splicing fields that have the same basic fields, target service interfaces, and target interface methods are merged to obtain a splicing field group.
[0216] Accordingly, the field determination module 13 is used to:
[0217] According to the target interface method corresponding to the splicing field group and the basic field value of the basic field in the to-be-spliced data corresponding to each target splicing field in the splicing field group, the target service interface is called to obtain the splicing field value of each target splicing field in the splicing field group.
[0218] In an optional implementation, the data splicing device 10 further includes a splitting module, which is used to:
[0219] Determine the query threshold of the target service interface and the amount of data to be spliced corresponding to the target splicing field;
[0220] If the data volume is greater than the query threshold of the target service interface, the data to be spliced corresponding to the target splicing field is split.
[0221] In an optional implementation, the data splicing device 10 further includes a configuration module. Before obtaining the splicing fields corresponding to the data to be spliced, the configuration module is used to:
[0222] Get the file package of the service interface, parse the file package, and obtain the interface method of the service interface, as well as the input parameter structure information and method return structure information of the interface method.
[0223] Based on the interface method of the service interface, the input parameter structure information of the interface method and the method return structure information, a first correspondence between the splicing field and the service interface and the interface method of the service interface is established, a second correspondence between the splicing field and the return field of the corresponding interface method is established, and a third correspondence between the basic field corresponding to the splicing field and the return field of the corresponding interface method is established, and an input parameter template corresponding to the interface method of the splicing field is established.
[0224] The service interface field data is obtained based on the first corresponding relationship, and the return field data of the interface method corresponding to the splicing field is obtained based on the second corresponding relationship and the third corresponding relationship.
[0225] The data splicing device provided in this embodiment pre-configures service interface field data including each splicing field and the service interface corresponding to the splicing field and the interface method of the service interface. After obtaining the target splicing field corresponding to the data to be spliced, the target service interface corresponding to the target splicing field and the target interface method of the target service interface can be determined according to the pre-configured service interface field data, and then the splicing field value of the target splicing field can be determined according to the target interface method of the target service interface and the basic field value of the basic field, and the target splicing field and the splicing field value of the target splicing field are spliced into the data to be spliced. In this way, for different splicing fields, when it is necessary to splice data, it is only necessary to obtain the corresponding interface method through the service interface field data, and the data value to be spliced can be obtained through the interface method, without the need to write additional code and deploy it, which reduces the amount of code, reduces the complexity and maintenance difficulty, and improves work efficiency.
[0226] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the data splicing device described above can refer to the corresponding process in the aforementioned method, and will not be elaborated here.
[0227] Based on the above, this embodiment provides a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the data splicing method of any of the aforementioned embodiments is implemented.
[0228] The readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.
[0229] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the readable storage medium described above can refer to the corresponding process in the aforementioned method, and will not be described in detail here.
[0230] In summary, the data splicing method, device, electronic device and readable storage medium provided by the present invention pre-configure the service interface field data including each splicing field and the service interface corresponding to the splicing field and the interface method of the service interface. After obtaining the target splicing field corresponding to the data to be spliced, the target service interface corresponding to the target splicing field and the target interface method of the target service interface can be determined according to the pre-configured service interface field data, and then according to the target interface method of the target service interface and the basic field value of the basic field, the splicing field value of the target splicing field can be determined, and the target splicing field and the splicing field value of the target splicing field are spliced into the data to be spliced. In this way, for different splicing fields, when it is necessary to splice data, it is only necessary to obtain the corresponding interface method through the service interface field data, and the data value to be spliced can be obtained through the interface method. The whole process is based on configuration implementation and takes effect immediately. No additional code needs to be written and deployed, which reduces the amount of code, reduces complexity and maintenance difficulty, improves work efficiency, and also takes into account system stability.
[0231] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0232] The above is a detailed introduction to a data splicing method, device, electronic device and readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the technical solution and its core idea of the present application. Ordinary technicians in this field should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some of the technical features therein with equivalents; and these modifications or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiments of the present application.
Claims
1. A data splicing method, characterized in that: The method comprises: Acquire a target splicing field corresponding to the data to be spliced; the data to be spliced contains a basic field corresponding to the target splicing field and a basic field value of the basic field, the target splicing field is a field that does not exist in the data to be spliced and needs to be spliced, and the basic field is a field that already exists in the data to be spliced; Determine the target service interface corresponding to the target splicing field and the target interface method of the target service interface according to the preconfigured service interface field data; the service interface field data includes each splicing field and the service interface corresponding to the splicing field and the interface method of the service interface; When executing the target interface method of the target service interface, the input parameter of the target interface method is obtained based on the basic field value of the basic field, the target service interface is called to obtain the splicing field value of the target splicing field, and the target splicing field and the splicing field value of the target splicing field are spliced into the data to be spliced.
2. The data splicing method according to claim 1, characterized in that: When executing the target interface method of the target service interface, obtaining the input parameter of the target interface method based on the basic field value of the basic field, and calling the target service interface to obtain the splicing field value of the target splicing field, including: Obtaining an input parameter template of the target interface method; the input parameter template is configured with dynamic query parameters of the target interface method; Based on the basic field value of the basic field, the dynamic query parameter of the input parameter template is replaced to obtain the input parameter of the target interface method; Based on the input parameters of the target interface method, the target interface method is executed, and the target service interface is called to obtain the splicing field value of the target splicing field.
3. The data splicing method according to claim 2, characterized in that: The step of executing the target interface method based on the input parameter of the target interface method and calling the target service interface to obtain the splicing field value of the target splicing field includes: Based on the input parameters of the target interface method, the target interface method is executed, and the target service interface is called to obtain query result data; the query result data includes at least one return field and a return field value corresponding to the return field; Acquire the return field data of the target interface method, and determine the first return field in the query result data corresponding to the target splicing field according to the return field data; the return field data includes the corresponding relationship data between the target splicing field and the return field of the target interface method; The return field value corresponding to the first return field in the query result data is determined as the splicing field value of the target splicing field.
4. The data splicing method according to any one of claims 1 to 3, characterized in that: The step of splicing the target splicing field and the splicing field value of the target splicing field into the data to be spliced includes: According to the return field data of the target interface method, determining a second return field in the query result data corresponding to the basic field; the return field data includes the corresponding relationship data between the basic field and the return field of the target interface method, and the basic field value of the basic field matches the return field value of the corresponding return field; Determining target data to be spliced corresponding to the query result data according to the return field value of the second return field; The target splicing field and the splicing field value of the target splicing field are spliced into the target data to be spliced.
5. The data splicing method according to claim 1, characterized in that: When executing the target interface method of the target service interface, before obtaining the input parameter of the target interface method based on the basic field value of the basic field and calling the target service interface to obtain the splicing field value of the target splicing field, the method further includes: Detect whether there is a splicing field with the same corresponding basic field, target service interface and target interface method in the target splicing field corresponding to the data to be spliced; Merge the splicing fields that have the same basic fields, target service interfaces, and target interface methods to obtain a splicing field group; When executing the target interface method of the target service interface, obtaining the input parameter of the target interface method based on the basic field value of the basic field, and calling the target service interface to obtain the splicing field value of the target splicing field, including: According to the target interface method corresponding to the splicing field group and the basic field value of the basic field in the to-be-spliced data corresponding to each target splicing field in the splicing field group, the target service interface is called to obtain the splicing field value of each target splicing field in the splicing field group.
6. The data splicing method according to claim 1, characterized in that: When executing the target interface method of the target service interface, before obtaining the input parameter of the target interface method based on the basic field value of the basic field and calling the target service interface to obtain the splicing field value of the target splicing field, the method further includes: Determine a query threshold of the target service interface and a data volume of the to-be-joined data corresponding to the target joint field; If the data volume is greater than the query threshold of the target service interface, the to-be-joined data corresponding to the target joint field is split.
7. The data splicing method according to claim 1, characterized in that: Before obtaining the target splicing field corresponding to the data to be spliced, the method further includes: Obtain a file package of the service interface, parse the file package, and obtain the interface method of the service interface, as well as the input parameter structure information and method return structure information of the interface method; Based on the interface method of the service interface, the input parameter structure information of the interface method and the method return structure information, a first correspondence between the splicing field and the service interface and the interface method of the service interface is established, a second correspondence between the splicing field and the return field of the corresponding interface method and a third correspondence between the basic field corresponding to the splicing field and the return field of the corresponding interface method are established, and an input parameter template of the interface method corresponding to the splicing field is established; The service interface field data is obtained based on the first corresponding relationship, and the return field data of the interface method corresponding to the splicing field is obtained based on the second corresponding relationship and the third corresponding relationship.
8. A data splicing device, characterized in that: The data splicing device comprises: A data acquisition module is used to acquire a target splicing field corresponding to the data to be spliced; the data to be spliced contains a basic field corresponding to the target splicing field and a basic field value of the basic field, the target splicing field is a field that does not exist in the data to be spliced and needs to be spliced, and the basic field is a field that already exists in the data to be spliced; A method determination module, used to determine the target service interface corresponding to the target splicing field and the target interface method of the target service interface according to pre-configured service interface field data; the service interface field data includes each splicing field and the service interface corresponding to the splicing field and the interface method of the service interface; A field determination module is used to obtain the input parameter of the target interface method based on the basic field value of the basic field when executing the target interface method of the target service interface, call the target service interface to obtain the splicing field value of the target splicing field, and splice the target splicing field and the splicing field value of the target splicing field into the data to be spliced.
9. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the data splicing method according to any one of claims 1 to 7 when executing the computer program.
10. A readable storage medium, characterized in that: The readable storage medium includes a computer program, and when the computer program is executed, the electronic device where the readable storage medium is located is controlled to execute the data splicing method according to any one of claims 1 to 7.
