A complex data processing method and device, electronic equipment and storage medium
By parsing the filtering rules of target data templates and format objects, a unified filtering process for complex data is achieved, solving the problem of low development efficiency caused by the customization of filtering programs in existing technologies and improving software development efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI SHIZHUANG INFORMATION TECHNOLOGY CO LTD
- Filing Date
- 2023-11-01
- Publication Date
- 2026-07-07
AI Technical Summary
In existing technologies, because each type of complex data has a unique format, specific filtering programs need to be written, resulting in slow processing speed and affecting software development efficiency.
By parsing the target data template to obtain a format object, an initial screening is performed based on the data type of the format object, and a second screening is performed by combining preset screening rules and the key-value information of the initial screening data, thus achieving unified filtering processing of complex data.
It reduces the difficulty for developers to design and debug filtering programs, and improves software development efficiency.
Smart Images
Figure CN117271853B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a complex data processing method, apparatus, electronic device, and storage medium. Background Technology
[0002] In modern software systems, network communication plays a crucial role, and its speed directly impacts the user experience. In communication involving complex data, filtering out unnecessary information is a vital way to improve network communication efficiency. Currently, due to the unique format of each type of complex data, specific filtering programs are typically required to filter each data block. This approach is slow and negatively impacts software development efficiency. Summary of the Invention
[0003] This application provides a complex data processing method, apparatus, electronic device, and storage medium that can uniformly filter various complex data blocks without requiring custom-written filtering programs. This reduces the difficulty for developers in designing and debugging filtering programs and improves software development efficiency.
[0004] Firstly, this application provides a method for processing complex data, the method comprising:
[0005] Parse the target data template to obtain the format object of the target data template, which is the template for retaining data after processing the complex data to be filtered;
[0006] The initial screening data is obtained by performing an initial screening of the complex data to be filtered based on the data type of the format object.
[0007] The target data is obtained by further filtering the initial screening data based on the key-value information of the initial screening data.
[0008] Furthermore, the step of parsing the target data template to obtain the format object of the target data template includes: determining the first data type of the fields in the target data template; if the first data type is an object, then determining the field as the format object; if the first data type is a string, then obtaining the address suffix of the string, determining the parser corresponding to the address suffix, and parsing the target data template based on the parser to obtain the format object.
[0009] Furthermore, the number of format objects is multiple; the initial screening of the complex data to be filtered based on the data type of the format objects to obtain preliminary screening data includes: for the current format object, obtaining a corresponding first data block from the complex data to be filtered, and obtaining a corresponding second data block from the target data template; determining the second data type of the second data block, and using a preset screening rule to screen the first data block based on the second data type to obtain intermediate data; sequentially traversing multiple format objects to obtain multiple intermediate data, and merging the multiple intermediate data as the preliminary screening data.
[0010] Furthermore, the step of using a preset filtering rule to filter the first data block based on the second data type to obtain intermediate data includes: if the second data type is neither an object nor an array, then the first data block is used as the intermediate data; if the second data type is an object, then the first data block is used as new complex data to be filtered, and the operation of initially filtering the complex data to be filtered based on the data type of the format object to obtain initial filtered data is repeated; if the second data type is an array, then the first data block is filtered based on the data type of the first array element in the second data block to obtain intermediate data.
[0011] Furthermore, the step of filtering the first data block based on the data type of the first array element in the second data block to obtain intermediate data includes: determining the third data type of the first array element in the second data block; if the third data type is an object, then each array element in the first data block is taken as a new complex data to be filtered, and the operation of initially filtering the complex data to be filtered based on the data type of the format object to obtain preliminary data is repeated; if the third data type is not an object, then the first data block is taken as the intermediate data.
[0012] Furthermore, the initial screening data includes multiple key-value names; the step of further filtering the initial screening data based on the key-value information of the initial screening data to obtain target data includes: for the current key-value name, obtaining the corresponding key-value information from the initial screening data; determining whether the key-value information is included in a pre-configured filter data value library; if included, deleting the current key-value name; and sequentially traversing multiple key-value names to obtain the target data.
[0013] Furthermore, before determining whether the key-value information is included in the pre-configured filter data value library, the method further includes: determining a fourth data type of the key-value information; if the fourth data type is a string, then performing the operation of determining whether the key-value information is included in the pre-configured filter data value library; if the fourth data type is an object or an array, then using the key-value information as new initial screening data, and repeatedly performing the operation of further filtering the initial screening data based on the key-value information of the initial screening data to obtain target data.
[0014] Secondly, this application provides a complex data processing apparatus, which includes:
[0015] The template parsing module is used to parse the target data template to obtain the format object of the target data template, wherein the target data template is the template for retaining data after processing the complex data to be filtered;
[0016] The first processing module is used to perform an initial screening of the complex data to be filtered based on the data type of the format object to obtain initial screening data.
[0017] The second processing module is used to further filter the initial screening data based on the key-value information of the initial screening data to obtain the target data.
[0018] Thirdly, this application provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the complex data processing method described in any embodiment of this application.
[0019] Fourthly, this application provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the complex data processing method described in any embodiment of this application.
[0020] To address the shortcomings of existing technologies, this application provides a method for processing complex data. This method offers the following advantages: It parses a user-preconfigured target data template to extract its format object; it then performs a deep traversal of the target data template's format object, initially filtering the complex data based on the data type of the corresponding data block; finally, it performs a second filtering based on the user-configured filter data value library and the key-value information of the initially filtered data, thus achieving the filtering of complex data. Compared to existing technologies, this application only requires configuring a target data template and can perform unified filtering of various complex data blocks without the need for custom-written filtering programs. This reduces the difficulty for developers in designing and debugging filtering programs, improving software development efficiency.
[0021] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on a computer-readable storage medium. This computer-readable storage medium may be packaged together with the processor of a complex data processing device, or it may be packaged separately from the processor of a complex data processing device; this application does not impose any limitations on this.
[0022] The descriptions of the second, third, and fourth aspects in this application can be referenced to the detailed description of the first aspect; and the beneficial effects described in the second, third, and fourth aspects can be referenced to the analysis of the beneficial effects of the first aspect, which will not be repeated here.
[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description.
[0024] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained. Attached Figure Description
[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0026] Figure 1 This is a first flowchart illustrating a complex data processing method provided in an embodiment of this application;
[0027] Figure 2This is a second flowchart illustrating a complex data processing method provided in an embodiment of this application.
[0028] Figure 3 This is a schematic diagram of the structure of a complex data processing device provided in an embodiment of this application;
[0029] Figure 4 This is a block diagram of an electronic device used to implement a complex data processing method according to an embodiment of this application. Detailed Implementation
[0030] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.
[0031] It should be noted that the terms "first," "second," "target," and "original," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in sequences other than those illustrated or described herein. Furthermore, the terms "comprising," "having," and any variations thereof are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0032] Figure 1 This is a first flowchart illustrating a complex data processing method provided in this application embodiment. This embodiment is applicable to filtering complex data to obtain the target data to be retained. The complex data processing method provided in this embodiment can be executed by the complex data processing apparatus provided in this application embodiment. This apparatus can be implemented by software and / or hardware and integrated into the electronic device executing this method.
[0033] See Figure 1 The method in this embodiment includes, but is not limited to, the following steps:
[0034] S110. Parse the target data template to obtain the format object of the target data template.
[0035] The target data template is the template for the data to be retained after processing the complex data to be filtered. It is a data template pre-configured by the user based on the data to be retained. This embodiment does not limit the file type of the target data template; selectable file types include YAML, XML, or JSON. The content of the same data can be written as complex data templates of different file types. The format object is the key-value name of the data that the user needs to retain.
[0036] Specifically, parsing the target data template to obtain the target data template format object includes: determining the first data type of the fields in the target data template; if the first data type is an object, it indicates that the field is the format object of the target data template, and the field is determined as a format object; if the first data type is a string, it indicates that the field is a template file address, and the address suffix of the string is obtained, the parser corresponding to the address suffix is determined, and the template file content in the target data template is parsed based on the parser to obtain the format object.
[0037] For example, if the template file address has an .xml extension, the corresponding parser is xml2js, which parses the .xml file content to obtain the target data template format object; if the template file address has a .yaml extension, the corresponding parser is js-yaml, which parses the .yaml file content to obtain the target data template format object; if the template file address has a .json extension, the corresponding parser is node, which parses the .json file content to obtain the target data template format object.
[0038] S120. Based on the data type of the format object, perform initial screening on the complex data to be filtered to obtain the initial screening data.
[0039] Optionally, the number of format objects can be multiple, such as person, country, and address. The complex data to be filtered can be a collection of data of different types, and a data block within it can have many levels of data types.
[0040] Specifically, the initial screening of complex data to be filtered, based on the data type of the format object, involves: obtaining the first data block corresponding to the current format object from the complex data to be filtered, and obtaining the second data block corresponding to the current format object from the target data template; determining the second data type of the second data block, and using a preset screening rule to screen the first data block based on the second data type to obtain intermediate data; following this method, each format object in multiple format objects is traversed sequentially to obtain multiple intermediate data; finally, the multiple intermediate data are merged to obtain the initial screening data.
[0041] For example, if the current format object is "person", a first data block about "person" is obtained from the complex data to be filtered, and a second data block about "person" is obtained from the target data template. It is determined whether the second data block's second data type is an array, string, or object, and then the processing method for the first data block is determined. Based on this processing method, the first data block is processed to obtain the first intermediate data. Following this method, the second intermediate data corresponding to the current format object being "country" and the third intermediate data corresponding to the current format object being "address" can be obtained. Finally, the first, second, and third intermediate data are merged to obtain the initial screening data.
[0042] S130. Based on the key-value information of the initial screening data, the initial screening data is further filtered to obtain the target data.
[0043] Optionally, the initial screening data may include multiple key-value names, such as: person includes height, degree, and certificate, certificate includes startTime and endTime; country includes province and city; address includes postalCode.
[0044] Specifically, the target data is obtained by further filtering the initial screening data based on the key-value information of the initial screening data. This includes: for the current key-value name, obtaining the key-value information corresponding to the current key-value name from the initial screening data; determining whether the key-value information is included in the pre-configured filter data value library; if it is included, deleting the current key-value name; and so on, traversing each key-value name in the multiple key-value names in turn to obtain the target data.
[0045] The filter data value library refers to the data values that the user pre-configures to filter out, such as null values, error values, or duplicate values. The target data refers to the data that needs to be retained after filtering the complex data to be filtered.
[0046] Furthermore, before determining whether the key-value information is included in the pre-configured filter data value library, the process includes: determining the fourth data type of the key-value information; if the fourth data type is a string, then the operation of determining whether the key-value information is included in the pre-configured filter data value library is performed; if the fourth data type is an object or an array, indicating that the key-value information of the current key-value name is also a preliminary screening data to be filtered again, then the key-value information is used as new preliminary screening data, and based on the pre-configured filter data value library, the operation of filtering the new preliminary screening data again based on the key-value information of the preliminary screening data to obtain the target data is repeated.
[0047] The technical solution provided in this embodiment obtains the format object of the target data template by parsing the target data template; performs initial screening on the complex data to be filtered based on the data type of the format object to obtain the initial screened data; and performs further screening on the initial screened data based on the key-value information of the initial screened data to obtain the target data. This invention parses the user-preconfigured target data template to extract the format object of the target data template; performs a deep traversal of the format object of the target data template; firstly, it performs initial screening on the complex data to be filtered based on the data type of the data block corresponding to the format object; and then performs further screening on the complex data to be filtered based on the user-configured filter data value library and the key-value information of the initial screened data, thereby achieving the filtering of complex data. Compared with the prior art, this application only requires configuring the target data template and can perform unified filtering processing on various complex data blocks without the need for custom-written specific filtering programs, thus reducing the difficulty for developers to design and debug filtering programs and improving software development efficiency.
[0048] The complex data processing method provided in the embodiments of this application is further described below. Figure 2 This is a second flowchart illustrating a complex data processing method provided in this application embodiment. This application embodiment is an optimization based on the above embodiments, specifically: the process of determining the intermediate data corresponding to the current format object during the initial screening of complex data to be filtered is explained in detail.
[0049] See Figure 2 The method in this embodiment includes, but is not limited to, the following steps:
[0050] S210. Obtain the first data block corresponding to the current format object from the complex data to be filtered, obtain the second data block corresponding to the current format object from the target data template, and determine the second data type of the second data block.
[0051] The second data type includes neither an object nor an array, either an object or an array. If the second data type is neither an object nor an array, then proceed to step S220; if the second data type is an object, then proceed to step S230; if the second data type is an array, then proceed to step S240.
[0052] S220. If the second data type is neither an object nor an array, then the first data block is used as intermediate data.
[0053] In this embodiment of the application, if the second data type of the second data block corresponding to the current format object in the target data template is neither an object nor an array (such as a string), then the first data block corresponding to the current format object in the complex data to be filtered is retained, and the first data block is used as intermediate data.
[0054] S230. If the second data type is an object, then the first data block is used as the new complex data to be filtered, and the operation of initially filtering the new complex data to be filtered based on the data type of the format object is repeated to obtain the initial screening data.
[0055] In this embodiment of the application, if the second data type of the second data block corresponding to the current format object in the target data template is an object, it indicates that the first data block is also a complex data to be filtered. Then the first data block is used as the new complex data to be filtered, and step S210 is repeated.
[0056] S240. If the second data type is an array, then determine the third data type of the first array element in the second data block.
[0057] In this embodiment, if the second data type of the second data block corresponding to the current format object in the target data template is an array, then the first data block is filtered according to the data type of the first array element in the second data block to obtain intermediate data. Specifically, the third data type of the first array element in the second data block is first determined, and the third data type includes either an object or not an object; then the processing method for the first data block is determined, and the first data block is processed based on the processing method to obtain intermediate data.
[0058] If the third data type of the first array element in the second data block is an object, then proceed to step S250; if the third data type of the first array element in the second data block is not an object, then proceed to step S260.
[0059] S250. If the third data type is an object, then each array element in the first data block is treated as a new complex data to be filtered, and the operation of initially filtering the complex data to be filtered based on the data type of the format object is repeatedly executed to obtain the initial filtered data.
[0060] In this embodiment of the application, if the third data type of the first array element in the second data block is an object, it indicates that each item in the first data block is also a complex data to be filtered. Then, each array element in the first data block is taken as a new complex data to be filtered, and step S210 is repeated in turn.
[0061] S260. If the third data type is not an object, then the first data block is used as intermediate data.
[0062] In this embodiment of the application, if the third data type of the first array element in the second data block is not an object (such as a string), then the first data block corresponding to the current format object in the complex data to be filtered is retained, and the first data block is used as intermediate data.
[0063] The technical solution provided in this embodiment obtains a first data block corresponding to the current format object from the complex data to be filtered, and a second data block corresponding to the current format object from the target data template. The second data block's second data type is determined. If the second data type is neither an object nor an array, the first data block is used as intermediate data. If the second data type is an object, the first data block is used as new complex data to be filtered, and the operation of initially filtering the new complex data based on the format object's data type to obtain initial-screened data is repeated. If the second data type is an array, the third data type of the first array element in the second data block is determined. If the third data type is an object, each array element in the first data block is used as new complex data to be filtered, and the operation of initially filtering the complex data based on the format object's data type to obtain initial-screened data is repeated. If the third data type is not an object, the first data block is used as intermediate data. This invention deeply traverses the format objects of the user-preconfigured target data template, and based on the data type of the data block corresponding to the format object, performs an initial screening of the complex data to be filtered to obtain initial-screened data. Compared to existing technologies, this application only requires configuring the target data template, enabling unified filtering of various complex data blocks without the need for custom-written filtering programs. This reduces the difficulty for developers in designing and debugging filtering programs and improves software development efficiency.
[0064] Figure 3 This is a schematic diagram of the structure of a complex data processing device provided in an embodiment of this application, as shown below. Figure 3 As shown, the device 300 may include:
[0065] The template parsing module 310 is used to parse the target data template to obtain the format object of the target data template, wherein the target data template is a template for retaining data after processing the complex data to be filtered;
[0066] The first processing module 320 is used to perform an initial screening of the complex data to be filtered based on the data type of the format object to obtain initial screening data.
[0067] The second processing module 330 is used to further filter the initial screening data based on the key-value information of the initial screening data to obtain target data.
[0068] Furthermore, the template parsing module 310 described above can be specifically used to: determine the first data type of the field in the target data template; if the first data type is an object, then determine the field as the format object; if the first data type is a string, then obtain the address suffix of the string, determine the parser corresponding to the address suffix, and parse the target data template based on the parser to obtain the format object.
[0069] Optionally, the number of the format objects can be multiple;
[0070] Furthermore, the aforementioned first processing module 320 can be specifically used to: for the current format object, obtain the corresponding first data block from the complex data to be filtered, and obtain the corresponding second data block from the target data template; determine the second data type of the second data block, and use a preset filtering rule to filter the first data block based on the second data type to obtain intermediate data; sequentially traverse multiple format objects to obtain multiple intermediate data, and merge the multiple intermediate data as the initial screening data.
[0071] Furthermore, the first processing module 320 described above can also be specifically used for: if the second data type is neither an object nor an array, then the first data block is used as the intermediate data; if the second data type is an object, then the first data block is used as new complex data to be filtered, and the operation of initially filtering the complex data to be filtered based on the data type of the format object to obtain the initial filtered data is repeated; if the second data type is an array, then the first data block is filtered based on the data type of the first array element in the second data block to obtain intermediate data.
[0072] Furthermore, the first processing module 320 described above can also be specifically used to: determine the third data type of the first array element in the second data block; if the third data type is an object, then each array element in the first data block is used as a new complex data to be filtered, and the operation of initially filtering the complex data to be filtered based on the data type of the format object is repeatedly executed to obtain the initial filtered data; if the third data type is not an object, then the first data block is used as the intermediate data.
[0073] Optionally, the initial screening data includes multiple key-value names;
[0074] Furthermore, the second processing module 330 described above can be specifically used to: obtain the corresponding key value information from the initial screening data for the current key value name; determine whether the key value information is included in the pre-configured filter data value library; if it is included, delete the current key value name; and sequentially traverse multiple key value names to obtain the target data.
[0075] Furthermore, the second processing module 330 described above can also be specifically used to: determine the fourth data type of the key-value information before determining whether the key-value information is included in the pre-configured filter data value library; if the fourth data type is a string, then perform the operation of determining whether the key-value information is included in the pre-configured filter data value library; if the fourth data type is an object or an array, then use the key-value information as new initial screening data, and repeatedly perform the operation of re-screening the initial screening data based on the key-value information of the initial screening data to obtain the target data.
[0076] The complex data processing apparatus provided in this embodiment can be applied to the complex data processing methods provided in any of the above embodiments, and has corresponding functions and beneficial effects.
[0077] Figure 4 This is a block diagram of an electronic device used to implement a complex data processing method according to embodiments of this application. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present application described and / or claimed herein.
[0078] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0079] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0080] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as complex data processing methods.
[0081] In some embodiments, the complex data processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the complex data processing method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the complex data processing method by any other suitable means (e.g., by means of firmware).
[0082] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0083] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0084] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0085] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0086] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.
[0087] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.
[0088] Note that the above are merely preferred embodiments and technical principles applied in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. For example, those skilled in the art can use the various forms of processes shown above to reorder, add, or delete steps; the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solution of this application can be achieved, and no limitations are imposed herein.
[0089] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. A method for processing complex data, characterized in that, The method includes: Parsing a target data template to obtain a format object of the target data template includes: determining the first data type of a field in the target data template; if the first data type is an object, then determining the field as the format object; if the first data type is a string, then obtaining the address suffix of the string, determining the parser corresponding to the address suffix, and parsing the target data template based on the parser to obtain the format object, wherein the target data template is a template for retaining data after processing the complex data to be filtered; The initial screening data is obtained by performing an initial screening of the complex data to be filtered based on the data type of the format object. The target data is obtained by further filtering the initial screening data based on the key-value information of the initial screening data.
2. The complex data processing method according to claim 1, characterized in that, The number of format objects is multiple; the initial screening of the complex data to be filtered based on the data type of the format objects to obtain the initial screened data includes: For the current format object, obtain the corresponding first data block from the complex data to be filtered, and obtain the corresponding second data block from the target data template; Determine the second data type of the second data block, and use a preset filtering rule to filter the first data block based on the second data type to obtain intermediate data; After iterating through multiple format objects, multiple intermediate data are obtained, and the multiple intermediate data are merged to form the initial screening data.
3. The complex data processing method according to claim 2, characterized in that, The step of filtering the first data block based on the second data type using preset filtering rules to obtain intermediate data includes: If the second data type is neither an object nor an array, then the first data block is used as the intermediate data; If the second data type is an object, then the first data block is used as the new complex data to be filtered, and the operation of initially filtering the complex data to be filtered based on the data type of the format object to obtain the initial screened data is repeated. If the second data type is an array, then the first data block is filtered based on the data type of the first array element in the second data block to obtain intermediate data.
4. The complex data processing method according to claim 3, characterized in that, The step of filtering the first data block based on the data type of the first array element in the second data block to obtain intermediate data includes: Determine the third data type of the first array element in the second data block; If the third data type is an object, then each array element in the first data block is taken as a new complex data to be filtered, and the operation of initially filtering the complex data to be filtered based on the data type of the format object is repeated to obtain the initial screened data. If the third data type is not an object, then the first data block is used as the intermediate data.
5. The complex data processing method according to claim 1, characterized in that, The initial screening data includes multiple key-value names; the process of further filtering the initial screening data based on the key-value information to obtain target data includes: For the current key-value name, obtain the corresponding key-value information from the initial screening data; Determine whether the key-value information is included in a pre-configured filtered data value library; If it is included, then delete the current key-value name; The target data is obtained by iterating through multiple key-value names in sequence.
6. The complex data processing method according to claim 5, characterized in that, Before determining whether the key-value information is included in the pre-configured filtered data value library, the method further includes: Determine the fourth data type of the key-value information; If the fourth data type is a string, then the operation of determining whether the key-value information is included in the pre-configured database of data values to be filtered is performed; If the fourth data type is an object or an array, then the key-value information is used as new initial screening data, and the operation of filtering the initial screening data again based on the key-value information of the initial screening data to obtain the target data is repeated.
7. A complex data processing device, characterized in that, The device includes: The template parsing module is used to parse the target data template to obtain the format object of the target data template, wherein the target data template is the template for retaining data after processing the complex data to be filtered; The first processing module is used to perform an initial screening of the complex data to be filtered based on the data type of the format object to obtain the initial screening data. The second processing module is used to further filter the initial screening data based on the key value information of the initial screening data to obtain the target data; Specifically, the template parsing module is used to determine the first data type of the fields in the target data template; if the first data type is an object, then the field is determined as the format object; if the first data type is a string, then the address suffix of the string is obtained, the parser corresponding to the address suffix is determined, and the target data template is parsed based on the parser to obtain the format object.
8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the complex data processing method according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the complex data processing method according to any one of claims 1 to 6.