Data processing method and device

Abstract

An embodiment of the present invention provides a data processing method and device, including: obtaining portrait data to be processed; determining multiple processing plug-ins for processing the portrait data to be processed, and the processing order between the multiple processing plug-ins; connecting the multiple processing plug-ins according to the processing order to obtain a portrait processing file; performing portrait processing on the portrait data to be processed according to the portrait processing file to obtain result data. By abstracting the portrait processing function into a series of processing plug-ins, it is convenient to expand different portrait processing plug-ins as needed. When the portrait system needs to change the processing parameters or even the processing type, the portrait processing file can be directly modified without modifying the system code, and flexible configuration can be achieved. Compared with this general streaming processing solution based on Flink, the present invention is specifically designed for portrait processing and storage, and can achieve high performance and high resource utilization.

Description

Technical Field

[0001] The present invention relates to the field of computer application technology, and in particular to a data processing method and a data processing device. Background Art

[0002] The recommendation project is a user personalized data generation system, that is, based on the user's personalized behavior and data, the data returned to the user is prioritized, and the data that the user is most interested in is returned first, thereby improving user satisfaction with the service and increasing user stickiness. In the recommendation project, the basis for providing personalized services to users with "thousands of faces" lies in the collection, processing and storage of user behavior data. The portrait system is the most important module in the recommendation foundation. Figure 1 As shown in the figure, in the recommendation project, steps such as recall, filtering, sorting, and rearrangement all rely on user data provided by the portrait system.

[0003] In existing technologies, image processing and access solutions are usually implemented based on Flink and big data storage suites. However, the shortcomings of image systems built based on Flink and big data suites are as follows:

[0004] 1. Difficulty in secondary development and management. Because Flink and Big Data Suite solutions are general-purpose stream processing solutions and are not specifically designed for image processing systems, a large amount of secondary development is required to implement image processing, including functional development and management function development.

[0005] 2. Low performance. The Flink-based image processing model requires a large number of high-cost operations such as table joins, resulting in relatively low performance and requiring more computing and storage resources. Summary of the Invention

[0006] In view of the problems of the above-mentioned screen system in secondary development and management difficulties and low performance, the embodiments of the present invention are proposed to provide a data processing method and a corresponding data processing device that overcome the above-mentioned problems or at least partially solve the above-mentioned problems.

[0007] An embodiment of the present invention discloses a data processing method, comprising:

[0008] Get the portrait data to be processed;

[0009] Determining a plurality of processing plug-ins for processing the portrait data to be processed, and a processing order among the plurality of processing plug-ins;

[0010] Connecting the plurality of processing plug-ins according to the processing sequence to obtain a portrait processing file;

[0011] The image data to be processed is subjected to image processing according to the image processing file to obtain result data.

[0012] Optionally, connecting the multiple processing plug-ins according to the processing order to obtain the image processing file includes:

[0013] Based on the IDL interactive data language, the multiple processing plug-ins are connected according to the processing sequence to obtain the image processing file.

[0014] Optionally, determining a plurality of processing plug-ins for processing the portrait data to be processed includes:

[0015] Determining the data type corresponding to the portrait data to be processed;

[0016] According to the data type, a plurality of processing plug-ins for processing the portrait data to be processed are determined.

[0017] Optionally, the image processing file includes multiple processing chains.

[0018] Optionally, performing image processing on the image data to be processed according to the image processing file to obtain result data includes:

[0019] The plurality of processing chains are respectively used to perform image processing on the image data to be processed to obtain result data.

[0020] Optionally, the method further includes:

[0021] An image management interface is displayed according to the image processing file, wherein the image management interface includes the multiple processing chains that have been connected.

[0022] Optionally, the multiple processing plug-ins have one-to-one corresponding image processing functions.

[0023] The embodiment of the present invention further discloses a data processing device, comprising:

[0024] A portrait data acquisition module, used to acquire portrait data to be processed;

[0025] a processing plug-in determining module, configured to determine a plurality of processing plug-ins for processing the image data to be processed, and a processing order among the plurality of processing plug-ins;

[0026] a portrait processing file generating module, configured to connect the plurality of processing plug-ins according to the processing sequence to obtain a portrait processing file;

[0027] The image processing module is used to perform image processing on the image data to be processed according to the image processing file to obtain result data.

[0028] Optionally, the image processing file generation module includes:

[0029] The plug-in connection submodule is used to connect the multiple processing plug-ins according to the processing sequence based on the IDL interactive data language to obtain the image processing file.

[0030] Optionally, the processing plug-in determination module includes:

[0031] A data type determination submodule, used to determine the data type corresponding to the portrait data to be processed;

[0032] The processing plug-in determination submodule is used to determine a plurality of processing plug-ins for processing the portrait data to be processed according to the data type.

[0033] Optionally, the image processing file includes multiple processing chains.

[0034] Optionally, the image processing module includes:

[0035] The portrait processing submodule is used to respectively use the multiple processing chains to perform portrait processing on the portrait data to be processed to obtain result data.

[0036] Optionally, the device further comprises:

[0037] The portrait management interface display module is used to display the portrait management interface according to the portrait processing file, and the portrait management interface includes the multiple processing chains that have been connected.

[0038] Optionally, the multiple processing plug-ins have one-to-one corresponding image processing functions.

[0039] An embodiment of the present invention further discloses an electronic device, including:

[0040] A processor and a storage medium, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor executes the machine-readable instructions to perform the data processing method as described in any one of the embodiments of the present invention.

[0041] An embodiment of the present invention further discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the data processing method as described in any one of the embodiments of the present invention is executed.

[0042] The embodiments of the present invention include the following advantages:

[0043] In a data processing method provided by an embodiment of the present invention, the process involves obtaining portrait data to be processed; determining a plurality of processing plug-ins for processing the portrait data to be processed, and a processing order between the plurality of processing plug-ins; connecting the plurality of processing plug-ins according to the processing order to obtain a portrait processing file; and performing portrait processing on the portrait data to be processed according to the portrait processing file to obtain result data. By abstracting the portrait processing function into a series of processing plug-ins, it is convenient to expand different portrait processing plug-ins as needed. When the portrait system needs to change processing parameters or even change the processing type, the portrait processing file can be directly modified without modifying the system code, and flexible configuration can be achieved. Compared with this general streaming processing solution based on Flink, the present invention is specifically designed for portrait processing and storage, and uses different data structures and memories to process and store portraits of different data types, achieving high performance and high resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the description of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 It is a structural diagram of a recommended project;

[0046] Figure 2 This is a flowchart of a data processing method provided by an embodiment of the present invention;

[0047] Figure 3 This is a flowchart of the steps of a process for determining a processing plug-in provided by an embodiment of the present invention;

[0048] Figure 4 is a schematic diagram of a connection-completed processing chain provided by an embodiment of the present invention;

[0049] Figure 5 is a structural block diagram of a data processing device provided by an embodiment of the present invention;

[0050] Figure 6 It is a structural block diagram of an electronic device of the present invention;

[0051] Figure 7 It is a structural block diagram of a computer-readable storage medium of the present invention. DETAILED DESCRIPTION

[0052] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments described are only a portion of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are also within the scope of protection of the present invention.

[0053] In the related art, the portrait system built based on Flink and big data suite has the problem of difficult secondary development and management and low performance. In order to solve this problem, an embodiment of the present invention provides a description and implementation framework of portrait chain processing based on IDL (Interactive Data Language), which uses the flexibility of the chain structure to solve a variety of portrait processing operations, specifically by obtaining the portrait data to be processed; determining multiple processing plug-ins for processing the portrait data to be processed, and the processing order between the multiple processing plug-ins; connecting the multiple processing plug-ins according to the processing order to obtain a portrait processing file; performing portrait processing on the portrait data to be processed according to the portrait processing file to obtain result data. The portrait processing system provided in the embodiment of the present invention has the following advantages:

[0054] 1. Easy to expand. Different image processing plug-ins can be expanded according to the agreed interface. The interface can be run locally or remotely, and each interface interacts with each other through direct network protocol.

[0055] 2. Flexible configuration: When the portrait system needs to change processing parameters or even the processing type, the portrait IDL can be modified directly without modifying the system code, allowing for flexible configuration.

[0056] 3. High performance. Compared with general-purpose stream processing solutions like Flink, this invention is specifically designed for image processing and storage. It uses different data structures and memories to process and store images of different data types, achieving high performance and high resource utilization.

[0057] Reference Figure 2 , shows a flowchart of a method for processing portrait data according to an embodiment of the present invention, which may specifically include the following steps:

[0058] Step S201, obtaining the portrait data to be processed;

[0059] Among them, the portrait data to be processed refers to user data that needs to be analyzed, mainly involving some user data related to user behavior habits. In some scenarios, it is necessary to analyze user behavior habits in order to make accurate recommendations to users. For example, based on scenarios such as selecting people, goods, and venues in e-commerce business, merchants or brands often need to obtain portrait data for portrait processing.

[0060] In a specific implementation, the portrait data to be processed can be collected through the user's operation records in the background of the platform, or can be collected through online or offline surveys, etc. The embodiment of the present invention does not limit this.

[0061] Step S202, determining a plurality of processing plug-ins for processing the portrait data to be processed, and a processing order among the plurality of processing plug-ins;

[0062] In an embodiment of the present invention, each processing plug-in has a one-to-one corresponding portrait processing function, such as indexing, compression, normalization, timeout processing, post-processing and other portrait processing functions. These portrait processing functions are abstracted into a series of processing plug-ins.

[0063] It should be noted that each processing plug-in of the present invention can be developed based on different frameworks. For example, the processing plug-in can be a CS architecture, placing a "thin" C-end in the image processing system, and at the same time running the "fat" S-end that consumes a lot of computing resources separately in a multi-instance manner, thereby achieving unlimited support for processing capabilities. For example, the index plug-in can be regarded as a CS architecture plug-in.

[0064] In an embodiment of the present invention, first, multiple processing plug-ins required to analyze the portrait data to be processed are determined, and the processing order between the multiple processing plug-ins is determined, so that in subsequent steps, these processing plug-ins are used in sequence according to the processing order between the processing plug-ins to perform portrait processing and storage on the portrait data to be processed.

[0065] Step S203, connecting the multiple processing plug-ins according to the processing sequence to obtain a portrait processing file;

[0066] In an embodiment of the present invention, multiple processing plug-ins can be connected according to their processing order to generate an image processing file. When connecting the processing plug-ins, they can be connected into multiple processing chains, each of which can independently process and store the image data to be processed. Because the dependencies between the various processing modules in the image processing file are clear, the processing plug-in interface is simple, making function development and management very convenient.

[0067] In practice, each processing plug-in is a reusable and configurable processing program. Furthermore, each plug-in has different input parameter settings. When the profiling system needs to change processing parameters or even the processing type, the profiling file can be modified directly without modifying the profiling system code, allowing for flexible configuration.

[0068] Step S204: performing image processing on the image data to be processed according to the image processing file to obtain result data.

[0069] In an embodiment of the present invention, image processing is performed on the image data to be processed using a processing plug-in described in the image processing file according to the image processing file to obtain result data. It should be noted that the processing plug-in described in the image processing file can have various different memories. After the image processing is performed on the image data to be processed to obtain result data, the generated result data can also be stored in the memory of the processing plug-in. This allows different data structures and memories to be used for processing and storage of images of different data types, thereby achieving high performance and high resource utilization.

[0070] In a preferred embodiment of the present invention, connecting the multiple processing plug-ins according to the processing sequence to obtain the image processing file includes:

[0071] Based on the IDL interactive data language, the multiple processing plug-ins are connected according to the processing sequence to obtain the image processing file.

[0072] IDL is a data analysis and visualization application and programming language. It's a visual language that combines openness, high-dimensional analysis capabilities, scientific computing capabilities, practicality, and visual analysis. It runs on a variety of hardware platforms, easily connects to C and C++, and supports the ODBC (Open Database Connectivity) database interface standard. IDL's built-in mathematical library functions significantly reduce the workload of developing image processing algorithms. Programs written in IDL can run unchanged on other IDL-capable platforms, making the resulting systems inherently portable.

[0073] In this embodiment of the present invention, multiple processing plug-ins are connected according to the processing sequence based on the interactive data language (IDL) to generate an image processing file. When the image processing system needs to change processing parameters or even the processing type, the image processing file can be modified directly based on IDL, without modifying the system code, allowing for flexible configuration.

[0074] In a preferred embodiment of the present invention, Figure 3As shown, the step S202 includes steps S301-S302: wherein, in step S301, the data type corresponding to the portrait data to be processed is determined; and in step S302, a plurality of processing plug-ins for processing the portrait data to be processed are determined according to the data type.

[0075] In an embodiment of the present invention, for the portrait data to be processed, the user can define the data type corresponding to the portrait data to be processed, so that multiple processing plug-ins for processing the portrait data to be processed can be determined based on the data type corresponding to the portrait data to be processed. In addition, if the user does not define the data type corresponding to the portrait data to be processed, the data type corresponding to the portrait data to be processed is determined as a default data type, and multiple processing plug-ins for processing the portrait data to be processed are determined based on the default data type.

[0076] Since the processing plug-in for processing the portrait data to be processed is determined according to the data type of the portrait data to be processed, the portrait system of the present invention can use different data structures for portrait processing and storage according to the characteristics of different types of portrait data.

[0077] In a preferred embodiment of the present invention, the image processing file includes multiple processing chains.

[0078] Specifically, each portrait data to be processed can be configured with multiple processing chains for portrait processing, that is, multiple processing chains can be connected into multiple processing chains based on IDL, so that each processing chain can realize different aspects of portrait processing on the portrait data to be processed, thereby enhancing the accuracy of the portrait processing results.

[0079] As an example, the image processing file is as follows:

[0080] message P1{

[0081] option(write.plugin)="A1";

[0082] option(write.plugin)="A2|index";

[0083] option(write.plugin)="A3|async";

[0084] option(write.plugin)="B1|A1";

[0085] option(read.plugin)="C1";}

[0086] The above-mentioned portrait IDL (i.e., portrait processing file) defines that portrait P1 has fields f1, f2, etc. At the same time, portrait P1 has 3 processing chains, including 2 write processing chains and 1 read processing chain. The schematic diagram of the portrait processing process is as follows Figure 4 shown. Figure 4 In the example, the first chain is chain A, used for writing. Chain A includes plugins 1, 2, and 3, as well as storage. Plugin 2 in chain A is an indexing plugin, with the index key set to index. Plugin 3 in chain A is a post-processing plugin, also an asynchronous processing plugin. Chain B is used for writing, and includes plugin 1 and storage. Chain C is used for reading, and includes plugins 1, 2, and storage. Data from plugin 2 in chain C can be called back to plugin 1.

[0087] It should be noted that the above-mentioned portrait processing file is only an example. In a specific implementation, the processing plug-ins contained in the portrait processing file can be set according to the needs of processing data, and different processing chains can be connected through these processing plug-ins. The embodiment of the present invention does not limit this.

[0088] In this embodiment of the present invention, a profiling system is configured using multiple processing plug-ins. This system supports different types of storage, such as Redis, Pika, and DGraph, allowing users to select different storage options to store the resulting profiling data based on data storage requirements. Furthermore, it supports the use of indexing technology to accelerate data location and caching technology to speed up data access.

[0089] In a preferred embodiment of the present invention, performing image processing on the image data to be processed according to the image processing file to obtain result data includes:

[0090] The plurality of processing chains are respectively used to perform image processing on the image data to be processed to obtain result data.

[0091] In embodiments of the present invention, multiple processing chains in the image processing file can be used to perform image processing on the image data to be processed to obtain result data. For example, the two write processing chains and one read processing chain in the above example can be used to perform image processing on the image data to be processed to obtain result data.

[0092] In a preferred embodiment of the present invention, the method further comprises:

[0093] A portrait management interface is displayed, wherein the portrait management interface includes the multiple processing chains that have been connected.

[0094] In this embodiment of the present invention, a portrait management interface can also be displayed, which includes multiple connected processing chains. This allows users to intuitively understand the portrait processing and storage type of each portrait data to be processed, display and monitor the data, and facilitate flexible configuration of portrait processing files.

[0095] In an embodiment of the present invention, the process is to obtain portrait data to be processed; determine multiple processing plug-ins for processing the portrait data to be processed, and the processing order between the multiple processing plug-ins; connect the multiple processing plug-ins according to the processing order to obtain a portrait processing file; perform portrait processing on the portrait data to be processed according to the portrait processing file to obtain result data. By abstracting the portrait processing function into a series of processing plug-ins, it is convenient to expand different portrait processing plug-ins as needed. When the portrait system needs to change the processing parameters or even the processing type, the portrait processing file can be directly modified without modifying the system code, and flexible configuration can be achieved. Compared with this general streaming processing solution based on Flink, the present invention is specifically designed for portrait processing and storage, and uses different data structures and memories to process and store portraits of different data types, so as to achieve high performance and high resource utilization.

[0096] It should be noted that for the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0097] Reference Figure 5 , shows a structural block diagram of a data processing device provided by an embodiment of the present invention, which may specifically include the following modules:

[0098] The portrait data acquisition module 501 is used to acquire the portrait data to be processed;

[0099] A processing plug-in determination module 502 is configured to determine a plurality of processing plug-ins for processing the image data to be processed, and a processing order among the plurality of processing plug-ins;

[0100] A portrait processing file generating module 503 is configured to connect the plurality of processing plug-ins according to the processing sequence to obtain a portrait processing file;

[0101] The portrait processing module 504 is used to perform portrait processing on the portrait data to be processed according to the portrait processing file to obtain result data.

[0102] In a preferred embodiment of the present invention, the image processing file generation module 503 includes:

[0103] The plug-in connection submodule is used to connect the multiple processing plug-ins according to the processing sequence based on the IDL interactive data language to obtain the image processing file.

[0104] In a preferred embodiment of the present invention, the processing plug-in determination module 502 includes:

[0105] A data type determination submodule, used to determine the data type corresponding to the portrait data to be processed;

[0106] The processing plug-in determination submodule is used to determine a plurality of processing plug-ins for processing the portrait data to be processed according to the data type.

[0107] In a preferred embodiment of the present invention, the image processing file includes multiple processing chains.

[0108] In a preferred embodiment of the present invention, the image processing module 504 includes:

[0109] The portrait processing submodule is used to respectively use the multiple processing chains to perform portrait processing on the portrait data to be processed to obtain result data.

[0110] In a preferred embodiment of the present invention, the device further comprises:

[0111] The portrait management interface display module is used to display the portrait management interface according to the portrait processing file, and the portrait management interface includes the multiple processing chains that have been connected.

[0112] In a preferred embodiment of the present invention, the plurality of processing plug-ins have one-to-one corresponding image processing functions.

[0113] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0114] The embodiment of the present invention further provides an electronic device, such as Figure 6 Shown, including:

[0115] A processor 601 and a storage medium 602, wherein the storage medium 602 stores machine-readable instructions executable by the processor 601. When the electronic device is running, the processor 601 executes the machine-readable instructions to perform the data processing method according to any one of the embodiments of the present invention, including:

[0116] Get the portrait data to be processed;

[0117] Determining a plurality of processing plug-ins for processing the portrait data to be processed, and a processing order among the plurality of processing plug-ins;

[0118] Connecting the plurality of processing plug-ins according to the processing sequence to obtain a portrait processing file;

[0119] The image data to be processed is subjected to image processing according to the image processing file to obtain result data.

[0120] Optionally, connecting the multiple processing plug-ins according to the processing order to obtain the image processing file includes:

[0121] Based on the IDL interactive data language, the multiple processing plug-ins are connected according to the processing sequence to obtain the image processing file.

[0122] Optionally, determining a plurality of processing plug-ins for processing the portrait data to be processed includes:

[0123] Determining the data type corresponding to the portrait data to be processed;

[0124] According to the data type, a plurality of processing plug-ins for processing the portrait data to be processed are determined.

[0125] Optionally, the image processing file includes multiple processing chains.

[0126] Optionally, performing image processing on the image data to be processed according to the image processing file to obtain result data includes:

[0127] The plurality of processing chains are respectively used to perform image processing on the image data to be processed to obtain result data.

[0128] Optionally, the method further includes:

[0129] An image management interface is displayed according to the image processing file, wherein the image management interface includes the multiple processing chains that have been connected.

[0130] Optionally, the multiple processing plug-ins have one-to-one corresponding image processing functions.

[0131] The embodiment of the present invention also provides a computer readable storage medium, such as Figure 7 As shown, the storage medium stores a computer program 701, which is executed by a processor to execute the data processing method according to any one of the embodiments of the present invention. The specific implementation and technical effects are similar and will not be described in detail here.

[0132] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0133] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, apparatus, or computer program products. Thus, embodiments of the present invention may take the form of a fully hardware embodiment, a fully software embodiment, or an embodiment combining software and hardware. Furthermore, embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0135] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0136] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0137] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0138] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0139] The data processing method and data processing device provided by the present invention are introduced in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of the present invention, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A data processing method, characterized in that: include: Obtain the portrait data to be processed by the recommendation system; Determining the data type corresponding to the portrait data to be processed; Analyzing, according to the data type, a plurality of processing plug-ins required for the portrait data to be processed, and determining a processing order among the plurality of processing plug-ins; Based on the IDL interactive data language and its built-in library functions, generating a portrait processing file according to the processing sequence; Performing image processing on the image data to be processed according to the image processing file to obtain result data; The image processing file contains multiple processing chains defined for the image data, each of the processing chains contains a function definition of the processing chain, and the function definition is used to indicate whether the processing chain is a write processing chain or a read processing chain; Furthermore, each of the processing chains includes at least two interconnected processing plug-ins, each of the processing plug-ins has a corresponding image processing function, and reusable processing plug-ins may or may not exist between different processing chains; Furthermore, each processing chain has a one-to-one corresponding memory, and the memory corresponds to the function definition and the image processing function; Furthermore, the image processing file contains a parameter change interface for each processing plug-in, wherein the parameter change interface is used to adjust the configuration of the corresponding processing plug-in, and the parameter change interface is different from the system code in the recommendation system; And, the method further comprises: An image management interface is displayed according to the image processing file, wherein the image management interface includes the multiple processing chains that have been connected.

2. A data processing device, characterized in that: The device is used to execute the data processing method according to claim 1, and the device includes: A portrait data acquisition module, used to acquire portrait data to be processed; a processing plug-in determining module, configured to determine a plurality of processing plug-ins for processing the image data to be processed, and a processing order among the plurality of processing plug-ins; a portrait processing file generating module, configured to connect the plurality of processing plug-ins according to the processing sequence to obtain a portrait processing file; The image processing module is used to perform image processing on the image data to be processed according to the image processing file to obtain result data.

3. An electronic device, characterized in that: include: A processor and a storage medium, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor executes the machine-readable instructions to perform the data processing method according to claim 1.

4. A computer-readable storage medium, characterized in that The storage medium stores a computer program, which, when executed by a processor, executes the data processing method according to claim 1 .

Images