A data processing method, apparatus, device, medium, and product

Abstract

The application discloses a data processing method, device, equipment, medium and product. The method comprises the following steps: obtaining to-be-processed data and target business logic; obtaining data source labels corresponding to the to-be-processed data in a preset data resource manager before a preset first management container calls the target business logic, determining target identifiers of data source connection information corresponding to the data source labels, and storing the target identifiers in the preset data resource manager; obtaining the target identifiers in the data resource manager before a preset second management container executes the target business logic, and judging whether the target identifiers are valid; determining a processing strategy of the to-be-processed data according to a judgment result, and executing the target business logic to process the to-be-processed data according to the processing strategy. The technical scheme of the embodiment of the application reduces the waste of development resources caused by development for different data sources, and reduces the complexity of program development.

Description

Technical Field

[0001] The present application relates to the field of program development technology, and in particular to a data processing method, apparatus, device, medium and product. Background Art

[0002] With the development of society and the continuous advancement of science and technology, people's needs in production and daily life are increasing. More and more of these needs are being met through internet and digital technologies. This requires relevant technical personnel to develop more digital entities that can meet people's needs. Regardless of the development model used, business system development requires interaction with data sources, and all business actions are supported by data from these data sources.

[0003] At present, from the initial informatization to digital reform, the business systems being developed are not limited to realizing a single business function, but are further connected with different systems. At the same time, there are more and more ways to input and output data in business systems, resulting in different development methods for different data sources, which easily wastes development resources. Summary of the Invention

[0004] The present application provides a data processing method, apparatus, device, medium and product to reduce the waste of development resources.

[0005] According to one aspect of the present application, a data processing method is provided, comprising:

[0006] Obtain the data to be processed and the target business logic;

[0007] Before the preset first management container calls the target business logic, it obtains the data source annotation corresponding to the to-be-processed data in the preset data resource manager, determines the target identifier of the data source connection information corresponding to the data source annotation, and stores the target identifier in the preset data source manager;

[0008] Before the preset second management container executes the target business logic, obtain the target identifier in the data source manager and determine whether the target identifier is valid;

[0009] Based on the judgment result, the processing strategy of the data to be processed is determined, and the target business logic is executed according to the processing strategy to process the data to be processed.

[0010] According to another aspect of the present application, there is provided a data processing device, comprising:

[0011] Data acquisition module, used to obtain data to be processed and target business logic;

[0012] An identity determining module is configured to acquire a data source label corresponding to the to-be-processed data in a preset data resource manager before a preset first management container invokes the target business logic, determine a target identity of data source connection information corresponding to the data source label, and store the target identity in the preset data source manager.

[0013] An identity judging module is configured to acquire the target identity in the data source manager before a preset second management container executes the target business logic, and judge whether the target identity is valid.

[0014] A business processing module is configured to determine a processing strategy of the to-be-processed data according to the judgment result, and perform the target business logic on the to-be-processed data according to the processing strategy.

[0015] According to another aspect of the present application, an electronic device is provided, which comprises:

[0016] at least one processor; and

[0017] a memory connected with the at least one processor in communication; wherein

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the data processing method according to any one of the embodiments of the present application.

[0019] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to implement the data processing method according to any one of the embodiments of the present application when executed by the processor.

[0020] According to another aspect of the present application, a computer program product is provided, which comprises a computer program for implementing the data processing method according to any one of the embodiments of the present application when executed by a processor.

[0021] In the technical solution of the embodiment of the present application, before the preset first management container calls the target business logic, the data source annotation corresponding to the data to be processed is obtained in the preset data resource manager, and the target identifier of the data source connection information corresponding to the data source annotation is determined, and the target identifier is stored in the preset data source manager; before the preset second management container executes the target business logic, the target identifier in the data source manager is obtained, and it is determined whether the target identifier is valid; based on the judgment result, the processing strategy of the data to be processed is determined, and the target business logic is executed according to the processing strategy to process the data to be processed. During the development of the program, the data source annotation is intercepted and checked in the first management container and the second management container respectively to determine the development method actually corresponding to the data source, so that different data source development methods are compatible in the same development process, which minimizes the waste of development resources caused by developing for different data sources, while reducing the complexity of program development and improving the developer's experience.

[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] Figure 1 This is a flow chart of a data processing method provided according to the first embodiment of the present application;

[0025] Figure 2 is a flow chart of a data processing method provided according to the second embodiment of the present application;

[0026] Figure 3 This is a structural diagram of a data processing device provided according to the third embodiment of the present application;

[0027] Figure 4 It is a structural diagram of an electronic device for implementing the data processing method of an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] Example 1

[0031] Figure 1 A flowchart of a data processing method is provided for the first embodiment of the present application. This embodiment is applicable to the case where programs of different development methods are developed on the same platform. The method can be executed by a flowchart device of a data processing method. The flowchart device of a data processing method can be implemented in the form of hardware and / or software. The flowchart device of a data processing method can be configured in an electronic device. Figure 1 As shown, the method includes:

[0032] S110: Obtain data to be processed and target business logic.

[0033] Among them, the data to be processed can be data that needs to be processed when developing a business system or developing functions associated with different systems for the business system. The business system can be any system developed by relevant technical personnel to realize a certain business function. The target business logic can be at least a part of the development logic for developing the business function. It is understandable that in the process of developing a business system, no matter which development model is adopted, it is necessary to interact with the data source, all business actions are derived from the data support of the data source, and the data to be processed is stored in each different data source. The type of data source can include but is not limited to structured databases, unstructured databases, memory databases, text databases and network interfaces, etc., and the embodiments of the present application are not limited to this. Of course, the data to be processed and the target business logic can be directly obtained when the business program or business system is developed, and the specific acquisition process varies according to the development process.

[0034] S120. Before the preset first management container calls the target business logic, obtain the data source label corresponding to the data to be processed in the preset data resource manager, determine the target identifier of the data source connection information corresponding to the data source label, and store the target identifier in the preset data source manager.

[0035] Among them, the first management container can be the core component of the entire development framework responsible for managing the life cycle, dependency injection and configuration of objects in the application during the program development process. For example, in the embodiment of the present application, the spring container in the spring framework can be used as the first management container. The data resource manager can be used to establish a connection with the data source manager to facilitate the identification and calling of the data source. The data source manager can be used to define and manage the connection information (such as connection interface information and connection method, etc.) between each different data source and the development framework. The data resource manager and the data source manager can be constructed using the server or the hardware device that the development framework relies on, and this embodiment of the present application does not limit this.

[0036] Data source annotations can be pre-defined annotations for different data sources during application development. Data source annotations can store annotation information and associate it with the data source in a predetermined manner. For example, during program development, developers can use a mapper class to construct the corresponding data source within the mapper method. This annotation records the data source identifier corresponding to the data source. For example, the Desired State Configuration (DSC) annotation can be used as a data source annotation. This DSC annotation records the data source identifier information. By parsing the DSC annotation, the data source identifier recorded can be obtained. The target identifier, i.e., the data source identifier of the data source to be processed, records the data source connection information. Therefore, it can be understood that when developers develop the program, they parse the pre-defined data source annotation to obtain the data source identifier of the corresponding data source. Then, by identifying the data source identifier, they determine the connection information for that data source.

[0037] Specifically, the first management container in the development framework used during program development retrieves the data source annotations corresponding to the data to be processed from a pre-built data resource manager before invoking the target business logic for program development. These data source annotations are also pre-defined and can be directly retrieved. These data source annotations are parsed to obtain the target identifier corresponding to the data to be processed, which is then stored in the data source manager for future development and use.

[0038] S130: Before the preset second management container executes the target business logic, obtain the target identifier in the data source manager and determine whether the target identifier is valid.

[0039] Among them, the second management container can be a development component located after the first management container in the program development framework. The second management container can be used to perform relevant configuration and management of the database, so that the database can be operated in a timely manner through the second management container during program development. Exemplarily, the second management container can adopt the mybatis container, which is used in combination with the spring container in the above example. For example, the spring container is responsible for managing the data source container. When the database operation transactions of mybatis are uniformly coordinated through the data resource container, the main configuration file of mybatis no longer needs to define the data source and transaction manager. It is only necessary to configure the settings related to SQL (Structured Query Language) mapping (such as aliases, plug-ins, etc.).

[0040] Before executing the target business logic, the second management container obtains the target identifier stored in the data source manager from the aforementioned steps and identifies it in the second management container to determine whether the target identifier is valid. The validity determination criteria include at least two factors: whether the target identifier exists and whether a corresponding data source exists for the existing target identifier.

[0041] S140: Determine a processing strategy for the data to be processed based on the judgment result, and execute the target business logic to process the data to be processed according to the processing strategy.

[0042] According to the judgment result of whether the target identifier is valid in the above steps, different processing strategies for the data to be processed are determined. That is to say, the processing strategies for the data to be processed are different when the target identifier is valid and when the target identifier is invalid. For example, if the target identifier is valid, it proves that the corresponding data source of the data to be processed exists, and the corresponding target business logic can be executed according to the data type and / or program development method of the data source to process the data to be processed; on the contrary, if the target identifier is invalid, it means that the target identifier does not exist or the data source corresponding to the target identifier does not exist, that is, there is no data type and / or program development method that can be used as a reference, then the target business logic can be executed according to the pre-set default method, and a bunch of data to be processed can be processed. Of course, the default method can be pre-set by the relevant developers according to the actual situation, and the embodiments of the present application do not limit this.

[0043] In the technical solution of the embodiment of the present application, before the preset first management container calls the target business logic, the data source annotation corresponding to the data to be processed is obtained in the preset data resource manager, and the target identifier of the data source connection information corresponding to the data source annotation is determined, and the target identifier is stored in the preset data source manager; before the preset second management container executes the target business logic, the target identifier in the data source manager is obtained, and it is determined whether the target identifier is valid; based on the judgment result, the processing strategy of the data to be processed is determined, and the target business logic is executed according to the processing strategy to process the data to be processed. During the development of the program, the data source annotation is intercepted and checked in the first management container and the second management container respectively to determine the development method actually corresponding to the data source, so that different data source development methods are compatible in the same development process, which minimizes the waste of development resources caused by developing for different data sources, while reducing the complexity of program development and improving the developer's experience.

[0044] In an optional implementation, before obtaining the data to be processed and the target business logic, the method may include:

[0045] A1. Define the connection information of all data sources in the data source manager in advance.

[0046] Data source connection information may include information on how to connect the program development system to the data source so that the two can exchange data. Examples include, but are not limited to, IP addresses (Internet Protocol addresses), port numbers, and connection methods. Connection information for all data sources is predefined and stored in the data source manager.

[0047] A2. Initialize the first management container to generate a data source identifier corresponding to each connection information in the data resource manager through the first management container.

[0048] In the process of initializing the first management container, it is necessary to call the data resource manager, and thus a path for calling the data resource manager is established. In this process, data source identifiers corresponding to different data sources are generated.

[0049] For example, the spring container will be initialized when it starts. While implementing its own initialization strategy, it can call the data resource manager and establish a path to call the data resource manager to facilitate subsequent program development and generate data source identifiers corresponding to different data sources.

[0050] A3. Generate a data source annotation based on the data source identifier and connection information, and store the data source annotation in the data resource manager.

[0051] The data source identifier and the data source connection information are annotated to obtain a data source annotation. The data source annotation can directly distinguish different data sources and can also be parsed to obtain the data source connection information. The generated data source annotation is stored in the data resource manager so that the first management container and the second management container can directly access it.

[0052] In the above implementation, an association is established between the data source identifier and the data source connection information, and a data source label is generated, effectively binding the data source identifier and the data source connection information together. This helps to further parse the data source connection information by obtaining the data source label during the development phase, helping developers to quickly determine the data source situation and develop in a corresponding manner, which can improve the efficiency of program development.

[0053] In an optional implementation, the step of storing the target identifier in a preset data source manager in S120 includes: storing the target identifier in a thread-level container in the data source manager;

[0054] The method further comprises:

[0055] B1. In response to at least one currently occupied current thread in the data source manager, each current thread is bound to a data source identifier of a data source being called through the thread to generate a thread scheduling chain.

[0056] Among them, the thread scheduling chain can be the association relationship between the thread and the processing task (hereinafter referred to as the task). It is understandable that different threads can be used to process different tasks and call data from different data sources. When storing the target identifier in the thread-level container of the data source manager, it is necessary to detect whether the data source identifier already exists in the container, that is, whether a thread is being used. If the target identifier exists, it means that in the current thread call, there are multiple business service calls and multiple data sources will be used. Therefore, the current thread and the data source identifier of the data source being used through the thread are bound to form a thread scheduling chain, thereby representing different business tasks executed by different threads.

[0057] B2. Generate parent-child thread relationships for each thread scheduling chain according to the business methods currently being executed by each thread.

[0058] Among them, the parent-child thread relationship can be a subordinate relationship between different thread scheduling chains. It can be understood that according to the different levels of calling business methods in program development, the business tasks executed by different threads have a subordinate relationship between superiors and subordinates.

[0059] Therefore, according to the subordinate relationship of the business methods being executed by each current thread, the parent-child thread relationship between the thread scheduling chains is generated, that is, the child threads are associated under the parent thread, which can also be understood as the relationship between the root node and the leaf node.

[0060] B3. Regulate all threads in the data source manager based on the parent-child thread relationship.

[0061] The relationship between different associated threads can be determined according to the parent-child thread relationship. Therefore, when a problem occurs in any thread, other threads can be promptly associated and adjusted accordingly to ensure normal service calls or prevent larger problems.

[0062] In a further optional implementation, the regulation of all threads in the data source manager based on the parent-child thread relationship as described in B3 may include: responding to an execution status error of any child thread, broadcasting the error step by step along the parent-child thread relationship, and terminating the business execution of all thread scheduling chains in the parent-child thread relationship.

[0063] When any child thread has a problem with its execution status and reports an error, the error is broadcast from the child to the parent level by level according to the parent-child thread relationship, so that all related thread scheduling chains can obtain the error status of the child thread and terminate the business execution of all thread scheduling chains in the parent-child thread relationship, further preventing more errors from occurring.

[0064] In the above two implementations, the influence relationship between threads is associated according to the business tasks being executed by each thread, so that during the program development process, when a problem occurs in a thread call, the spread of errors can be prevented in a timely manner, which helps to ensure the stability of program development and also reduces the waste of development resources to a certain extent.

[0065] Example 2

[0066] Figure 2 This is a flow chart of a data processing method provided in the second embodiment of the present application. The present embodiment further refines the operation of processing the data to be processed according to the judgment result in S140 based on the above embodiment. Figure 2 As shown, the method includes:

[0067] S210: Obtain data to be processed and target business logic.

[0068] S220. Before the preset first management container calls the target business logic, obtain the data source label corresponding to the data to be processed in the preset data resource manager, determine the target identifier of the data source connection information corresponding to the data source label, and store the target identifier in the preset data source manager.

[0069] S230: Before the preset second management container executes the target business logic, obtain the target identifier in the data source manager and determine whether the target identifier is valid.

[0070] S240 : In response to the target identifier being valid, determine target connection information of a target data source and a target data source type of the target data source of the data to be processed according to the target identifier.

[0071] The target data source type may be the type of the data source corresponding to the target identifier, including, but not limited to, a structured database, an unstructured database, an in-memory database, a text database, and a network interface. If the target identifier exists and has a corresponding data source, the target data source represented by the identifier is identified based on the target identifier, and the type of the target data source is simultaneously detected and obtained.

[0072] S250: Create a target executor corresponding to the target data source type.

[0073] The target executor can develop an abstract executor for program-related processing of the data to be processed. This abstract executor can recommend abstract methods for adding, deleting, modifying, and querying data based on the characteristics of structured query statements. Based on the target data source type, a corresponding abstract executor is established, which uses the corresponding abstract method to perform corresponding operations such as adding, deleting, modifying, and querying on the data to be processed in the target data source.

[0074] S260: Call out the database query statement corresponding to the target business logic in the pre-set database mapping agent, and put the database query statement into the target executor to process the data to be processed.

[0075] Among them, the database mapping agent can be a component used for data conversion or mapping. Its core purpose is to build a bridge between different data models or systems to achieve data adaptation and conversion. For example, this embodiment can use the mapper class in mybatis as a database mapping agent to map database tables to objects in the program, thereby achieving decoupling of database operations from code.

[0076] The database query statements (e.g., SQL statements) corresponding to the target business logic are pre-set in the database mapping proxy. Therefore, the database query statements corresponding to the target logic are retrieved from the database mapping proxy and placed into the constructed target executor to process the pending data. It will be understood that the created target executor corresponds to the target data source type, and the database query statements corresponding to the target business logic are executed by the target executor in the business logic method corresponding to the target data source type.

[0077] In an optional embodiment, placing the database query statement into the target executor to process the data to be processed as described in S260 may include: determining the target operation corresponding to the database query statement in the operation parent class pre-set in the target executor; and processing the data to be processed according to the target operation.

[0078] The operation parent class can be used to encapsulate predetermined data operation logic, avoiding code duplication during program development and providing unified add, delete, modify, and query functionality for subclasses. The database query statement corresponds to the database operation to be executed (the target operation). By requesting the corresponding add, delete, modify, and query operation logic in the database query statement from the parent class, the corresponding add, delete, modify, and query operations are performed on the data to be processed.

[0079] In the technical solution of the embodiment of the present application, under the premise that the target identifier is valid, the target connection information and the target data source type of the target data source are parsed and obtained, and a corresponding executor is created accordingly, so that the data to be processed is executed through the executor with data operations corresponding to the target data source type. This enables developers to perform development operations on different data sources on the same platform during the program development phase, greatly improving development efficiency while also reducing the waste of development resources.

[0080] Based on the above embodiments and implementation methods, the present application also provides a practical example, which is as follows:

[0081] First, manage the data source configuration: define a data source manager and define the connection information of different data sources in the data source manager, such as structured databases, in-memory databases, text databases, network interfaces, unstructured databases, and other types of connection information. The data resource manager implements Spring's initialization call strategy. When Spring starts, it calls the data resource manager for initialization and calls the target data source information that needs to be connected to establish a connection. A connection container is formed in the memory to store the connection information. The connection container stores the data source identifier and the KV (Key-Value, label-value) corresponding to the data source connection. The KV value represents the connection information of the data source.

[0082] Secondly, define the program development: for different data sources, during application development, define the @DSC annotation (data source customization). When the Mapper class is constructed, the corresponding data source is marked on the Mapper method, for example: @DSC ("job-hbase"). Job-hbase is only used as an example here and has no exact meaning. "job-hbase" is the label identifier of the connection container. In all Mapper.xml configurations, regardless of the data, the corresponding SQL statement configuration is uniformly adopted. If the data source is a network interface, its interface name is used as the table name for SQL statement configuration.

[0083] Third, business data layer call: define two-layer interceptors, Spring container level and mybatis container level interceptors. The Spring container level interceptor mainly intercepts when the method is called, and intercepts when there is a @DSC annotation method. When the annotation exists, extract the current annotation, and obtain the data source identifier of the current annotation configuration, and establish a thread-level data container in the data source manager, and put the data source identifier into the container. At the same time, when putting it into the container, it is necessary to determine whether the container already has a data source identifier. If so, it means that there are multiple business service calls in the current thread call, and there will be many data sources used. At this time, in the data container, it is necessary to build a data source tag identification chain and a thread scheduling chain. First, the id of the current thread (Identity document, identity identification number), and bind the data source identifier of the current container to build the root node of the data source label identification chain. Secondly, a new thread is built, and the thread id and the new data source identifier are bound to it as the identification chain node of the new business data source label, which is inserted under the root node (equivalent to the thread scheduling chain formed in the previous article). The newly built thread is executed to call the corresponding business method to realize scheduling decoupling from the main business. The main thread continuously detects whether the thread exists. If it exists, it waits. If it does not exist, the main thread wakes up and executes. During the entire probing process, in addition to monitoring the thread status, it is also necessary to monitor the thread execution status. If an execution error occurs, the thread execution error must be captured and broadcast to the upper layer to terminate the execution of all thread chains.

[0084] The mybatis container-level interceptor defines the mybatis interceptor at the Executor level. In this interceptor, the data source identifier of the current thread is obtained from the data source manager. If the data identifier is empty, it indicates that the current data processing method is the default method. The interceptor object is called to trigger the execution of the mybatis target method and return the result. If there is a data source identifier in the data source manager, the data source connection is obtained in the connection container through the label identifier, and the connection target data source type of the connection is obtained through the data source connection. The data factory is called to create a dialect executor DEXE object of the target data source type, and the execution SQL statement of mybatis is obtained through boundSql, and the SQL statement is passed to the dialect executor DEXE for execution, and the direct result is returned.

[0085] The dialect executor DEXE is actually an executor specially created for each different data source and compatible with the development method of that data source. For the dialect executor, first define the abstract executor of the dialect executor. This abstract executor establishes abstract methods for adding, deleting, modifying and querying based on the characteristics of the SQL language DML (Data Manipulation Language). According to each connected data source and the abstract executor of the dialect executor, different types of dialect executors are implemented, mainly including common data source dialect executors, but you can also expand your own dialect executor. If you expand your own dialect executor, implement the abstract executor of the dialect executor to expand it. After the expansion is completed, the program automatically loads the custom extended dialect executor through SPI configuration. For common dialect executors, mainly including Redis dialect executor, interface dialect executor, MongoDB dialect executor, ES dialect executor and other dialect executors, the implementation logic of different dialect executors is as follows:

[0086] Added: For the added implementation, by parsing the SQL statement, the insert statement is parsed into the key-value pair ins_obj of kv, and different insert execution processes are dynamically generated according to the type of different data sources;

[0087] Redis dialect executor: For Redis, all data is uniformly stored as the map data type, the table name of the SQL statement is used as the Redis key, and ins_obj is used as the value for insertion.

[0088] Interface dialect executor: obtains the request address of the data source through the tag identifier, creates a post request object based on the request address, uses ins_obj as the parameter content of the body, and sends the post request for execution.

[0089] Distributed database dialect executors, such as the MongoDB dialect executor and the ES (Elastic Stack) dialect executor, are implemented in the same way. They obtain the ins_obj object, convert the SQL statement's table name into a MongoDB or ES insert statement, and execute it to complete data insertion.

[0090] Delete, modify, and query: These three DML operations parse SQL statements, first extracting the where conditions and limit parameters of the SQL statements as data filtering parameters, and then constructing execution objects in different dialects. For example, for a certain interface, the parameters are generated into different request objects for POST and GET based on the DML operation type, and the parameters are executed as body or URL request parameters. For types such as Redis, MongoDB, and ES, the corresponding execution statements are generated from the parameters. After all data is executed, if it is a query statement, the SQL statement returns the result, and the execution results of different executors are extracted by field extraction to generate the return object.

[0091] Example 3

[0092] Figure 3 This is a structural diagram of a data processing device provided in Example 3 of this application. Figure 3 As shown, the device 300 includes:

[0093] Data acquisition module 310, used to obtain data to be processed and target business logic;

[0094] The identifier determination module 320 is configured to obtain the data source annotation corresponding to the to-be-processed data from the preset data resource manager before the preset first management container calls the target business logic, determine the target identifier of the data source connection information corresponding to the data source annotation, and store the target identifier in the preset data source manager;

[0095] The identification judgment module 330 is used to obtain the target identification in the data source manager and judge whether the target identification is valid before the preset second management container executes the target business logic;

[0096] The business processing module 340 is used to determine a processing strategy for the data to be processed based on the judgment result, and execute the target business logic to process the data to be processed according to the processing strategy.

[0097] In the technical solution of the embodiment of the present application, before the preset first management container calls the target business logic, the data source annotation corresponding to the data to be processed is obtained in the preset data resource manager, and the target identifier of the data source connection information corresponding to the data source annotation is determined, and the target identifier is stored in the preset data source manager; before the preset second management container executes the target business logic, the target identifier in the data source manager is obtained, and it is determined whether the target identifier is valid; based on the judgment result, the processing strategy of the data to be processed is determined, and the target business logic is executed according to the processing strategy to process the data to be processed. During the development of the program, the data source annotation is intercepted and checked in the first management container and the second management container respectively to determine the development method actually corresponding to the data source, so that different data source development methods are compatible in the same development process, which minimizes the waste of development resources caused by developing for different data sources, while reducing the complexity of program development and improving the developer's experience.

[0098] In an optional implementation, the service processing module 340 may include:

[0099] a connection information determining unit, configured to determine, in response to the target identifier being valid, target connection information of a target data source of the data to be processed and a target data source type of the target data source according to the target identifier;

[0100] An executor creation unit, used to create a target executor corresponding to the target data source type;

[0101] The business logic execution unit is used to call out the database query statement corresponding to the target business logic in the pre-set database mapping agent, and put the database query statement into the target executor to process the data to be processed.

[0102] In an optional implementation, the business logic execution unit may include:

[0103] The target operation determination subunit is used to determine the target operation corresponding to the database query statement in the operation parent class pre-set in the target executor;

[0104] The data processing subunit is used to process the data to be processed according to the target operation.

[0105] In an optional embodiment, the apparatus 300 includes:

[0106] A connection information definition module is used to pre-define the connection information of all data sources in the data source manager;

[0107] A data source identifier generation module, configured to initialize the first management container so as to generate a data source identifier corresponding to each connection information in the data resource manager through the first management container;

[0108] The data source annotation storage module is used to generate a data source annotation based on the data source identifier and connection information, and store the data source annotation in the data resource manager.

[0109] In an optional implementation, the identifier determination module 320 may be specifically configured to: store the target identifier in a thread-level container in the data source manager;

[0110] The apparatus 300 further includes:

[0111] a scheduling chain determining module, configured to, in response to at least one currently occupied current thread in the data source manager, bind each current thread to a data source identifier of a data source being called through the thread, and generate a thread scheduling chain;

[0112] The parent-child relationship determination module is used to generate parent-child thread relationships for each thread scheduling chain according to the business method being executed by each current thread;

[0113] The thread control module is used to control all threads in the data source manager based on the parent-child thread relationship.

[0114] In an optional embodiment, the thread control module can be specifically used to: in response to an execution status error of any child thread, broadcast the error step by step along the parent-child thread relationship, and terminate the business execution of all thread chains in the parent-child thread relationship.

[0115] The data processing device provided in the embodiments of the present application can execute the data processing method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

[0116] Example 4

[0117] Figure 4 A schematic diagram of an electronic device 10 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0118] like Figure 4 As shown, electronic device 10 includes at least one processor 11 and memory, such as read-only memory (ROM) 12 and random access memory (RAM) 13, communicatively connected to at least one processor 11. The memory stores computer programs executable by the at least one processor. Processor 11 can perform various appropriate actions and processes based on the computer programs stored in ROM 12 or loaded from storage unit 18 into RAM 13. RAM 13 can also store various programs and data required for the operation of electronic device 10. Processor 11, ROM 12, and RAM 13 are interconnected via bus 14. An input / output (I / O) interface 15 is also connected to bus 14.

[0119] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0120] Processor 11 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. Processor 11 executes the various methods and processes described above, such as the data processing method.

[0121] In some embodiments, the data processing method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the data processing method in any other suitable manner (e.g., via firmware).

[0122] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0123] Computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0124] In the context of the present application, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or apparatus. A computer-readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0125] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device that has: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0126] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0127] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0128] The present application also discloses a computer program product, comprising a computer program that, when executed by a processor, implements the data processing method provided in any of the embodiments of the present application. This program product and the data processing method disclosed in each embodiment of the present application share the same inventive concept and are therefore not described in detail here.

[0129] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of this application can be achieved. This is not limited herein.

[0130] The foregoing detailed description has set forth various embodiments of the devices and / or processes via the use of specific terminology. However, embodiments thereof can be practiced with the exact description not being set forth but with the same essence; the principles set forth herein can be practiced to achieve the present application. The scope of protection is not limited to the embodiments described and shown.

Claims

1. A data processing method, characterized in that: include: Obtain the data to be processed and the target business logic; Before the preset first management container calls the target business logic, the interceptor corresponding to the first management container obtains the data source label corresponding to the to-be-processed data in the preset data resource manager, determines the target identifier of the data source connection information corresponding to the data source label, stores the target identifier in the preset data source manager, checks whether the data source identifier already exists in the data source manager, and generates a thread scheduling chain for all identifiers and corresponding data sources in response to the existence of the data source identifier; wherein the data source connection information of the corresponding data source is recorded in the data source identifier; Before the preset second management container executes the target business logic, obtaining the target identifier in the data source manager through the interceptor corresponding to the second management container, and determining whether the target identifier is valid; According to the judgment result, a processing strategy for the data to be processed is determined, and the target business logic is executed according to the processing strategy to process the data to be processed.

2. The method according to claim 1, characterized in that Determining a processing strategy for the data to be processed based on the judgment result, and executing the target business logic to process the data to be processed according to the processing strategy, includes: In response to the target identifier being valid, determining target connection information of a target data source of the data to be processed and a target data source type of the target data source according to the target identifier; Creating a target executor corresponding to the target data source type; The database query statement corresponding to the target business logic is called out in a pre-set database mapping agent, and the database query statement is placed in the target executor to process the data to be processed.

3. The method according to claim 2, characterized in that The step of placing the database query statement into the target executor to process the data to be processed includes: Determine the target operation corresponding to the database query statement in the operation parent class pre-set in the target executor; The data to be processed is processed according to the target operation.

4. The method according to claim 1, wherein Before obtaining the data to be processed and the target business logic, the method includes: Predefine connection information of all data sources in the data source manager; Initializing the first management container to generate a data source identifier corresponding to each piece of connection information in the data resource manager through the first management container; The data source annotation is generated according to the data source identifier and the connection information, and the data source annotation is stored in the data resource manager.

5. The method according to claim 1, wherein Storing the target identifier in a preset data source manager includes: Storing the target identifier in a thread-level container in the data source manager; The method further comprises: In response to at least one currently occupied current thread existing in the data source manager, each of the current threads is bound to a data source identifier of a data source being called through the thread, to generate a thread scheduling chain; Generate parent-child thread relationships for each thread scheduling chain according to the business method being executed by each current thread; All threads in the data source manager are regulated according to the parent-child thread relationship.

6. The method according to claim 5, characterized in that The regulating all threads in the data source manager according to the parent-child thread relationship includes: In response to an execution status error of any child thread, the error is broadcast step by step along the parent-child thread relationship, and the business execution of all thread chains in the parent-child thread relationship is terminated.

7. A data processing device, characterized in that: include: Data acquisition module, used to obtain data to be processed and target business logic; an identifier determination module, configured to obtain, before a preset first management container calls a target business logic, a data source label corresponding to the to-be-processed data in a preset data resource manager through an interceptor corresponding to the first management container, determine a target identifier corresponding to the data source connection information of the data source label, store the target identifier in a preset data source manager, verify whether the data source label already exists in the data source manager, and, in response to the existence of the data source label, generate a thread scheduling chain for all identifiers and corresponding data sources; wherein the data source label records the data source connection information of the corresponding data source; An identifier judgment module is used to obtain the target identifier in the data source manager through the interceptor corresponding to the second management container before the preset second management container executes the target business logic, and to judge whether the target identifier is valid; The business processing module is used to determine a processing strategy for the data to be processed according to the judgment result, and execute the target business logic to process the data to be processed according to the processing strategy.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the data processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data processing method according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the data processing method according to any one of claims 1 to 6.

Images