A method and device for generating a relationship diagram

Abstract

The present invention provides a relationship graph generation method and device, the method comprising: creating multiple first target objects and multiple second target objects due to deep learning, the multiple first target objects and the multiple second target objects are all used to generate data and / or process data; controlling the multiple first target objects to make reservations with the multiple second target objects, one first target object makes reservations with one or more second target objects, and multiple first target objects make reservations with the same second target object; generating a relationship graph between the multiple first target objects and the multiple second target objects between operators of each layer in the deep learning according to the reservation results, which can solve the problem in the related technology that the relationship between producers and consumers cannot be counted before running, and by controlling the producer to make reservations with the consumer before running, a relationship graph between producers and consumers is generated, thereby realizing the statistics of the relationship between producers and consumers before running.

Description

Technical Field

[0001] The present invention relates to the technical field of software engineering, and in particular to a method and device for generating a relationship graph. Background Art

[0002] Design patterns refer to the summary of code design experience that has been repeatedly used and known by most people, and has been catalogued and classified. Among them, the producer-consumer pattern is a commonly used programming design pattern. The concept is as follows: a module is responsible for producing data, and a module is responsible for processing the produced data. The module that produces data is called a producer, and the module that processes the data is called a consumer.

[0003] Suppose there is a producer B at present. Only when a consumer A takes data from where producer B stores data can the current consumer-producer relationship be finally determined (now we know that A is the consumer and B is the producer). That is, the corresponding relationship can only be known at runtime. If you want to count the total consumer-producer relationship before consumer A takes data, it cannot be counted. Many software applications need to display the corresponding relationships described by all software data flows on the WEB or other places at the beginning (for example, tensorflow can display the current neural network data flow diagram on tensorboard before running the forward calculation of the neural network).

[0004] With respect to the problem in related technologies that the relationship between producers and consumers cannot be counted before operation, no solution has been proposed yet. Summary of the invention

[0005] The embodiment of the present invention provides a relationship graph generation method and device to at least solve the problem in the related art that the relationship between producers and consumers cannot be counted before operation.

[0006] According to an embodiment of the present invention, a method for generating a relationship graph is provided, characterized by comprising:

[0007] Creating a plurality of first target objects and a plurality of second target objects for deep learning, wherein the plurality of first target objects and the plurality of second target objects are both used to generate and / or process data;

[0008] Controlling the multiple first target objects to make reservations with the multiple second target objects, wherein one first target object makes reservations with one or more second target objects, and / or multiple first target objects make reservations with the same second target object;

[0009] A relationship graph between the multiple first target objects and the multiple second target objects between operators in each layer in the deep learning is generated according to the reservation result.

[0010] Optionally, after generating a relationship graph between the plurality of first target objects and the plurality of second target objects between operators in each layer in the deep learning according to the reservation result, the method further includes:

[0011] Controlling the plurality of first target objects to generate data;

[0012] The plurality of second target objects are controlled to obtain the data generated by the plurality of first target objects, and process the data.

[0013] Optionally, controlling the plurality of second target objects to obtain data generated by the plurality of first target objects includes:

[0014] The multiple second target objects are controlled to obtain the data generated by the first target object corresponding to the identifier of the first target object according to the identifier of the first target object carried in the received reservation request.

[0015] Optionally, controlling the first target object to make an appointment with the second target object includes:

[0016] The reservation is made by controlling the first target object to send a reservation request to the second target object, wherein the reservation request carries at least one of the following: an identifier of the first target object, a generated data format, generated data content, memory size, and a running node.

[0017] Optionally, generating a relationship graph between the first target object and the second target object between operators at each layer in the deep learning according to the reservation result includes:

[0018] Establishing a correspondence between the plurality of first target objects and the second target objects scheduled by the plurality of first target objects;

[0019] The relationship graph is generated according to the corresponding relationship.

[0020] According to another embodiment of the present invention, there is also provided a relationship diagram generating device, comprising:

[0021] A creation module, used to create a plurality of first target objects and a plurality of second target objects for deep learning, wherein the plurality of first target objects and the plurality of second target objects are used to generate and / or process data;

[0022] A reservation module, used for controlling the multiple first target objects to make reservations with the multiple second target objects, wherein one first target object makes reservations with one or more second target objects, and / or multiple first target objects make reservations with the same second target object;

[0023] A generation module is used to generate a relationship diagram between the multiple first target objects and the multiple second target objects between each layer of operators in the deep learning according to the reservation result.

[0024] Optionally, the device further comprises:

[0025] A first control module, used for controlling the plurality of first target objects to generate data;

[0026] The second control module is used to control the multiple second target objects to obtain the data generated by the multiple first target objects and process the data.

[0027] Optionally, the second control module includes:

[0028] The first control unit is used to control the multiple second target objects to obtain the data generated by the first target object corresponding to the identifier of the first target object according to the identifier of the first target object carried in the received reservation request.

[0029] Optionally, the reservation module includes:

[0030] A reservation unit, used to make a reservation by controlling the first target object to send a reservation request to the second target object, wherein the reservation request carries at least one of the following: an identifier of the first target object, a generated data format, generated data content, a memory size, and a running node.

[0031] Optionally, the generating module includes:

[0032] An establishing unit, configured to establish a corresponding relationship between the plurality of first target objects and the second target objects reserved by the plurality of first target objects;

[0033] A generating unit is used to generate the relationship graph according to the corresponding relationship.

[0034] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.

[0035] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0036] Through the present invention, multiple first target objects and multiple second target objects are created for deep learning, wherein the multiple first target objects and the multiple second target objects are used to generate and / or process data; the multiple first target objects are controlled to make reservations with the multiple second target objects, wherein one first target object makes reservations with one or more second target objects, and / or multiple first target objects make reservations with the same second target object; and a relationship graph between the multiple first target objects and the multiple second target objects between operators of each layer in the deep learning is generated according to the reservation results, which can solve the problem in the related art that the relationship between producers and consumers cannot be counted before running. By controlling the producer (i.e., the first target object) to make reservations with the consumer (i.e., the second target object) before running, a relationship graph between producers and consumers is generated, thereby realizing the statistics of the relationship between producers and consumers before running. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0038] Figure 1 is a hardware structure block diagram of a mobile terminal of a method for generating a relationship graph according to an embodiment of the present invention;

[0039] Figure 2 is a flowchart of a method for generating a relationship graph according to an embodiment of the present invention;

[0040] Figure 3 is a flowchart of a node creation phase according to an embodiment of the present invention;

[0041] Figure 4 is a flowchart of a node creation phase according to an embodiment of the present invention;

[0042] Figure 5 is a flowchart of a node creation phase of adding a reservation mode according to an embodiment of the present invention;

[0043] Figure 6 is a block diagram of a relationship diagram generating device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0045] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0046] Example 1

[0047] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 is a hardware structure block diagram of a mobile terminal of a method for generating a relationship diagram according to an embodiment of the present invention, such as Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0048] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the relationship diagram generation method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0049] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0050] In this embodiment, a method for generating a relationship graph running on the above mobile terminal or network architecture is provided. Figure 2 is a flow chart of a method for generating a relationship graph according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0051] Step S202, creating a plurality of first target objects and a plurality of second target objects for deep learning, wherein the plurality of first target objects and the plurality of second target objects are both used to generate and / or process data;

[0052] Step S204, controlling the multiple first target objects to make reservations with the multiple second target objects, wherein one first target object makes reservations with one or more second target objects, and / or multiple first target objects make reservations with the same second target object;

[0053] That is, in this embodiment, one producer (ie, the first target object) can make reservations for multiple consumers (ie, the second target object), and multiple producers can also make reservations for the same consumer.

[0054] Furthermore, the reservation is made by controlling the first target object to send a reservation request to the second target object, wherein the reservation request carries at least one of the following: the identifier of the first target object, the generated data format, the generated data content, the memory size, and the running node.

[0055] Step S206, generating a relationship diagram between the multiple first target objects and the multiple second target objects between operators in each layer in the deep learning according to the reservation result.

[0056] Furthermore, a correspondence is established between the plurality of first target objects and the second target objects reserved by the plurality of first target objects, and the relationship graph is generated according to the correspondence.

[0057] Through the above steps S202 to S206, multiple first target objects and multiple second target objects are created for deep learning, and the multiple first target objects are controlled to make reservations with the multiple second target objects, wherein one first target object makes reservations with one or more second target objects, and / or multiple first target objects make reservations with the same second target object; a relationship graph between the multiple first target objects and the multiple second target objects between operators at each layer in the deep learning is generated according to the reservation results, which can solve the problem in the related technology that the relationship between producers and consumers cannot be counted before running. By controlling the producer to make reservations with the consumer before running, a relationship graph between producers and consumers is generated, thereby realizing the statistics of the relationship between producers and consumers before running.

[0058] In an embodiment of the present invention, after generating a relationship diagram between the multiple first target objects and the multiple second target objects between operators at each layer in the deep learning according to the reservation result, the multiple first target objects are controlled to generate data; the multiple second target objects are controlled to obtain the data generated by the multiple first target objects and process the data. Furthermore, the multiple second target objects are controlled to obtain the data generated by the first target object corresponding to the identifier of the first target object according to the identifier of the first target object carried in the received reservation request.

[0059] In the embodiment of the present invention, multiple producers (corresponding to the above-mentioned first target object) and multiple consumers (corresponding to the above-mentioned second target object) are initially created. Some consumers choose to register with the corresponding producers. At this time, a relationship diagram between the registered consumers and the registered producers is generated for interface presentation. Next, multiple producers generate data, and the consumers who have made reservations before fetch data from the corresponding producers for processing (verify whether it is the same producer as the one currently obtained), and those who have not made reservations still fetch data from the corresponding producers in the old way for processing.

[0060] The following uses a case of target detection as an example to illustrate the embodiment of the present invention.

[0061] The input image is preprocessed and then sent to deep learning for calculation. After post-processing, the target detection box is output.

[0062] Preprocessing node: enlarges or reduces the image sent in and converts the image format, and then performs some preprocessing required for deep learning, such as mean subtraction and normalization, to meet the input requirements of deep learning operations.

[0063] Deep learning operation node: performs deep learning operations on the preprocessed image and then outputs the results to memory.

[0064] Detection post-processing node: Post-process the memory results output after the deep learning operation and calculate the target detection box.

[0065] All the following descriptions of producers and consumers in this case use memory as an example. For example, if A creates this memory, A is the producer, and B uses someone else's memory, B is the consumer. When the producer creates memory, it will store the memory information in the corresponding location. When the consumer is running, it will retrieve the memory information from the corresponding location. If B fails to retrieve the information, at this time, the producer may not have been created yet.

[0066] Figure 3 is a flowchart of the node creation phase according to an embodiment of the present invention, such as Figure 3 As shown, including:

[0067] Image Node:

[0068] Step S301, out: no image input in the creation phase, no producer yet;

[0069] Preprocessing nodes:

[0070] Step S302, in: the image is passed in from the outside of the runtime, and the input memory is the consumer;

[0071] Step S303, out: output memory resources to the input memory of the deep learning node, and the output is also a consumer;

[0072] Deep Learning Node:

[0073] Step S304, in: the input memory is created by the deep learning node itself, so it is the generator;

[0074] Step S305, out: the output memory is created by the deep learning network itself, so the output is also the producer;

[0075] Post-detection processing nodes:

[0076] Step S306, in: the input memory comes from the output memory of the deep learning node, so it is a consumer;

[0077] Step S307, out: the output memory is created by the post-detection processing node itself, so the output is the producer;

[0078] At this time, the in and out of the preprocessing node cannot count the corresponding memory information, because there is no corresponding producer when it is created. After the detection, the consumer in the post-processing node can directly obtain the memory information from the deep learning operation node producer because the previous node has been created.

[0079] Figure 4 is a flowchart of the node creation phase according to an embodiment of the present invention, such as Figure 4 As shown, including:

[0080] Image Node:

[0081] Step S401, out: running stage, image input, no producer yet;

[0082] Preprocessing nodes:

[0083] Step S402, in: as a consumer, obtains memory information from the producer of the image input node;

[0084] Step S403, out: obtain the producer from the deep learning node as a consumer (because the producer is created in the above step S304);

[0085] Deep Learning Node:

[0086] Step S404, in: the input memory is created by the deep learning node itself, so it is the generator;

[0087] Step S405, out: the output memory is created by the deep learning network itself, so the output is also the producer;

[0088] Post-detection processing nodes:

[0089] Step S406, in: obtaining memory information from the deep learning node as a consumer;

[0090] Step S407, out: the output memory is created by the post-detection processing node itself, so the output is the producer;

[0091] At this point, the memory statistics of each node can be counted, but the entire process needs to be run once, which may take too long.

[0092] Figure 5 is a flowchart of the node creation phase of adding a reservation mode according to an embodiment of the present invention, such as Figure 5 As shown, including:

[0093] Image Node:

[0094] Step S501, out: There is no image input in the creation phase, there is no producer yet, and an appointment is made in advance to inform the memory information of the image input;

[0095] Preprocessing nodes:

[0096] Step S502, in: the image is passed in from the outside during runtime, and the input memory is the consumer (at this time, there is no image input and no producer);

[0097] Step S503, out: output memory resources to the input memory of the deep learning node, and the output is also a consumer (there is no image input and no producer at this time);

[0098] Deep Learning Node:

[0099] Step S504, in: the input memory originates from the deep learning node itself, so it is the generator. At this time, the consumer is notified of the memory information by appointment;

[0100] Step S505, out: the output memory is created by the deep learning network itself, so the output is also the producer;

[0101] Post-detection processing nodes:

[0102] Step S506, in: the input memory comes from the output memory of the deep learning node, so it is a consumer;

[0103] Step S507, out: the output memory is created by the post-detection processing node itself, so the output is the producer (at this time, the memory information can be obtained from the producer of the out part of the deep learning);

[0104] After adding the reservation mode, the memory status information of all nodes can be obtained only during the creation phase, which is convenient for external presentation, whether the information is displayed directly on the client or on the web page.

[0105] Example 2

[0106] According to another embodiment of the present invention, a relationship diagram generating device is provided. Figure 6 is a block diagram of a relationship diagram generating device according to an embodiment of the present invention. Figure 6 As shown, including:

[0107] A creation module 62, configured to create a plurality of first target objects and a plurality of second target objects for deep learning, wherein the plurality of first target objects and the plurality of second target objects are both used to generate data and / or process data;

[0108] The reservation module 64 is used to control the multiple first target objects to make reservations with the multiple second target objects, wherein one first target object makes reservations with one or more second target objects, and / or multiple first target objects make reservations with the same second target object;

[0109] The generation module 66 is used to generate a relationship diagram between the multiple first target objects and the multiple second target objects between each layer of operators in the deep learning according to the reservation result.

[0110] Optionally, the device further comprises:

[0111] A first control module, used for controlling the plurality of first target objects to generate data;

[0112] The second control module is used to control the multiple second target objects to obtain the data generated by the multiple first target objects and process the data.

[0113] Optionally, the second control module includes:

[0114] The first control unit is used to control the multiple second target objects to obtain the data generated by the first target object corresponding to the identifier of the first target object according to the identifier of the first target object carried in the received reservation request.

[0115] Optionally, the reservation module 64 includes:

[0116] A reservation unit, used to make a reservation by controlling the first target object to send a reservation request to the second target object, wherein the reservation request carries at least one of the following: an identifier of the first target object, a generated data format, generated data content, a memory size, and a running node.

[0117] Optionally, the generating module 66 includes:

[0118] An establishing unit, configured to establish a corresponding relationship between the plurality of first target objects and the second target objects reserved by the plurality of first target objects;

[0119] A generating unit is used to generate the relationship graph according to the corresponding relationship.

[0120] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0121] Example 3

[0122] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0123] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0124] S1, creating a plurality of first target objects and a plurality of second target objects for deep learning, wherein the plurality of first target objects and the plurality of second target objects are both used to generate data and / or process data;

[0125] S2, controlling the multiple first target objects to make reservations with the multiple second target objects, wherein one first target object makes reservations with one or more second target objects, and / or multiple first target objects make reservations with the same second target object;

[0126] S3, generating a relationship diagram between the multiple first target objects and the multiple second target objects between operators in each layer in the deep learning according to the reservation result.

[0127] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0128] Example 4

[0129] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0130] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0131] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0132] S1, creating a plurality of first target objects and a plurality of second target objects for deep learning, wherein the plurality of first target objects and the plurality of second target objects are both used to generate data and / or process data;

[0133] S2, controlling the multiple first target objects to make reservations with the multiple second target objects, wherein one first target object makes reservations with one or more second target objects, and / or multiple first target objects make reservations with the same second target object;

[0134] S3, generating a relationship diagram between the multiple first target objects and the multiple second target objects between operators in each layer in the deep learning according to the reservation result.

[0135] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0136] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0137] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A relationship graph generation method, applied to a server, characterized in that: include: Creating a plurality of first target objects and a plurality of second target objects for deep learning, wherein the plurality of first target objects and the plurality of second target objects are both used to generate data and / or process data; Controlling the multiple first target objects to make reservations with the multiple second target objects before running, comprising: making reservations by controlling the first target objects to send reservation requests to the second target objects, wherein the reservation request carries at least one of the following: an identifier of the first target object, a generated data format, generated data content, a memory size, and a running node, wherein one first target object makes reservations with one or more second target objects, and / or multiple first target objects make reservations with the same second target object; A relationship graph between the multiple first target objects and the multiple second target objects between operators in each layer in the deep learning is generated according to the reservation result.

2. The method according to claim 1, characterized in that After generating a relationship graph between the plurality of first target objects and the plurality of second target objects between operators in each layer in the deep learning according to the reservation result, the method further includes: Controlling the plurality of first target objects to generate data; The plurality of second target objects are controlled to obtain the data generated by the plurality of first target objects, and process the data.

3. The method according to claim 2, characterized in that Controlling the plurality of second target objects to obtain the data generated by the plurality of first target objects includes: The multiple second target objects are controlled to obtain the data generated by the first target object corresponding to the identifier of the first target object according to the identifier of the first target object carried in the received reservation request.

4. The method according to claim 1, characterized in that: Generating a relationship diagram between the first target object and the second target object between operators at each layer in the deep learning according to the reservation result includes: Establishing a correspondence between the plurality of first target objects and the second target objects scheduled by the plurality of first target objects; The relationship graph is generated according to the corresponding relationship.

5. A relationship diagram generating device, characterized in that: include: A creation module, used to create a plurality of first target objects and a plurality of second target objects for deep learning, wherein the plurality of first target objects and the plurality of second target objects are used to generate and / or process data; A reservation module, used for controlling the multiple first target objects to make reservations with the multiple second target objects before running, comprising: making reservations by controlling the first target objects to send reservation requests to the second target objects, wherein the reservation request carries at least one of the following: an identifier of the first target object, a generated data format, generated data content, a memory size, and a running node, wherein one first target object makes reservations with one or more second target objects, and / or multiple first target objects make reservations with the same second target object; A generation module is used to generate a relationship diagram between the multiple first target objects and the multiple second target objects between each layer of operators in the deep learning according to the reservation result.

6. The device according to claim 5, characterized in that The device also includes: A first control module, used for controlling the plurality of first target objects to generate data; The second control module is used to control the multiple second target objects to obtain the data generated by the multiple first target objects and process the data.

7. The device according to claim 6, characterized in that The second control module includes: The first control unit is used to control the multiple second target objects to obtain the data generated by the first target object corresponding to the identifier of the first target object according to the identifier of the first target object carried in the received reservation request.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 4 when executed.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 4.

Images