Visual processing equipment distributed management method, system and device, equipment and medium

Abstract

The embodiment of the invention provides a distributed management method, system, device and equipment for visual processing equipment and a medium, and relates to the technical field of machine vision, the method is applied to an integrated end, and the method comprises the following steps: responding to a first operation, and connecting with the visual processing equipment; in response to the second operation, a visual sub-scheme of the visual processing device is managed, the visual processing device executing the visual sub-scheme. According to the scheme, development resource and time investment can be reduced, the flexibility and expansibility of a development mode are improved, and the problems of hardware resource preemption and visual system performance fluctuation among multiple machine vision services are solved.

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to a distributed management method, system, device, equipment and medium for visual processing equipment. Background Art

[0002] In order to realize multi-device collaborative machine vision applications, multiple vision processing devices need to be integrated into the vision system.

[0003] At present, there are many manufacturers and types of visual processing devices. In order to integrate a variety of visual processing devices into the visual system and jointly realize corresponding machine vision applications, R&D personnel need to integrate the SDKs of multiple visual processing devices on the integration side. The integration side calls the SDKs of multiple visual processing devices to execute multiple machine vision services.

[0004] This management method of visual processing equipment requires a large investment of resources and time for visual system development, has poor development flexibility, and has poor scalability. In addition, multiple machine vision services need to be run in the visual system, and hardware resource preemption is likely to occur when multiple machine vision services are executed, which in turn causes fluctuations in visual system performance. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a method, system, device, equipment and medium for distributed management of visual processing equipment, so as to reduce development resources and time investment, improve the flexibility and scalability of development methods, and solve the problems of hardware resource preemption and visual system performance fluctuation among multiple machine vision services. The specific technical solution is as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for distributed management of a visual processing device, which is applied to an integration terminal, and the method includes:

[0007] In response to a first operation, connecting with a visual processing device;

[0008] In response to a second operation, a visual sub-scheme of the visual processing device is managed, and the visual processing device executes the visual sub-scheme.

[0009] In some embodiments, the integrated end is connected to a plurality of visual processing devices;

[0010] The step of managing the visual sub-scheme of the visual processing device comprises:

[0011] configuring a vision sub-schema for at least one vision processing device;

[0012] The visual sub-scheme is sent to the visual processing device so that the visual processing device executes the visual sub-scheme.

[0013] In some embodiments, the step of managing the visual sub-schema of the visual processing device includes:

[0014] Add a visual sub-scheme of the visual processing device to the comprehensive scheme.

[0015] In some embodiments, the integrated end is connected to a plurality of visual processing devices;

[0016] The step of adding the visual sub-scheme of the visual processing device to the comprehensive scheme comprises:

[0017] Add a vision sub-schema for at least one vision processing device to the comprehensive scheme.

[0018] In some embodiments, the step of managing the visual sub-schema of the visual processing device includes:

[0019] Delete the vision sub-schema of the vision processing device.

[0020] In some embodiments, the method further comprises:

[0021] A deletion instruction is sent to the visual processing device so that the visual processing device deletes the visual sub-scheme.

[0022] In some embodiments, the step of managing the visual sub-schema of the visual processing device includes:

[0023] modifying a visual sub-schema of the visual processing device;

[0024] The modified visual sub-scheme is sent to the visual processing device so that the visual processing device executes the modified visual sub-scheme.

[0025] In some embodiments, the step of managing the visual sub-schema of the visual processing device includes:

[0026] Obtain the visual processing result obtained by the visual processing device executing the visual sub-scheme.

[0027] In some embodiments, the step of managing the visual sub-schema of the visual processing device includes:

[0028] The visual sub-scheme of the visual processing device is saved, and / or the comprehensive scheme is saved, wherein the comprehensive scheme includes the visual sub-scheme.

[0029] In some embodiments, the method further comprises:

[0030] Creating a first class object of a vision sub-schema of the vision processing device;

[0031] The step of managing the visual sub-scheme of the visual processing device comprises:

[0032] The visual sub-scheme is managed using the attribute information of the first type of objects.

[0033] In some embodiments, the method further comprises:

[0034] Creating a management identifier of the visual processing device bound to the first type of object;

[0035] Use the management identifier to connect to the communication service running in the visual processing device, and use the management identifier to connect to the file service running in the visual processing device.

[0036] In some embodiments, the step of using the management identifier to connect to a communication service running in the visual processing device includes:

[0037] Creating a communication identifier for the visual processing device according to the device information of the visual processing device;

[0038] Using the communication identifier, establishing a message sending channel with the visual processing device, and binding the communication identifier with the management identifier;

[0039] Obtaining the port number of the message reporting channel of the visual processing device through the message sending channel;

[0040] According to the port number, creating a reporting communication identifier of the visual processing device;

[0041] The reporting communication identifier is used to subscribe to the port of the message reporting channel, and the reporting communication identifier is bound to the management identifier.

[0042] In some embodiments, the step of creating a communication identifier for the visual processing device according to the device information of the visual processing device includes:

[0043] According to the device information of the visual processing device, a communication identifier of the visual processing device is created through the integration interface on the integration end, and the integration interface is abstracted from multiple communication interfaces through a polymorphic method.

[0044] In some embodiments, the step of managing the visual sub-scheme using the attribute information of the first type of object includes:

[0045] Using the attribute information of the first type of object, calling the management identifier bound to the first type of object;

[0046] The communication identifier bound to the management identifier is called, and an operation command for the visual sub-scheme is sent to the visual processing device through the message sending channel.

[0047] In some embodiments, the step of managing the visual sub-scheme using the attribute information of the first type of object includes:

[0048] Using the attribute information of the first type of object, calling the management identifier bound to the first type of object;

[0049] The reporting communication identifier bound to the management identifier is called, and the port of the message reporting channel is polled using a callback thread to obtain the operation message for the visual sub-scheme reported by the visual processing device.

[0050] In some embodiments, the step of using the management identifier to connect to the file service running in the visual processing device includes:

[0051] Creating a file transfer identifier for the visual processing device according to the device information of the visual processing device;

[0052] Use the file transfer identifier to connect to the file service running in the visual processing device, and bind the file transfer identifier to the management identifier.

[0053] In some embodiments, the step of managing the visual sub-scheme using the attribute information of the first type of object includes:

[0054] According to the encapsulation class of the components included in the visual sub-scheme, create a second type of object of the components included in the visual sub-scheme;

[0055] According to the hierarchical structure of the components included in the visual sub-scheme, each second-category object is organized to obtain a visual sub-scheme.

[0056] In a second aspect, an embodiment of the present application provides a visual processing device distributed management system, including an integration terminal and a visual processing device, wherein the integration terminal is connected to the visual processing device;

[0057] The visual processing device is used to execute the visual sub-scheme;

[0058] The integrated end is used to connect to the visual processing device in response to a first operation; and to manage the visual sub-scheme of the visual processing device in response to a second operation, and the visual processing device executes the visual sub-scheme.

[0059] In some embodiments, the integration end is further used to create a first type of object of the visual sub-scheme of the visual processing device; and use the attribute information of the first type of object to manage the visual sub-scheme.

[0060] In some embodiments, the visual processing device is further used to run communication services and file services;

[0061] The integration end is specifically used to create a management identifier of the visual processing device bound to the first type of object; use the management identifier to connect to the communication service, and use the management identifier to connect to the file service.

[0062] In some embodiments, the visual processing device is specifically used to:

[0063] A polymorphic interface is used to enable monitoring of a first port, wherein the polymorphic interface is abstracted from a plurality of communication interfaces by a polymorphic method; an idle second port is obtained, and the polymorphic interface is used to enable publishing of the second port.

[0064] In some embodiments, a message format is pre-agreed upon between the visual processing device and the integration terminal.

[0065] In a third aspect, an embodiment of the present application provides a visual processing device distributed management device, which is applied to an integration terminal, and the device includes:

[0066] A connection module, configured to connect to a visual processing device in response to a first operation;

[0067] A management module is used to manage the visual sub-scheme of the visual processing device in response to a second operation, and the visual processing device executes the visual sub-scheme.

[0068] In some embodiments, the integrated end is connected to a plurality of visual processing devices;

[0069] The management module is specifically used to: configure a visual sub-scheme of at least one visual processing device; and send the visual sub-scheme to the visual processing device so that the visual processing device executes the visual sub-scheme.

[0070] In some embodiments, the management module is specifically used to add a visual sub-scheme of the visual processing device to the comprehensive scheme.

[0071] In some embodiments, the integrated end is connected to a plurality of visual processing devices;

[0072] The management module is specifically used to add a visual sub-scheme of at least one visual processing device to the comprehensive scheme.

[0073] In some embodiments, the management module is specifically used to: delete the visual sub-scheme of the visual processing device.

[0074] In some embodiments, the management module is further used to: send a deletion instruction to the visual processing device so that the visual processing device deletes the visual sub-scheme.

[0075] In some embodiments, the management module is specifically used to: modify the visual sub-scheme of the visual processing device;

[0076] The modified visual sub-scheme is sent to the visual processing device so that the visual processing device executes the modified visual sub-scheme.

[0077] In some embodiments, the management module is specifically used to obtain the visual processing results obtained by the visual processing device executing the visual sub-scheme.

[0078] In some embodiments, the management module is specifically used to: save the visual sub-scheme of the visual processing device, and / or save the comprehensive scheme, wherein the comprehensive scheme includes the visual sub-scheme.

[0079] In some embodiments, the management module is further used to:

[0080] Creating a first class object of a vision sub-schema of the vision processing device;

[0081] The visual sub-scheme is managed using the attribute information of the first type of objects.

[0082] In some embodiments, the management module is further used to:

[0083] Creating a management identifier of the visual processing device bound to the first type of object;

[0084] Use the management identifier to connect to the communication service running in the visual processing device, and use the management identifier to connect to the file service running in the visual processing device.

[0085] In some embodiments, the management module is specifically used to:

[0086] Creating a communication identifier for the visual processing device according to the device information of the visual processing device;

[0087] Using the communication identifier, establishing a message sending channel with the visual processing device, and binding the communication identifier with the management identifier;

[0088] Obtaining the port number of the message reporting channel of the visual processing device through the message sending channel;

[0089] According to the port number, creating a reporting communication identifier of the visual processing device;

[0090] The reporting communication identifier is used to subscribe to the port of the message reporting channel, and the reporting communication identifier is bound to the management identifier.

[0091] In some embodiments, the management module is specifically used to:

[0092] According to the device information of the visual processing device, a communication identifier of the visual processing device is created through the integration interface on the integration end, and the integration interface is abstracted from multiple communication interfaces through a polymorphic method.

[0093] In some embodiments, the management module is specifically used to:

[0094] Using the attribute information of the first type of object, calling the management identifier bound to the first type of object;

[0095] The communication identifier bound to the management identifier is called, and an operation command for the visual sub-scheme is sent to the visual processing device through the message sending channel.

[0096] In some embodiments, the management module is specifically used to:

[0097] Using the attribute information of the first type of object, calling the management identifier bound to the first type of object;

[0098] The reporting communication identifier bound to the management identifier is called, and the port of the message reporting channel is polled using a callback thread to obtain the operation message for the visual sub-scheme reported by the visual processing device.

[0099] In some embodiments, the management module is specifically used to:

[0100] Creating a file transfer identifier for the visual processing device according to the device information of the visual processing device;

[0101] Use the file transfer identifier to connect to the file service running in the visual processing device, and bind the file transfer identifier to the management identifier.

[0102] In some embodiments, the management module is specifically used to:

[0103] According to the encapsulation class of the components included in the visual sub-scheme, create a second type of object of the components included in the visual sub-scheme;

[0104] According to the hierarchical structure of the components included in the visual sub-scheme, each second-category object is organized to obtain a visual sub-scheme.

[0105] In a fourth aspect, an embodiment of the present application provides an electronic device, including:

[0106] Memory, used to store computer programs;

[0107] The processor is used to implement any of the above methods when executing the program stored in the memory.

[0108] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above-described methods is implemented.

[0109] In a sixth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the methods described above.

[0110] Beneficial effects of the embodiments of the present application:

[0111] In the technical solution provided by the embodiment of the present application, the visual processing device is connected to the integration end. After joining the visual system, the integration end is responsible for managing the visual sub-solution, and the visual processing device is responsible for running the visual sub-solution. In this way, when the visual system includes multiple visual processing devices, each visual processing device can run its own visual sub-solution, and the integration end does not need to execute the visual sub-solutions of multiple visual processing devices, that is, the integration end does not need to integrate multiple SDKs, and the entire visual system can support the dynamic increase and decrease of visual processing devices, reducing the development resources and time investment required for the development of the visual system, improving the flexibility and scalability of the development method, and solving the problem of hardware resource preemption between multiple machine vision services on the integration end side and the fluctuation of visual system performance.

[0112] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0113] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0114] Figure 1 A schematic diagram of a structure of a distributed management system for a visual processing device provided in an embodiment of the present application;

[0115] Figure 2a A schematic diagram of the first structure of the visual system provided in the embodiment of the present application;

[0116] Figure 2b A second structural schematic diagram of a visual system provided in an embodiment of the present application;

[0117] Figure 3 A schematic diagram of a first flow chart of a distributed management method for a visual processing device provided in an embodiment of the present application;

[0118] Figure 4a A schematic diagram of a first representation form of a visual sub-scheme provided in an embodiment of the present application;

[0119] Figure 4b A schematic diagram of a second representation form of the visual sub-scheme provided in an embodiment of the present application;

[0120] Figure 4c A schematic diagram of a comprehensive management interface provided in an embodiment of the present application;

[0121] Figure 5 A second flow chart of the distributed management method of the visual processing device provided in the embodiment of the present application;

[0122] Figure 6 A schematic diagram of an implementation of the connection communication service provided in an embodiment of the present application;

[0123] Figure 7 A schematic diagram of an implementation of the connection file service provided in an embodiment of the present application;

[0124] Figure 8 A schematic diagram of the operation process of the visual processing device provided in an embodiment of the present application;

[0125] Fig. 9 A schematic diagram of the operation process of the integration terminal provided in the embodiment of the present application;

[0126] Fig.10 A schematic diagram of the management architecture of the integrated terminal provided in the embodiment of the present application;

[0127] Fig.11a A schematic diagram of a device login interface provided in an embodiment of the present application;

[0128] Fig.11b A schematic diagram of a comprehensive solution management interface provided in an embodiment of the present application;

[0129] Fig.11c A first schematic diagram of the solution configuration interface provided in the embodiment of the present application;

[0130] Fig.11dA second schematic diagram of the solution configuration interface provided in the embodiment of the present application;

[0131] Fig.11e A schematic diagram of an operation interface provided in an embodiment of the present application;

[0132] Fig.11f A schematic diagram of the operating effect of the operating interface provided in the embodiment of the present application;

[0133] Fig.12 A schematic diagram of a structure of a distributed management device for visual processing equipment provided in an embodiment of the present application;

[0134] Fig.13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0135] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0136] To facilitate understanding, the terms appearing in the embodiments of the present application are explained below.

[0137] Machine vision: using machines to replace human eyes for measurement and judgment, that is, using computers to simulate human visual functions, extracting information from the images of objective objects, processing and understanding them for actual detection, measurement and control. Among them, the equipment responsible for visual processing can be called visual processing equipment, including visual processing cards, smart cameras, etc.

[0138] Vision solution: A solution for executing machine vision tasks. A vision sub-solution is a vision solution executed by a vision processing device. Multiple vision sub-solutions constitute a vision solution executed by a vision system, i.e., a comprehensive solution.

[0139] Vision processing card: A high-speed serial computer expansion bus standard (Peripheral Component Interconnect Express, PCIe) expansion card integrated with the Linux system, which can independently run image processing and analysis algorithms and provide calculation results or control signals.

[0140] Smart camera: A highly integrated miniature machine vision system. Smart cameras integrate image acquisition and processing functions, as well as communication functions, to provide a multifunctional, modular, highly reliable, and easy-to-implement machine vision solution.

[0141] Handle: An identifier used to identify an object or item, which can be used to describe a window, file, etc.

[0142] Polymorphism: A reference type has multiple states in different situations, or calls methods implemented in different subclasses through pointers to the parent class.

[0143] Thread safety: A concept in computer program code during multithreaded programming. In a program with multiple threads that share data and execute in parallel, thread-safe code uses synchronization mechanisms to ensure that each thread can execute normally and correctly without unexpected situations such as data pollution.

[0144] At present, in order to integrate a variety of visual processing devices into the visual system to collaboratively realize the corresponding machine vision applications, R&D personnel need to develop SDK software for various visual processing devices, integrate multiple visual processing devices into the visual system through the SDK software of multiple visual processing devices, and manage each visual processing device by calling the SDK software of each visual processing device. This management method of visual processing devices requires a large investment of development resources and time, and the development method is less flexible. In addition, the SDK software of multiple visual processing devices needs to be run in the visual system. When realizing multi-device collaborative machine vision applications, hardware resource preemption is likely to occur between multiple SDK software, which in turn causes fluctuations in the performance of the visual system.

[0145] To solve the above problems, the present application provides a distributed management system for visual processing devices, such as Figure 1 As shown, the distributed management system includes an integration terminal 101 and one or more visual processing devices 102 .

[0146] The visual processing device 102 is used to execute the visual sub-scheme. The visual sub-scheme can be configured in the visual processing device 102 by the integration terminal 101 through commands.

[0147] The integrated terminal 101 is used to connect with the visual processing device 102 in response to a first operation; and to manage the visual sub-scheme of the visual processing device 102 in response to a second operation, so that the visual processing device 102 executes the visual sub-scheme.

[0148] In the embodiment of the present application, the visual system may include a primary distributed management system, such as Figure 2aIn the visual system shown, cameras 11 to 12 are connected to a visual processing card 16, and camera 13 is connected to a visual processing card 17. Visual processing cards 16 and 17 can be inserted into a server using a PCIe interface. Cameras 14 and 15 are connected to the server via a network cable. Among them, cameras 11 to 13 are responsible for collecting images and do not have a visual processing function. Visual processing card 16, visual processing card 17, camera 14 and camera 15 have a visual processing function. Visual processing card 16, visual processing card 17, camera 14, camera 15, and server can be regarded as a first-level distributed management system, in which visual processing card 16, visual processing card 17, camera 14 and camera 15 are visual processing devices 102, and the server is an integrated terminal 101.

[0149] The vision system can also include a multi-level distributed management system, such as Figure 2b In the visual system shown, cameras 21 and 22 are connected to visual processing card 26, and camera 23 is connected to visual processing card 27. Visual processing card 26 and visual processing card 27 can use PCIe interface and be inserted into the server. Camera 24 and camera 25 are connected to the server via a network cable. Among them, camera 23 is responsible for collecting images and does not have visual processing function. Camera 21, camera 22, visual processing card 26, visual processing card 27, camera 24 and camera 25 have visual processing function. Figure 2b In the visual system shown, camera 21, camera 22, and visual processing card 26 can be regarded as a first-level distributed management system, in which camera 21 and camera 22 are visual processing devices 102, and visual processing card 26 is an integrated terminal 101; visual processing card 26, visual processing card 27, camera 24, camera 25, and server can be regarded as a second-level distributed management system, in which visual processing card 26, visual processing card 27, camera 24 and camera 25 are visual processing devices 102, and the server is an integrated terminal 101.

[0150] In the embodiment of the present application, each level of distributed management system can adopt the above-mentioned method to perform distributed management.

[0151] In the technical solution provided by the embodiment of the present application, the visual processing device is connected to the integration end. After joining the visual system, the integration end is responsible for managing the visual sub-solution, and the visual processing device is responsible for running the visual sub-solution. In this way, when the visual system includes multiple visual processing devices, each visual processing device can run its own visual sub-solution, and the integration end does not need to execute the visual sub-solutions of multiple visual processing devices, that is, the integration end does not need to integrate multiple SDKs, and the entire visual system can support the dynamic increase and decrease of visual processing devices, reducing the development resources and time investment required for the development of the visual system, improving the flexibility and scalability of the development method, and solving the problem of hardware resource preemption between multiple machine vision services on the integration end side and the fluctuation of visual system performance.

[0152] In addition, each visual processing device executes its own visual sub-plan, and the integration end only needs to manage the visual sub-plan but does not need to run the visual sub-plan, thus achieving logical decoupling and hardware isolation between visual processing devices, reducing the maintenance complexity and difficulty of the visual processing devices, and improving the maintainability and performance stability of the visual processing devices.

[0153] In the embodiment of the present application, the visual processing device 102 has a built-in operating system, such as Windows system and Linux system, and can support multiple communication modes such as network communication or PCIe communication. The algorithm framework of the modules, processes and other entities required for the execution of the visual scheme is configured in the visual processing device 102. After the visual processing device 102 is started, the visual processing device 102 can start the algorithm framework, and after obtaining the visual sub-scheme configured by the integration terminal 101, execute the visual sub-scheme to call the corresponding modules, processes and other entities to process the image.

[0154] The integration terminal 101 can be a device with processing functions such as a server, a cluster or a visual processing device, or it can be software running on any electronic device. The integration terminal 101 can connect to one or more visual processing devices 102 and manage the visual processing devices 102, such as configuring, modifying, deleting, querying, controlling and other management operations on the visual sub-schemes executed by the visual processing devices 102, including selecting cameras, adding and modifying operators, and performing parameter adjustment.

[0155] Specifically, after the integrated end 101 is connected to a visual processing device 102 , the following management operation (ie, the second operation) can be performed on the visual sub-scheme of the visual processing device 102 .

[0156] 1) Configuration. The integration terminal 101 can configure the visual sub-scheme of the visual processing device 102; and send the visual sub-scheme of the visual processing device 102 to the visual processing device 102. In this way, the visual processing device 102 executes the visual sub-scheme of the visual processing device 102. After the integration terminal 101 completes the configuration of the visual sub-scheme, each visual processing device 102 can independently run the visual sub-scheme.

[0157] In the embodiment of the present application, after the integrated end 101 is connected to a visual processing device 102, the visual sub-scheme of the visual processing device 102 may be configured or not.

[0158] In this case, if the integrated terminal 101 is connected to multiple visual processing devices 102, the integrated terminal 101 can configure the visual sub-scheme of at least one visual processing device 102, that is, the integrated terminal 101 can configure the visual sub-scheme of some or all of the connected visual processing devices 102.

[0159] 2) Comprehensive management: The integration terminal 101 can add the visual sub-scheme of the visual processing device 102 to the comprehensive scheme.

[0160] In the embodiment of the present application, after the integration end 101 is connected to a visual processing device 102, the visual sub-scheme of the visual processing device 102 can be added or not added to the comprehensive scheme.

[0161] In this case, if the integrated end 101 is connected to multiple visual processing devices 102, the integrated end 101 can add a visual sub-scheme of at least one visual processing device 102 to the comprehensive scheme, that is, the integrated end 101 can add some or all of the visual sub-schemes of the visual processing devices 102 to the comprehensive scheme.

[0162] The integrated end 101 can add the visual sub-scheme of the visual processing device 102 to the comprehensive scheme after completing the configuration of the visual sub-scheme of the visual processing device 102; the integrated end 101 can configure the visual sub-scheme of the visual processing device 102 after adding the visual sub-scheme of the visual processing device 102 to the comprehensive scheme, and there is no limitation on this.

[0163] In an embodiment of the present application, an integrated solution may include visual sub-solutions of multiple visual processing devices 102. The integration terminal 101 uses an integrated solution to jointly manage the visual sub-solutions of multiple visual processing devices 102, such as controlling the execution timing of multiple visual sub-solutions, exchanging image data of multiple visual sub-solutions, and aggregating processing results of multiple visual sub-solutions.

[0164] 3) Delete.

[0165] In the embodiment of the present application, the integration terminal 101 may delete the visual sub-scheme of one or more visual processing devices 102. The visual sub-scheme may have been added to the comprehensive scheme or may not have been added to the comprehensive scheme.

[0166] After deleting a visual sub-scheme of a visual processing device 102, the integration terminal 101 may also send a deletion instruction to the visual processing device 102 so that the visual processing device 102 deletes the visual sub-scheme of the visual processing device 102. In this way, the energy consumption of the visual processing device 102 can be reduced.

[0167] After deleting a visual sub-scheme of a visual processing device 102, the integration terminal 101 may not send a deletion instruction to the visual processing device 102. In this way, the visual processing device 102 can continue to execute the visual sub-scheme of the visual processing device 102. When the visual processing device 102 is required to execute the visual sub-scheme again, there is no need to send the visual sub-scheme again, thereby improving the management efficiency of the visual processing device.

[0168] 4) Modification.

[0169] In an embodiment of the present application, after the integrated end 101 is connected to a visual processing device 102, the visual sub-scheme of the visual processing device 102 can be modified; the modified visual sub-scheme is sent to the visual processing device 102, so that the visual processing device 102 executes the modified visual sub-scheme.

[0170] 5) Control.

[0171] In the embodiment of the present application, after the integrated terminal 101 is configured with a visual processing device 102, the visual processing result obtained by the visual processing device 102 executing the visual sub-scheme can be obtained. The visual processing result can be an intermediate result or a final result of the visual processing device 102 executing the visual sub-scheme.

[0172] 6) Save.

[0173] In the embodiment of the present application, after obtaining a visual sub-scheme of a visual processing device 102, the integration terminal 101 can save the visual sub-scheme of the visual processing device 102. After adding one or more visual sub-schemes to the comprehensive scheme and obtaining the comprehensive scheme, the integration terminal 101 can save the comprehensive scheme.

[0174] After the integrated terminal 101 saves the visual sub-scheme or the comprehensive scheme, the user can subsequently operate the integrated terminal 101 to load the visual sub-scheme or the comprehensive scheme, and the user does not need to reconfigure the visual sub-scheme and the comprehensive scheme, thereby improving the management efficiency of the visual system and reducing the resource time requirement.

[0175] 7) Query.

[0176] In the embodiment of the present application, after obtaining the visual sub-scheme and the comprehensive scheme of the visual processing device 102, the integration terminal 101 can query the visual sub-scheme and the comprehensive scheme in real time.

[0177] Through the above management operations, distributed management of machine vision tasks based on multiple vision processing devices is achieved.

[0178] In the embodiment of the present application, in order to facilitate the management of the visual processing device 102, the integration terminal 101 can create a class object of the visual sub-scheme executed by the visual processing device 102, such as a first-class object, and bind the first-class object to the management identifier of the visual processing device 102. In addition, the integration terminal 101 obtains attribute information such as the function name, workstation name, included process name and function module name, and the hierarchical relationship between the process and the function module of the first-class object. The integration terminal 101 uses the attribute information of the first-class object to manage the visual sub-scheme of the visual processing device 102.

[0179] For example, when configuring the visual sub-scheme of the visual processing device 102, the integration terminal 101 can configure the parameter value of the first type of object; use the attribute information of the first type of object, call the management identifier bound to the first type of object, and send the visual sub-scheme to the visual processing device 102.

[0180] When adding the visual sub-scheme of the visual processing device 102 into the comprehensive scheme, the integration terminal 101 may integrate the attribute information of the first category of objects into the comprehensive scheme.

[0181] When the integration terminal 101 is connected to multiple visual processing devices 102, the integration terminal 101 can use the same method to create the first-class objects of the visual sub-schemes of the multiple visual processing devices 102, and integrate all or part of the attribute information of the first-class objects in the comprehensive scheme, so as to integrate multiple visual sub-schemes into the same comprehensive scheme. Multiple visual processing devices 102 respectively execute visual sub-schemes, and the integration terminal 101 uses one comprehensive scheme to jointly manage the visual sub-schemes of multiple visual processing devices 102.

[0182] The integration terminal 101 can create at least one first-class object, and the integration terminal can delete some or all of the first-class objects in at least one first-class object. The comprehensive scheme can include attribute information of some or all of the first-class objects in at least one first-class object, and the integration terminal 101 can delete the attribute information of some or all of the first-class objects included in the comprehensive scheme. This realizes the dynamic addition and deletion of visual processing devices connected to the integration terminal, and the dynamic addition and deletion of visual sub-schemes.

[0183] In addition, the visual processing devices 102 are uniformly abstracted as first-class objects. In this way, the graphical interface and the interface are basically unaware of the differences in the visual processing devices. By dynamically adding, deleting, configuring and controlling the visual processing devices 102 through simple operations, the existing visual solutions can be modified and expanded conveniently, which reduces the maintenance effort and difficulty and reduces the resource and time requirements.

[0184] In some embodiments, the visual processing device 102 can also be used to run communication services and file services; the integration terminal 101 can also be used to create a management identifier of the visual processing device 102; use the management identifier to connect to the communication service, and use the management identifier to connect to the file service.

[0185] In the embodiment of the present application, the visual processing device 102 deploys service programs of the management system in the built-in operating system, such as communication services and file services. The communication service may include sending communication services and / or reporting communication services. The file service may be a File Transfer Protocol (FTP) service, a User Datagram Protocol (UDP) service, or other types of file services, which are not limited.

[0186] After the visual processing device 102 is started, the visual processing device 102 can initialize the communication service and the file service. The communication service can be used to transmit message type data with the integration terminal 101 to complete commands and responses; the file service can be used to transmit file type data with the integration terminal 101 to complete file exchange, such as file upload, download, rename, information query and other functions.

[0187] The integration terminal 101 creates a management identifier of the visual processing device, uses the management identifier to connect to the communication service running in the visual processing device 102, and uses the management identifier to connect to the file service running in the visual processing device 102. At this point, the connection between the integration terminal 101 and the visual processing device 102 is completed. The integration terminal 101 calls the management identifier to access the visual processing device 102, and exchange command messages and files with the visual processing device 102, which facilitates the management of the visual processing device 102 and the management of the entire machine vision task.

[0188] In some embodiments, the visual processing device 102 can be specifically used to: use a polymorphic interface to enable listening on a first port, where the polymorphic interface abstracts multiple communication interfaces through a polymorphic method; obtain an idle second port, and use the polymorphic interface to enable publishing on the second port.

[0189] Among them, the communication interface can be a Socket interface in network communication or a PCIe interface in PCIe communication. The visual processing device 102 can abstract multiple communication interfaces through a polymorphic method to obtain a polymorphic interface. Using this polymorphic interface, it can be realized that in the network communication mode, the visual processing device 102 uses the Socket interface to complete the communication with the integrated terminal 101; it can be realized that in the PCIe communication mode, the visual processing device 102 uses the PCIe interface (i.e., PCIe expansion card) to complete the communication with the integrated terminal 101; it can also be realized that in other communication modes, the visual processing device 102 uses the corresponding interface to complete the communication. Through the polymorphic interface, the visual processing device 102 can be compatible with the differences between different communication methods. Accordingly, the visual system can flexibly select visual processing devices 102 of different types and manufacturers to apply to different application scenarios.

[0190] There are multiple ports under the polymorphic interface. The first port can be fixed to any port among the multiple ports, and the second port can be any idle port among the multiple ports, and the second port can change dynamically.

[0191] The visual processing device 102 can use the communication service (such as issuing the communication service) to start the communication monitoring, that is, to start monitoring the first port and wait for the integrated terminal 101 to send a command message. The completion of starting the first port monitoring indicates that the construction of the sending channel of the message data is completed. Through the sending channel (i.e., the message sending channel), the visual processing device 102 can receive the command message sent by the integrated terminal 101. In the embodiment of the present application, the opening of the monitoring of the first port can be implemented in a Request-Response mode, or in other modes, which is not limited.

[0192] In addition, the visual processing device 102 can use communication services (such as reporting communication services) to start communication publishing, that is, start publishing on the second port to report in real time the reply messages generated during the operation of the visual processing device 102, such as the reply messages corresponding to the command messages sent by the integration terminal 101.

[0193] The completion of opening the second port for publishing indicates that the reporting channel for message data is constructed. Through the reporting channel (i.e., the message reporting channel), the visual processing device 102 can publish command messages to the integration terminal 101. In the embodiment of the present application, opening the second port for publishing can be implemented in a publish-subscribe mode or in other modes, which is not limited.

[0194] In order to realize the management of the visual processing device 102 by the integration terminal 101, the message format can be pre-agreed between the visual processing device 102 and the integration terminal 101. This helps the visual processing device 102 and the integration terminal 101 to understand the command message from the other end, and then accurately process the command message and complete the communication service's response to the command.

[0195] For example, the visual processing device 102 and the integration terminal 101 pre-agree that the message format is command code + parameter. The visual processing device 102 uses the communication service to receive the command message sent by the integration terminal 101, disassembles the command message according to the pre-agreement message format, obtains the command code and parameters, and performs actions according to the command code using the corresponding parameters; constructs a reply message corresponding to the command message according to the pre-agreement message format, and uses the communication service and the port for receiving the command message to feed back the reply message to the integration terminal 101. In addition, if asynchronous actions are involved, such as the visual processing device 102 receives the command message for a period of time, completes the command message processing, and obtains the reply message corresponding to the command message, the visual processing device 102 can use the communication service and use the publishing port to feed back the reply message to the integration terminal 101.

[0196] Based on the above distributed management system, the present application embodiment provides a distributed management method for visual processing devices, such as Figure 3 As shown, it is applied to Figure 1 The integrated end shown in the figure comprises the following steps:

[0197] Step S301, in response to a first operation, connecting with a visual processing device;

[0198] Step S302, in response to the second operation, manage the visual sub-scheme of the visual processing device, and the visual processing device executes the visual sub-scheme.

[0199] In the technical solution provided by the embodiment of the present application, the visual processing device is connected to the integration end. After joining the visual system, the integration end is responsible for managing the visual sub-solution, and the visual processing device is responsible for running the visual sub-solution. In this way, when the visual system includes multiple visual processing devices, each visual processing device can run its own visual sub-solution, and the integration end does not need to execute the visual sub-solutions of multiple visual processing devices, that is, the integration end does not need to integrate multiple SDKs, and the entire visual system can support the dynamic increase and decrease of visual processing devices, reducing the development resources and time investment required for the development of the visual system, improving the flexibility and scalability of the development method, and solving the problem of hardware resource preemption between multiple machine vision services on the integration end side and the fluctuation of visual system performance.

[0200] In the above step S301, the first operation is used to connect the integrated terminal to the visual processing device. The first operation may include inputting the address and device serial number of the visual processing device, etc. In order to facilitate the integrated terminal to provide a human-computer interaction interface, the user performs the first operation in the human-computer interaction interface, and then the integrated terminal is connected to the visual processing device.

[0201] In the above step S302, the second operation is used for the integrated end to manage the visual processing device. The management operation can refer to the above related description. To facilitate the management of the visual sub-scheme, the integrated end can provide a human-computer interaction interface, and the user performs the second operation in the human-computer interaction interface, so that the integrated end manages the visual processing device.

[0202] In some embodiments, to facilitate the management of visual sub-schemes, the integration end may create a first-class object of the visual sub-scheme of the visual processing device; the above step S302 may be: using the attribute information of the first-class object to manage the visual sub-scheme.

[0203] The attribute information of the first-class object may include, but is not limited to, the function name, workstation name, process name and function module name, and the hierarchical relationship between the process and the function module, etc. The integration end encapsulates the visual sub-scheme class. After the integration end is connected to the visual processing device, the integration end dynamically creates an object of the visual sub-scheme class, i.e., the first-class object.

[0204] In an embodiment of the present application, the integration end can dynamically create objects of the business class (i.e., the second class objects of the components) in the first class objects according to the encapsulation class of each component included in the visual sub-scheme, i.e., the business class, create the second class objects of the components included in the visual sub-scheme, and organize each second class object according to the hierarchical structure of the components included in the visual sub-scheme to obtain the visual sub-scheme. Among them, the integration end can open the properties and interfaces of each business class object to the outside world, hide the management identifier and the integration interface, and facilitate the user to configure the visual sub-scheme. To facilitate the configuration of the visual sub-scheme, the integration end can display the comprehensive management interface through the human-computer interaction interface. Users can configure the algorithm group, operator module, camera module, image data and other components included in the visual sub-scheme on the human-computer interaction interface to generate a visual sub-scheme.

[0205] like Figure 4a and Figure 4b The visual sub-schema shown, Figure 4a and Figure 4b The visual sub-schemes in the Figure 4aVisual sub-scheme 1 is presented in the form of full data. For example, visual sub-scheme 1 includes a process and business objects such as a global module. The process includes business objects such as a group and module 1. Module 1 includes subscription relationships, connection relationships, configuration parameters, and display data for input and output (IO). Figure 4b The visual sub-scheme 2 is expressed in the form of result data. For example, the visual sub-scheme 2 includes business objects such as process 1 to process 3 and global module, process 1 includes business objects such as group, module 1 and module 2, and group includes business objects such as module 3. Process 1 to process 3, global module, group, module 1 to module 3 have IO functions respectively. Figure 4a The visual sub-scheme can be presented in the form in which the visual sub-scheme is displayed. Figure 4b The presentation form of the visual sub-scheme may be the display form when the visual sub-scheme is hidden, and there is no limitation on the presentation forms of various visual sub-schemes.

[0206] In an embodiment of the present application, after the integration end completes the configuration of the visual sub-scheme, it obtains the attribute information of the first-category object of the visual sub-scheme, such as the name of the first-category object, the name of the business-class object, etc., and can use the attribute information of the first-category object to manage the visual sub-scheme, such as integrating the attribute information of the first-category object into the comprehensive scheme, etc.

[0207] The integration end may also store multiple visual sub-schemes in advance. The integration end obtains the identifier of the visual sub-scheme, such as the name of the first-category object, and uses the identifier of the visual sub-scheme to obtain the attribute information of the first-category object, such as the name of the first-category object and the name of the object of each business class, and then uses the attribute information of the first-category object to manage the visual sub-scheme, such as integrating the attribute information of the first-category object into the comprehensive scheme.

[0208] In some embodiments, the integration end uses the attribute information of the first type of object to manage the visual sub-scheme, which may be: using the attribute information of the first type of object, performing a second operation on the first type of object to obtain an operation command; and sending the operation command to the visual processing device. The second operation performed on the first type of object is an operation on the visual sub-scheme, which may include but is not limited to operations such as image display, parameter configuration, operation control, and data display.

[0209] To facilitate the management of visual processing equipment, a human-computer interaction interface can be displayed on the integrated terminal. The human-computer interaction interface includes operation controls of the visual sub-solution, such as Figure 4c The comprehensive management interface shown may include an image display control, a parameter configuration control, an operation control control, and a data display control, which displays information of Scheme 1 and Scheme 2 and hides the content of Scheme N.

[0210] The integration end can display operation controls in the human-computer interaction interface, and the user can click, select, and perform other operations on the operation controls; based on the user's operation on the operation controls, the integration end operates the first-class objects of the visual sub-scheme, thereby achieving the purpose of operating the first-class objects using the operation controls on the comprehensive management interface. The integration end manages the visual processing device based on the user's operation on the first-class objects, such as generating an operation command based on the operation on the first-class objects, and sending the operation command to the visual processing device.

[0211] Take the image display control as an example. The user clicks the image display control on the integrated management interface of the integrated terminal, and then selects the first-class object to be displayed and the type of image to be displayed based on the attribute information of each first-class object, such as selecting the binary image in the first-class object 1 to be displayed; the integrated terminal generates an operation command for obtaining the binary image according to the user's selection, and sends the operation command to the visual processing device 1 corresponding to the selected first-class object 1. The visual processing device 1 sends the binary image processed by the image binarization module to the integrated terminal according to the operation command. The integrated terminal displays the binary image.

[0212] In an embodiment of the present application, the integrated terminal may pre-acquire an image from a visual processing device. The integrated terminal uses the attribute information of the first type of object to perform a second operation on the first type of object to obtain an operation command; after obtaining the operation command, the integrated terminal may use the attribute information of the first type of object to extract an image corresponding to the operation command from the local integrated terminal. For example, the operation command is to display a binary image. The integrated terminal extracts the binary image from the local integrated terminal and displays it.

[0213] In the embodiment of the present application, the integrated end can also use other methods to manage the visual processing device, which is not limited to this.

[0214] In addition, in the embodiment of the present application, the integrated terminal can provide a human-computer interaction interface, and the integrated terminal displays the comprehensive solution or visual sub-solutions in the human-computer interaction interface. The user can intuitively understand each visual sub-solution based on the comprehensive solution and accurately operate the corresponding first-class object.

[0215] In some embodiments, Figure 5 As shown, a distributed management method of a visual processing device is also provided, which is applied to an integration terminal. The method may include the following steps:

[0216] Step S501, in response to the first operation, connect to the visual processing device; see the relevant description of the above step S301.

[0217] Step S502, in response to the second operation, creating a management identifier of a visual processing device bound to the first type of object;

[0218] In the embodiment of the present application, the management identifier is an identifier for accessing the visual processing device, and each visual processing device has a unique management identifier, which can be a handle, a pointer, or a structure. Before binding the first-class object to the management identifier of the visual processing device, the integration end creates an identifier for accessing the visual processing device, namely the management identifier, which is the access identifier of the visual processing device.

[0219] After subsequently creating the first-class object, the integration end binds the first-class object to the management identifier.

[0220] In some embodiments, the integration terminal can also bind the management identifier to the device serial number of the visual processing device. This facilitates the technical solution provided by the embodiment of the present application to be applied to special scenarios where devices need to be identified, thereby expanding the application scope of the embodiment of the present application.

[0221] Step S503, using the management identifier, connecting to the communication service running in the visual processing device, and using the management identifier, connecting to the file service running in the visual processing device;

[0222] In an embodiment of the present application, the integrated end uses a management identifier to respectively connect to the communication service and file service running in the visual processing device to realize the transmission of messages and files.

[0223] After running various services such as communication services and file services in the visual processing device, the integration end can use the communication services and file services to exchange messages and files with the visual processing device. For example, it can use the communication services to exchange command messages with the visual processing device to configure the visual sub-scheme executed by the visual processing device; for another example, it can use the file services to obtain and display the images collected by the visual processing device, or pass the images collected by the visual processing device to other visual processing devices.

[0224] Step S504, creating a first class object of a visual sub-schema of a visual processing device;

[0225] Step S505: Use the attribute information of the first type of objects to manage the visual sub-scheme.

[0226] When the integrated end operates the first type of object, it can use the attribute information of the first type of object, call the management identifier bound to the first type of object, and interact with the visual processing device to transmit the operation of the first type of object to the visual processing device, thereby realizing access and management of the visual processing device.

[0227] For the above steps S504 to S505, please refer to the above related descriptions.

[0228] In the technical solution provided in the embodiment of the present application, the visual processing device executes the visual sub-solution. The integration end binds the first class of objects of the visual sub-solution to the management identifier of the visual processing device. In addition, the integration end integrates the attribute information of the first class of objects into the comprehensive solution, and uses the attribute information of the first class of objects to find the first class of objects, and further obtain the management identifier, and call the management identifier to access the corresponding visual processing device, and manage the visual processing device, such as managing the visual sub-solution executed by the visual processing device.

[0229] In an embodiment of the present application, when there are multiple visual processing devices in the visual system, the integration end can uniformly abstract the visual processing devices, so that the visual processing devices are uniformly represented as the attribute information of the first-class objects in the comprehensive solution, and the differences between the visual processing devices can be shielded. In this case, multiple visual processing devices can respectively execute their own visual sub-solutions, and the integration end accesses the corresponding visual processing devices through the objects in the comprehensive solution to manage the visual processing devices. The integration end does not need to integrate multiple SDKs, and the entire visual system can support the dynamic increase and decrease of visual processing devices, which reduces the development resources and time investment required for the development of the visual system, improves the flexibility and scalability of the development method, and solves the problems of hardware resource preemption and visual system performance fluctuations between multiple machine vision businesses on the integration end side.

[0230] In addition, in the technical solution provided by the embodiment of the present application, the integrated end connects the communication service and file service running in the visual processing device through the management identifier, thereby realizing the diversity of information interaction between the integrated end and the visual processing device, and helping to expand the application scenarios of the distributed management system. In addition, the integrated end abstracts the visual processing device into an undifferentiated management identifier. When managing the visual processing device, the integrated end does not need to care about the processing logic and hardware type of the visual processing device. Using the management identifier, each visual processing device can be accessed, thereby realizing logical decoupling and hardware isolation between visual processing devices, supporting dynamic addition and subtraction of various types of visual processing devices, reducing the complexity and difficulty of maintaining the visual processing device, and improving the maintainability and performance stability of the visual processing device.

[0231] In some embodiments, Figure 6 As shown, the integrated device can connect to the communication service using the following steps:

[0232] Step S601, creating a communication identifier for the visual processing device according to the device information of the visual processing device;

[0233] In the embodiment of the present application, the device information may include an Internet Protocol (IP) address or a PCIe serial number, etc. The integration end creates a communication identifier for the visual processing device, and the communication identifier includes the device information of the visual processing device. The communication identifier may be a handle, a pointer, or a structure.

[0234] In some embodiments, the integrated terminal can abstract multiple communication interfaces through a polymorphic method to obtain an integrated interface. The generation of the integrated interface can refer to the relevant description of the above-mentioned polymorphic interface, which will not be repeated here. The integrated terminal creates a downward communication identifier of the visual processing device through the integrated interface on the integrated terminal based on the device information of the visual processing device. Subsequently, the integrated terminal can call the downward communication identifier on the integrated interface to access the visual processing device that supports communication methods such as network communication or PCIe communication. Through the integrated interface, the integrated terminal can be compatible with the differences between different communication methods. Accordingly, the visual system can flexibly select visual processing devices of different types and manufacturers for different application scenarios.

[0235] In an embodiment of the present application, the integrated end can also independently deploy multiple communication interfaces, and create a communication identifier for the visual processing device through the communication interface of the visual processing device based on the device information of the visual processing device.

[0236] The integration end may also deploy multiple communication interfaces and multiple integration interfaces, and there is no limitation on this.

[0237] Step S602, using the issued communication identifier to establish a message issuing channel with the visual processing device, and binding the issued communication identifier with the management identifier;

[0238] After creating the communication identifier of the visual processing device, the integrated end can call the communication identifier to connect with the visual processing device and establish a message sending channel. In addition, after creating the communication identifier of the visual processing device, the integrated end can also bind the communication identifier to the management identifier. In this way, when the integrated end calls the management identifier, it can call the communication identifier bound to the management identifier, and then send a command message to the visual processing device.

[0239] Step S603, obtaining the port number of the message reporting channel of the visual processing device through the message sending channel;

[0240] After the message sending channel is established, the integration end can send a command message to obtain the port number of the message reporting channel (such as the port number of the second port mentioned above) to the visual processing device through the message sending channel; then, the visual processing device returns a reply message to the integration end through the message sending channel, and the reply message includes the port number of the message reporting channel. The integration end extracts the port number of the message reporting channel from the reply message.

[0241] In the embodiment of the present application, after the message sending channel is established, the integration end can also allocate the port number of the message reporting channel; and send a command message for configuring the port number to the visual processing device through the message sending channel, and the command message carries the port number allocated by the integration end. The visual processing device extracts the port number allocated by the integration end from the command message, and then configures the port number of the message reporting channel to the port number carried in the command message.

[0242] Step S604, creating a reporting communication identifier of the visual processing device according to the obtained port number;

[0243] After the integration end obtains the port number of the message reporting channel, it creates a reporting communication identifier of the visual processing device, and the reporting communication identifier includes the obtained port number of the message reporting channel. The reporting communication identifier can be a handle, a pointer or a structure.

[0244] Step S605: Use the reporting communication identifier to subscribe to the port of the message reporting channel, and bind the reporting communication identifier to the management identifier.

[0245] After creating the reporting communication identifier of the visual processing device, the integrated end can call the reporting communication identifier to subscribe to the port of the message reporting channel to obtain the command message (such as reply message) reported by the visual processing device. At this point, the message reporting channel is established. In addition, after creating the reporting communication identifier of the visual processing device, the integrated end can also bind the reporting communication identifier to the management identifier. In this way, when the integrated end calls the management identifier, it can call the reporting communication identifier bound to the management identifier to obtain the command message from the visual processing device.

[0246] In the technical solution provided in the embodiment of the present application, the integrated end sends messages to the corresponding visual processing device and obtains messages reported by the corresponding visual processing device through a unique identifier (i.e., a management identifier), thereby realizing the basis required for distributed management of the visual processing device.

[0247] In some embodiments, Figure 7 As shown, the integrated device can connect to the file service using the following steps:

[0248] Step S701, creating a file transfer identifier of the visual processing device according to the device information of the visual processing device;

[0249] The integration end creates a file transfer identifier of the visual processing device, such as an FTP / UDP identifier, which includes device information of the visual processing device. The file transfer identifier can be a handle, a pointer, or a structure.

[0250] Step S702, use the file transfer identifier to connect to the file service running in the visual processing device, and bind the file transfer identifier to the management identifier.

[0251] After creating the file transfer identifier of the visual processing device, the integration end can call the file transfer identifier to connect to the file service running in the visual processing device. In addition, after creating the file transfer identifier of the visual processing device, the integration end can also bind the file transfer identifier to the management identifier. In this way, when the integration end calls the file transfer identifier, it can call the file transfer identifier bound to the management identifier, and then exchange files with the visual processing device.

[0252] In the technical solution provided in the embodiment of the present application, the integrated end exchanges files with the corresponding visual processing device through a unique identifier (i.e., a management identifier), thereby realizing the basis required for distributed management of the visual processing device.

[0253] In the embodiment of the present application, the integrated end realizes the connection between the integrated end and the visual processing device by managing the identification, issuing the communication identification, reporting the communication identification and the file transfer identification. Using this connection, the integrated end and the visual processing device can exchange messages and files without worrying about the processing logic and hardware type of the visual processing device, etc., thereby realizing logical decoupling and hardware isolation between visual processing devices, supporting dynamic increase and decrease of various types of visual processing devices, reducing the complexity and difficulty of maintaining the visual processing devices, and improving the maintainability and performance stability of the visual processing devices.

[0254] In an embodiment of the present application, the above-mentioned downward communication identifier, upward communication identifier and file transfer identifier can be integrated into an integrated interface, and the management identifier is bound to the integrated interface. In this way, the integrated end can call the corresponding integrated interface through the management identifier to realize the interaction of messages and files with the visual processing device, thereby further improving the maintainability and performance stability of the visual processing device.

[0255] In some embodiments, based on the above-mentioned message sending channel, the integration end can adopt the following steps to manage the visual processing device: use the attribute information of the first type of object to call the management identifier bound to the first type of object; call the sending communication identifier bound to the management identifier, and send the operation command for the visual sub-scheme to the visual processing device through the message sending channel.

[0256] In the embodiment of the present application, the operation command is a command message of the operation (such as the second operation) performed by the integrated end on the visual processing device. The operation command can be a periodically generated operation command, or a command message generated by the integrated end for operating various attributes / interfaces of the visual sub-scheme, or a command message generated by a user manually operating the integrated scheme, which is not limited.

[0257] After obtaining the operation command for the visual sub-scheme, the integrated end can use the attribute information of the first type of objects in the comprehensive scheme to determine the first type of objects, and then call the management identifier bound to the first type of objects. The management identifier is also bound to the downward communication identifier in the integrated interface. After calling the management identifier, the integrated end can further call the downward communication identifier bound to the management identifier to access the visual processing device that executes the above-mentioned visual sub-scheme, that is, to issue an operation command to the visual processing device.

[0258] In some embodiments, based on the above-mentioned message reporting channel, the integration end can adopt the following steps to manage the visual processing device: use the attribute information of the first type of object to call the management identifier bound to the first type of object; call the reporting communication identifier bound to the management identifier, and use the callback thread to poll the port of the message reporting channel to obtain the operation messages for the visual sub-schemes reported by multiple visual processing devices.

[0259] In an embodiment of the present application, the operation message may be a message generated after the visual processing device processes the command message sent by the integration end, such as the above-mentioned reply message, or the operation message may also be other command messages that the visual processing device actively reports to the integration end. The visual processing device sends the operation message to the integration end through the message reporting channel. The integration end determines the first-class object by using the attribute information of the first-class object in the comprehensive solution, and then calls the management identifier bound to the first-class object. The management identifier is also bound to the reporting communication identifier in the integration interface. After calling the management identifier, the integration end can further call the reporting communication identifier bound to the management identifier to use the callback thread to poll the port of the message reporting channel. The port of the message reporting channel has been subscribed, so the integration end uses the callback thread to poll the port of the message reporting channel to obtain the operation message reported by the visual processing device through the message reporting channel.

[0260] Combine the following Figure 8 The operation process of the visual processing device side shown in Fig. 9 The operation process of the integration end shown, and Fig.10 The management architecture of the integrated terminal shown provides a detailed description of the distributed management method of the visual processing device provided in the embodiment of the present application.

[0261] The operation process of the visual processing device includes:

[0262] Step S801, initializing the service.

[0263] After the visual processing device starts the service program normally, it starts the algorithm framework and initializes the communication service and file service. In the embodiment of the present application, the services of the visual processing device (including communication services and file services) implement communication monitoring and publishing for message sending and reporting, and implement file services for file exchange. The initialization steps of the communication service and file service may specifically include:

[0264] Step 1.1: The service starts communication monitoring and waits for the integration end to send messages.

[0265] The service starts listening on the specified port (such as the first port mentioned above). In order to be compatible with different modes such as network communication and PCIe communication, and retain the expansion capability, the service can use polymorphic methods to abstract communication-related interfaces to be compatible with the differences between different communication modes, so that the network uses Socket to complete communication and PCIe uses expansion card drivers to complete communication. At this point, the message data delivery channel is built.

[0266] Step 1.2: The service starts communication publishing and reports the messages generated during operation in real time.

[0267] The service randomly obtains an idle port (such as the second port mentioned above) and starts publishing. In addition, the service uses polymorphic methods to abstract communication-related interfaces. The message data reporting channel is built.

[0268] Step 1.3, define the message data format and complete the service's response to the command.

[0269] The visual processing device and the integration end pre-agree on the format of sending and reporting messages, such as using a command code + parameter mode.

[0270] Step 1.4, the device-side service implements the file service for file exchange.

[0271] Implement corresponding file services based on FTP or UDP to provide functions such as file upload, download, rename, and information query.

[0272] Step S802: Receive a command message, wherein the command message includes a command code and parameters.

[0273] In the embodiment of the present application, the visual processing device receives a command message sent by the integration terminal through a designated port using a communication service. The command message may instruct the visual processing device to perform any one or more operations in step S803.

[0274] Step S803, configure the visual sub-scheme, execute the visual sub-scheme, and feedback the execution result of the sub-scheme.

[0275] In an embodiment of the present application, the visual processing device disassembles the command message, obtains the command code and parameters, and uses the parameters to perform actions according to the command code.

[0276] In step S803, the visual processing device uses parameters according to the command code to configure the visual sub-scheme, such as configuring the parameters of a module in the visual sub-scheme, configuring the parameters of a process in the visual sub-scheme, etc.

[0277] The visual processing device uses parameters according to the command code to execute the current visual sub-program.

[0278] The visual processing device uses parameters according to the command code and can also obtain the execution result of the current visual sub-scheme, and use the communication service to send a reply message to the integration end through the publishing port.

[0279] Step S804, determines whether to close the running process of the visual sub-plan; if so, it means that the visual sub-plan does not need to be executed anymore, and then the visual processing device closes the running process of the visual sub-plan, that is, closes the communication service and file service, and exits the visual sub-plan; if not, it means that the visual sub-plan still needs to be executed, and returns to step S802 to continue receiving command messages sent by the integration end.

[0280] In the embodiment of the present application, each visual processing device can use Figure 8 The process shown accesses the integration end and completes the operation of the visual sub-solution.

[0281] The operation process of the integration end includes:

[0282] Step S901, creating a management identifier: The integration end creates a management identifier as an access identifier of the visual processing device, and uses the management identifier to access the corresponding visual processing device.

[0283] In addition, after the visual processing device is connected, the integrated end will bind the management identifier to the device serial number to facilitate the use of the technical solution provided in the embodiment of the present application in special scenarios where device identification is required.

[0284] Step S902, connect to the device-side communication service and enable the callback thread.

[0285] The integration end creates a communication identifier through the integration interface abstracted by the polymorphic method according to the IP address or PCIe serial number in the device information of the visual processing device; uses the communication identifier to connect the specified port of the specified device and connect the message sending channel to realize message sending. After successfully connecting the message sending channel, the integration end uses the command to obtain the port number of the message reporting channel, uses the port number to create a reporting communication identifier, subscribes to the port of the message reporting channel, and enables the callback thread, and uses the callback thread to poll the port of the message reporting channel to realize message reporting.

[0286] Step S903, connecting to the file service.

[0287] The integration end creates a file transfer identifier according to the IP address or PCIe serial number in the device information of the visual processing device, and uses the file transfer identifier to connect to the file service running in the visual processing device.

[0288] In the embodiment of the present application, the execution order of step S902 and step S903 is not limited.

[0289] Step S904, associating the identifier with the device.

[0290] The integration end connects the sending communication identifier, reporting communication identifier created in step S902, and the file transfer identifier created in step S903 with the management identifier created in step S902, so as to associate the identifier with the device.

[0291] Through the above steps S901 to S904, the integrated end uses the device information to connect to the specified visual processing device.

[0292] After step S901 to step S904, the integration end can send messages to the corresponding visual processing device and obtain reported messages through the unique management identifier, call the file service to exchange files, and preliminarily realize the basis required for device management. Furthermore, the integration end can also abstractly encapsulate concepts such as visual sub-schemes, algorithm groups, and operator modules to support the use of graphical interfaces and secondary development.

[0293] Step S905: create a sub-scheme object.

[0294] In the embodiment of the present application, the integration end encapsulates the visual sub-scheme class. The integration end dynamically creates objects of the visual sub-scheme class, that is, first-class objects (i.e., sub-scheme objects). When the visual system includes multiple visual processing devices, the integration end dynamically creates multiple objects of the visual sub-scheme class.

[0295] Step S906, associating the scheme with the identifier.

[0296] The integration end associates the management identifier with the sub-schema object, calls the integration interface through the management identifier, and interacts with the visual processing device. Among them, the integration interface integrates the communication identifier (such as the above-mentioned communication identifier for sending and the communication identifier for reporting), the file transfer identifier, the callback thread, and the variable management unit. The variable management unit is used to manage the variable data required during the operation of the communication identifier for sending, the communication identifier for reporting, the file transfer identifier, and the callback thread. At this time, the management architecture of the integration end is as follows: Fig.10 As shown, the solution interface of the comprehensive solution includes solution interfaces of multiple visual sub-solutions, and these multiple solution interfaces are used to configure the visual sub-solutions and manage the visual processing devices.

[0297] Step S907, configuring the visual sub-scheme through the object.

[0298] The integration end uses the properties and interfaces of the sub-scheme object to operate the visual processing device. For the algorithm group, operator module, camera module, image data and other components that the visual sub-scheme may contain, the integration end dynamically creates the above encapsulation class objects (such as the second class objects) according to their encapsulation classes and the needs of the visual sub-scheme, and organizes the hierarchical structure of the above encapsulation class objects to form a visual sub-scheme. The visual sub-scheme opens various properties and interfaces to the outside world, hiding the identification and integration interface.

[0299] Through the above steps S901 to S907, the connection and configuration of a single visual processing device are realized. Each visual processing device can work independently. The integration end repeatedly executes the above steps S901 to S907 to obtain sub-scheme objects of multiple visual processing devices, and realize the integration of multiple visual processing devices in the visual system. The integration end can select functions, workstations, etc., and add custom names to these sub-scheme objects to distinguish the visual sub-schemes of different visual processing devices. The integration end can form a key-value pair list with custom names and sub-scheme objects as member variables of the comprehensive scheme class. The interface in the comprehensive scheme supports access to sub-scheme objects with specified names. At the same time, since each sub-scheme object is thread-safe, the integration end can operate the corresponding visual processing devices through each sub-scheme object in parallel.

[0300] Step S908: Send a command message, wherein the command message includes a command code and parameters.

[0301] In the embodiment of the present application, after all visual processing devices are connected to the integration end, the integration end executes step S908, and uses the communication service to send a command message to the visual processing device through the designated port. The visual processing device executes any one or more operations in step S803 according to the command message. Correspondingly, the integration end executes any one or more operations in step S909.

[0302] Step S909, configure the comprehensive plan, execute the comprehensive plan, and receive feedback of the execution result.

[0303] In the embodiment of the present application, the command message instructs the visual processing device to configure the visual sub-scheme. Accordingly, the integration end configures the comprehensive scheme in the comprehensive scheme interface to configure the visual sub-schemes included in the comprehensive scheme.

[0304] The command message instructs the visual processing device to execute the visual sub-scheme. Correspondingly, the integration end instructs the visual processing device to execute the visual sub-scheme in the comprehensive scheme.

[0305] The command message instructs the visual processing device to feedback the execution result. Correspondingly, the integration end receives the feedback execution result.

[0306] Step S910, determines whether to close the running process of the comprehensive plan; if so, it means that the comprehensive plan does not need to be executed anymore, and then the integration end closes the running process of the comprehensive plan, that is, exits the comprehensive plan; if not, it means that the comprehensive plan still needs to be executed, and returns to step S908 to continue sending command messages to the visual processing device.

[0307] Combine the following Figure 11a~Figure 11f The graphical interface provided by the integrated terminal shown illustrates the distributed management method of the visual processing device provided in the embodiment of the present application.

[0308] The integrated end display is as follows Fig.11a The user enters the IP address of the visual processing device in area 111, such as Fig.11a After entering 10.11.1.1 and clicking the OK button, the integration end connects to the corresponding visual processing device based on 10.11.1.1 and the managed device serial number K111111. Fig.11a As shown, the user enters a total of 2 IP addresses, and accordingly, the integrated end is connected to 2 visual processing devices.

[0309] The integrated end display is as follows Fig.11b In the integrated solution management interface shown, the user clicks the "+" button 112 to create a sub-solution object in the integrated solution, such as Fig.11b Shown with Fig.11a The two visual processing devices shown correspond to Solution 1 and Solution 2. The user can edit / configure the sub-solution object by clicking the edit button 113, and the user can delete the sub-solution object by clicking the delete button 114.

[0310] The user clicks on the sub-scheme object (or clicks on the edit button 113) to configure the visual sub-scheme. For example, the integration end configuration is as follows Fig.11c Solution 1 shown in the figure, and configure as follows Fig.11d Scheme 2 shown.

[0311] After the visual sub-scheme is configured, the user can continue to configure the comprehensive scheme. Fig.11e The operation interface 115 shown is used to perform a summary operation on multiple sub-schemes. Fig.11e The modules including foundation, layout and chart shown on the left side of the interface are the specific functional modules provided by the comprehensive solution to realize image display, parameter configuration, operation control and data display. Fig.11e In the interface, the modules such as identification, data source, layout and appearance shown on the right side are the modules for parameter control of the selected functional module. Fig.11e The examples are for illustration only and are not intended to be limiting. Fig.11f It is a schematic diagram of the operation effect of the operation interface 115 after the comprehensive solution is configured.

[0312] After configuring the integrated solution, the user can save it. Later, the user can load the integrated solution by operating the integration terminal. The user does not need to reconfigure the integrated solution, which improves the management efficiency of the visual system and reduces the resource time demand.

[0313] Corresponding to the above-mentioned distributed management method of visual processing equipment, the embodiment of the present application also provides a distributed management device of visual processing equipment, such as Fig.12 As shown, applied to the integrated end, the device includes:

[0314] A connection module 1201, configured to connect to a visual processing device in response to a first operation;

[0315] The management module 1202 is used to manage the visual sub-scheme of the visual processing device in response to the second operation, and the visual processing device executes the visual sub-scheme.

[0316] In some embodiments, the integrated end is connected to a plurality of visual processing devices;

[0317] The management module 1202 can be specifically used to: configure a visual sub-scheme of at least one visual processing device; and send a visual sub-scheme to the visual processing device so that the visual processing device executes the visual sub-scheme.

[0318] In some embodiments, the management module is specifically used to add a visual sub-scheme of a visual processing device to a comprehensive scheme.

[0319] In some embodiments, the integrated end is connected to a plurality of visual processing devices;

[0320] The management module 1202 may be specifically used to add a visual sub-scheme of at least one visual processing device to the comprehensive scheme.

[0321] In some embodiments, the management module 1202 can be specifically used to: delete the visual sub-scheme of the visual processing device.

[0322] In some embodiments, the management module 1202 may also be used to send a deletion instruction to the visual processing device so that the visual processing device deletes the visual sub-scheme.

[0323] In some embodiments, the management module 1202 may be specifically configured to: modify a visual sub-schema of a visual processing device;

[0324] The modified visual sub-scheme is sent to the visual processing device so that the visual processing device executes the modified visual sub-scheme.

[0325] In some embodiments, the management module 1202 can be specifically used to obtain the visual processing results obtained by the visual processing device executing the visual sub-scheme.

[0326] In some embodiments, the management module 1202 may be specifically used to: save a visual sub-scheme of a visual processing device, and / or save a comprehensive scheme, wherein the comprehensive scheme includes a visual sub-scheme.

[0327] In some embodiments, the management module 1202 can also be used to: create a first-class object of a visual sub-scheme of a visual processing device; and manage the visual sub-scheme using the attribute information of the first-class object.

[0328] In some embodiments, the management module 1202 can also be used to: create a management identifier of the visual processing device bound to the first type of object; use the management identifier to connect to the communication service running in the visual processing device, and use the management identifier to connect to the file service running in the visual processing device.

[0329] In some embodiments, the management module 1202 may be specifically configured to:

[0330] According to the device information of the visual processing device, a communication identifier for the visual processing device is created;

[0331] Use the issued communication identifier to establish a message issuing channel with the visual processing device, and bind the issued communication identifier to the management identifier;

[0332] Obtain the port number of the message reporting channel of the visual processing device through the message sending channel;

[0333] According to the port number, create a reporting communication identifier of the visual processing device;

[0334] Use the reporting communication identifier, subscribe to the port of the message reporting channel, and bind the reporting communication identifier with the management identifier.

[0335] In some embodiments, the management module 1202 may be specifically configured to:

[0336] According to the device information of the visual processing device, a communication identifier for the visual processing device is created through the integration interface on the integration end. The integration interface abstracts multiple communication interfaces through a polymorphic method.

[0337] In some embodiments, the management module 1202 may be specifically configured to:

[0338] Using the attribute information of the first type of object, calling the management identifier bound to the first type of object;

[0339] The communication identifier bound to the management identifier is called, and the operation command for the visual sub-scheme is sent to the visual processing device through the message sending channel.

[0340] In some embodiments, the management module 1202 may be specifically configured to:

[0341] Using the attribute information of the first type of object, calling the management identifier bound to the first type of object;

[0342] Call the reporting communication identifier bound to the management identifier, use the callback thread to poll the port of the message reporting channel, and obtain the operation message for the visual sub-scheme reported by the visual processing device.

[0343] In some embodiments, the management module 1202 may be specifically configured to:

[0344] Creating a file transfer identifier for the visual processing device according to device information of the visual processing device;

[0345] Use the file transfer identifier to connect to the file service running in the visual processing device, and bind the file transfer identifier to the management identifier.

[0346] In some embodiments, the management module 1202 may be specifically configured to:

[0347] According to the encapsulation class of the components included in the visual sub-scheme, create the second class object of the components included in the visual sub-scheme;

[0348] According to the hierarchical structure of the components included in the visual sub-scheme, each second-category object is organized to obtain the visual sub-scheme.

[0349] In the technical solution provided by the embodiment of the present application, the visual processing device is connected to the integration end. After joining the visual system, the integration end is responsible for managing the visual sub-solution, and the visual processing device is responsible for running the visual sub-solution. In this way, when the visual system includes multiple visual processing devices, each visual processing device can run its own visual sub-solution, and the integration end does not need to execute the visual sub-solutions of multiple visual processing devices, that is, the integration end does not need to integrate multiple SDKs, and the entire visual system can support the dynamic increase and decrease of visual processing devices, reducing the development resources and time investment required for the development of the visual system, improving the flexibility and scalability of the development method, and solving the problem of hardware resource preemption between multiple machine vision services on the integration end side and the fluctuation of visual system performance.

[0350] In the technical solution of this application, the operations involved in obtaining, storing, using, processing, transmitting, providing and disclosing user personal information are all carried out with the user's authorization.

[0351] Corresponding to the distributed management method of the visual processing device described above, the embodiment of the present application further provides an electronic device, such as the above-mentioned integrated terminal, such as Fig.13 As shown, including:

[0352] Memory 1301, used for storing computer programs;

[0353] The processor 1302 is used to implement any of the above-mentioned methods when executing the program stored in the memory 1301.

[0354] In addition, the above electronic device may further include a communication bus and / or a communication interface, and the processor 1302, the communication interface, and the memory 1301 communicate with each other via the communication bus.

[0355] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0356] The communication interface is used for communication between the above electronic device and other devices.

[0357] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0358] The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0359] In another embodiment provided in the present application, a computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is executed by a processor, the distributed management method of any of the above-mentioned visual processing devices is implemented.

[0360] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the distributed management method of any visual processing device in the above-mentioned embodiments.

[0361] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0362] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0363] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, storage medium and program product embodiments, since they are basically similar to the system and method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the system and method embodiments.

[0364] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A distributed management method for visual processing equipment, characterized in that: Applied to the integration end, the method includes: In response to a first operation, connecting with a visual processing device; In response to a second operation, a visual sub-scheme of the visual processing device is managed, and the visual processing device executes the visual sub-scheme.

2. The method according to claim 1, characterized in that The integrated end is connected to a plurality of visual processing devices; The step of managing the visual sub-scheme of the visual processing device comprises: configuring a vision sub-schema for at least one vision processing device; The visual sub-scheme is sent to the visual processing device so that the visual processing device executes the visual sub-scheme.

3. The method according to claim 1, characterized in that The step of managing the visual sub-scheme of the visual processing device comprises: Add a visual sub-scheme of the visual processing device to the comprehensive scheme.

4. The method according to claim 3, characterized in that The integrated end is connected to a plurality of visual processing devices; The step of adding the visual sub-scheme of the visual processing device to the comprehensive scheme comprises: Add a vision sub-schema for at least one vision processing device to the comprehensive scheme.

5. The method according to claim 1, characterized in that The step of managing the visual sub-scheme of the visual processing device comprises: Delete the vision sub-schema of the vision processing device.

6. The method according to claim 5, characterized in that The method further comprises: A deletion instruction is sent to the visual processing device so that the visual processing device deletes the visual sub-scheme.

7. The method according to claim 1, characterized in that The step of managing the visual sub-scheme of the visual processing device comprises: modifying a visual sub-schema of the visual processing device; The modified visual sub-scheme is sent to the visual processing device so that the visual processing device executes the modified visual sub-scheme.

8. The method according to claim 1, characterized in that The step of managing the visual sub-scheme of the visual processing device comprises: Obtain the visual processing result obtained by the visual processing device executing the visual sub-scheme.

9. The method according to claim 1, characterized in that: The step of managing the visual sub-scheme of the visual processing device comprises: The visual sub-scheme of the visual processing device is saved, and / or the comprehensive scheme is saved, wherein the comprehensive scheme includes the visual sub-scheme.

10. The method according to claim 1, characterized in that The method further comprises: Creating a first class object of a vision sub-schema of the vision processing device; The step of managing the visual sub-scheme of the visual processing device comprises: The visual sub-scheme is managed using the attribute information of the first type of objects.

11. The method according to claim 10, characterized in that The method further comprises: Creating a management identifier of the visual processing device bound to the first type of object; Use the management identifier to connect to the communication service running in the visual processing device, and use the management identifier to connect to the file service running in the visual processing device.

12. The method according to claim 11, characterized in that The step of using the management identifier to connect to the communication service running in the visual processing device includes: Creating a communication identifier for the visual processing device according to the device information of the visual processing device; Using the communication identifier, establishing a message sending channel with the visual processing device, and binding the communication identifier with the management identifier; Obtaining the port number of the message reporting channel of the visual processing device through the message sending channel; According to the port number, creating a reporting communication identifier of the visual processing device; The reporting communication identifier is used to subscribe to the port of the message reporting channel, and the reporting communication identifier is bound to the management identifier.

13. The method according to claim 12, characterized in that The step of creating a communication identifier for the visual processing device according to the device information of the visual processing device includes: According to the device information of the visual processing device, a communication identifier of the visual processing device is created through the integration interface on the integration end, and the integration interface is abstracted from multiple communication interfaces through a polymorphic method.

14. The method according to claim 12 or 13, characterized in that The step of managing the visual sub-scheme by using the attribute information of the first type of object comprises: Using the attribute information of the first type of object, calling the management identifier bound to the first type of object; The communication identifier bound to the management identifier is called, and an operation command for the visual sub-scheme is sent to the visual processing device through the message sending channel.

15. The method according to claim 12 or 13, characterized in that The step of managing the visual sub-scheme by using the attribute information of the first type of object comprises: Using the attribute information of the first type of object, calling the management identifier bound to the first type of object; The reporting communication identifier bound to the management identifier is called, and the port of the message reporting channel is polled using a callback thread to obtain the operation message for the visual sub-scheme reported by the visual processing device.

16. The method according to claim 11, characterized in that The step of using the management identifier to connect to the file service running in the visual processing device includes: Creating a file transfer identifier for the visual processing device according to the device information of the visual processing device; Use the file transfer identifier to connect to the file service running in the visual processing device, and bind the file transfer identifier to the management identifier.

17. The method according to any one of claims 11 to 13, characterized in that: The step of managing the visual sub-scheme by using the attribute information of the first type of object comprises: According to the encapsulation class of the components included in the visual sub-scheme, create a second type of object of the components included in the visual sub-scheme; According to the hierarchical structure of the components included in the visual sub-scheme, each second-category object is organized to obtain a visual sub-scheme.

18. A distributed management system for visual processing devices, characterized in that: It includes an integrated terminal and a visual processing device, wherein the integrated terminal is connected to the visual processing device; The visual processing device is used to execute the visual sub-scheme; The integrated end is used to connect to the visual processing device in response to a first operation; and to manage the visual sub-scheme of the visual processing device in response to a second operation, and the visual processing device executes the visual sub-scheme.

19. The system according to claim 18, characterized in that The integration end is also used to create a first type of object of the visual sub-scheme of the visual processing device; and use the attribute information of the first type of object to manage the visual sub-scheme.

20. The system according to claim 19, characterized in that The visual processing device is also used to run communication services and file services; The integration end is specifically used to create a management identifier of the visual processing device bound to the first type of object; use the management identifier to connect to the communication service, and use the management identifier to connect to the file service.

21. The system according to claim 20, characterized in that The visual processing device is specifically used for: A polymorphic interface is used to enable monitoring of a first port, wherein the polymorphic interface is abstracted from a plurality of communication interfaces by a polymorphic method; an idle second port is obtained, and the polymorphic interface is used to enable publishing of the second port.

22. The system according to any one of claims 19 to 21, characterized in that: A message format is pre-agreed between the visual processing device and the integration terminal.

23. A distributed management device for visual processing equipment, characterized in that: Applied to an integrated terminal, the device comprises: A connection module, configured to connect to a visual processing device in response to a first operation; A management module is used to manage the visual sub-scheme of the visual processing device in response to a second operation, and the visual processing device executes the visual sub-scheme.

24. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-17 when executing a program stored in a memory.

25. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 17 is implemented.

Images