Systems and Edge Devices

Abstract

A system and an edge device are disclosed. The system includes one or more edge devices and an integrated management device configured to manage the one or more edge devices. The edge device includes a photodetector. The integrated management device manages a first training model and a first condition in association with each other. The first condition sets a condition of the photodetector used in generating the first training model. The integrated management device is configured to transmit the first training model and the first condition to the edge device.

Description

Technical Field

[0001] The present invention relates to a system and an edge device. Background Art

[0002] Image recognition and classification processing using machine learning and deep learning is used in various technical fields. U.S. Patent Application Publication No. 2019 / 0278640 discusses a repository service for managing algorithm data (training models) for machine learning in a container and delivering algorithm data.

[0003] In the management system described in U.S. Patent Application Publication No. 2019 / 0278640, the operation of machine learning is facilitated by providing a repository service for machine learning. However, in the case of using one or more edge devices each including a photodetector, the conditions set in each edge device may vary. Considering the transmission of data to the edge device, there is a possibility that the desired reasoning accuracy cannot be achieved due to the changing conditions set in each edge device. Summary of the invention

[0004] One aspect of the present disclosure is to improve the inference accuracy in edge devices. In an embodiment, a system includes: one or more edge devices; and an integrated management device configured to manage the one or more edge devices, wherein the edge device includes a photodetector, wherein the integrated management device manages a first training model and a first condition in association with each other, the first condition sets a condition of the photodetector used when generating the first training model, and wherein the integrated management device is configured to transmit the first training model and the first condition to the edge device.

[0005] In another embodiment, a system includes: two or more edge devices; and an integrated management device configured to manage the two or more edge devices, wherein the edge device includes a photodetector, and wherein the integrated management device manages a first training model and a first condition in association with each other, the first condition setting a condition of the photodetector used when generating the first training model.

[0006] In yet another embodiment, an edge device includes: a photodetector, wherein the edge device is configured to receive a first training model transmitted to the edge device and a condition set when the first training model is generated.

[0007] Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a block diagram illustrating an edge device management system according to a first exemplary embodiment.

[0009] FIG. 2A to FIG. 2F are diagrams illustrating the first exemplary embodiment and a comparative example.

[0010] Figure 3 is a flowchart illustrating the operation according to the first exemplary embodiment.

[0011] Figure 4 An example of the container according to the first exemplary embodiment is illustrated.

[0012] Figure 5 is a block diagram illustrating an edge device management system according to a second exemplary embodiment.

[0013] Figure 6 is a block diagram illustrating an edge device management system according to a third exemplary embodiment.

[0014] Figure 7 is a block diagram illustrating an edge device management system according to a fourth exemplary embodiment.

[0015] Figure 8 is a conceptual diagram illustrating an application according to the fifth exemplary embodiment.

[0016] Fig. 9 is a diagram illustrating an example of a container according to the first exemplary embodiment.

[0017] Fig.10 ] are diagrams illustrating other examples of the container according to the first exemplary embodiment.

[0018] Fig.11 is a flowchart illustrating an example of a process of deployment to an edge device according to the first exemplary embodiment. DETAILED DESCRIPTION

[0019] The exemplary embodiments described below are intended to implement the technical ideas of the present invention, rather than to limit the present invention. The sizes of the components shown in the drawings and the positional relationships therebetween may be exaggerated to make the description clear. In the following description, the same configuration is assigned the same reference numerals, and its repeated description will be omitted.

[0020] In the following description, where the description applies to a similar configuration, a suffix of a reference numeral such as a or b will be omitted.

[0021] (First Exemplary Embodiment)

[0022] Reference Figures 1 to 4 as well as Figures 9 to 11 An edge device management system (hereinafter referred to as a system) according to a first exemplary embodiment is described.

[0023] Figure 1is a diagram illustrating a basic configuration of the system. The system includes one or more edge devices 300 and an integrated management apparatus 200. The edge device 300 and the integrated management apparatus 200 are connected so that a training model managed by the integrated management apparatus 200 can be deployed to the edge device 300. The edge device 300 and the integrated management apparatus 200 can be arranged at physically separate locations as long as data such as a training model can be deployed from the integrated management apparatus 200 to the edge device 300 using wireless communication or wired communication. For example, the edge device 300 may be arranged in a factory, and the integrated management apparatus 200 may be arranged in a management center at a remote location. In addition, one of the edge device 300 and the integrated management apparatus 200 may be in one country, and the other may be in another country.

[0024] The system can be used, for example, as an inspection system. The following will describe a case where the system is an inspection system. In addition to the inspection system, the system according to the exemplary embodiment can also be used as various systems. Examples of various systems include an image recognition system for determining whether specific data exists in image data and an automatic sorting system in a distribution center.

[0025] The following will refer to Figure 1 A system according to the present exemplary embodiment is described.

[0026] (Edge device 300)

[0027] The edge device 300 includes a photodetector 301 and a computer 302. The computer 302 includes at least an input unit, a storage unit, and an output unit. Information on the control of the photodetector 301 is sent from the integrated management apparatus 200 to the input unit of the computer 302.

[0028] For example, an image sensor, a photometric sensor, or a distance measuring sensor may be used as the photo detector 301. A case where the photo detector 301 is an image sensor will be described below.

[0029] The computer 302 controls the photodetector 301. In addition, the computer 302 may store the training model transmitted from the integrated management device 200 in the storage unit. The computer 302 may also store the set conditions and the obtained data when generating the training model in the storage unit. Figure 1In the embodiment of the present invention, the training model and the imaging conditions are managed in a container in the integrated management device 200, and the container is deployed to the computer 302. The computer 302 controls the photodetector 301 based on the information in the container. The container is a virtually constructed "execution environment for a specific application". In the present exemplary embodiment, the container is an execution environment for performing imaging through the photodetector 301 and performing reasoning based on trained data. The introduction of the container enables the execution environment to be constructed quickly and easily, and the load of environmental management can be reduced. Specifically, without performing management using containers as in the present exemplary embodiment, since multiple applications are linked, if a specific application is updated, there is a possibility of failure in other applications. The introduction of containers can reduce the possibility of failure and reduce the load of management. In the following exemplary embodiments, the case of using containers will be described. However, methods other than containers can be used as long as the imaging conditions and training models are managed in association with each other.

[0030] (Integrated Management Device 200)

[0031] The integrated management device 200 controls one or more edge devices 300. Figure 1 In the embodiment, the integrated management apparatus 200 controls two or more edge devices 300. However, the number of edge devices 300 to be controlled may be one. The integrated management apparatus 200 includes at least an edge device environment providing unit 210. Figure 1 In the embodiment, the integrated management device 200 further includes a training model database 220 , an imaging condition database 230 , an integrated management device endpoint 240 and a training execution unit 250 .

[0032] The training model database 220 manages a plurality of training models generated based on a plurality of imaging conditions, such as a first training model generated for a first object (a first workpiece) based on a first imaging condition and a second training model generated for the first workpiece based on a second imaging condition. In other words, the training model database 220 includes a first training model and a second training model generated for the "same workpiece" based on different imaging conditions. Here, it is not necessary that the "same workpiece" is exactly the same workpiece. For example, in the case where the workpiece is product A, multiple products A may correspond to the same workpiece. Specifically, in the case where product A is a red ink cartridge, one red ink cartridge may be used as the workpiece, and other red ink cartridges of the same type may be used as the same workpiece. The training model database 220 may manage a third training model generated for a second object (a second workpiece) based on the first imaging condition. In other words, the training model database 220 may include a plurality of training models generated by imaging the same object based on different imaging conditions, or may include a plurality of training models generated by imaging different objects based on the same imaging condition. In addition, the training model database 220 may include a plurality of training models generated by imaging different objects based on different imaging conditions. In addition, the training model database 220 may include all these types of training models.

[0033] The imaging condition database 230 manages each imaging condition used to generate the training model to be managed by the training model database 220. The integrated management apparatus 200 is configured to transfer the training model and the imaging condition used to generate the training model.

[0034] The edge device environment providing unit 210 transmits the training model and the imaging conditions associated with each other to the edge device 300. Figure 1 In the example, the training model and the imaging condition are managed in association with each other by the container 211, and the container 211 is deployed to the edge device 300. In other words, the container 211 manages the training model and the imaging condition for imaging the training model in association with each other. Figure 1, for containers 211a to 211c, imaging is performed based on the same imaging conditions. The imaging conditions represent, for example, exposure time and gain. Specifically, in each of containers 211a to 211c, an exposure time of 1ms and a gain of twice are managed as imaging conditions. Containers 211a to 211c may manage training models of the same workpiece or may manage training models of different workpieces as will be described in the second exemplary embodiment. The imaging conditions may include various other conditions, such as International Organization for Standardization (ISO) sensitivity settings, F-number settings, presence / absence of high dynamic range, and white balance adjustment. The imaging conditions may be conditions of the photodetector 301 included in the edge device 300 managed by the integrated management device 200, but are not limited thereto. For example, a training model may be generated based on an image obtained by a photodetector included in an edge device that is not managed by the integrated management device 200, and the imaging conditions and training models used in the generation may be managed in the container.

[0035] like Fig. 9 As shown in FIG, the container 211 may include an inference application 308 and a machine learning library 310 in addition to the imaging parameters 307 and the training model 309. When the container 211 is deployed to the computer 302, the edge device environment acquisition unit 311 acquires information such as imaging conditions, firmware versions, and installation information about the edge device 300 via the imaging application 312 and the imaging library 313. Subsequently, the consistency is checked by referring to the imaging parameters 307 in the container 211, and then the container 211 is deployed.

[0036] In this process, even if there is inconsistency, information about the edge device 300 can be Fig.11 The process shown in was rewritten and used.

[0037] Fig.11An example of a deployment process is illustrated. In step S1, the container 211 is deployed to the computer 302 of the edge device 300. In step S2, the integrated management device 200 obtains information about the edge device 300 via the edge device environment acquisition unit 311. Subsequently, in step S3, the integrated management device 200 checks whether there is consistency between the edge device information and the container information. The information includes the installation information, library, and firmware version of the edge device 300. If there is consistency ("yes" in step S3), the operation proceeds to step S6. In step S6, the container 211 is deployed. Then, in step S7, the process ends. If there is no consistency ("no" in step S3), the operation proceeds to step S4. In step S4, it is confirmed by the operator whether to execute the update process for making the edge device 300 consistent with the container 211. If the operator does not agree to execute ("no" in step S4), the operation proceeds to step S7. In step S7, the process ends. If the operator agrees to execute ("Yes" in step S4), the operation proceeds to step S5. In step S5, information about the edge device 300 is updated. The information can be automatically updated using electronic media or communication, or can be updated manually. For example, in the case where the information is to be updated manually, the operator is notified of the manual update so that the operator can issue instructions. When the updating of the information about the edge device 300 is completed, the operation proceeds to step S6. In step S6, the container 211 is deployed to the edge device 300. Then, in step S7, the process ends.

[0038] like Fig.10 As shown, the configuration of the container 211 may be a configuration that does not include the machine learning library 310 in order to be lightweight. In other words, the container 211 includes the inference application 308 and the training model 309, and the machine learning library 310 is outside the container 211. In this case, consistency between the machine learning library 310 and the inference application 308 or the training model 309 is required, and a library parameter 314 is added for this purpose. In terms of size, the library parameter is generally smaller than the machine learning library 310. Therefore, if the container 211 includes the library parameter 314 but does not include the machine learning library 310, the container 211 may be lightweight compared to the case where the machine learning library 310 is included.

[0039] In the case where the container information and the edge device information are not identical and thus there is an inconsistency when the container 211 is deployed to the edge device 300 , similar container information or optimal container information may be selected and delivered.

[0040] When the operator 700 inputs information indicating the workpiece to be inspected by each edge device 300 into the integrated management device endpoint 240 of the integrated management device 200, the information indicating the workpiece is sent from the integrated management device endpoint 240 to the edge device environment providing unit 210. In other words, the operator 700 inputs information for conversion into the integrated management device endpoint 240 of the integrated management device 200. Subsequently, the training model for the target workpiece and the imaging conditions used when generating the training model are sent to the edge device environment providing unit 210 and managed in association with each other in the container 211. Then, the target container 211 is transferred to the edge device 300 corresponding thereto. For example, in the case where the same workpiece is to be inspected by the edge devices 300a, 300b, and 300c, information about the container 211a is input into the computers 302a, 302b, and 302c. Alternatively, the container 211 a may be transferred to the computer 302 a , and containers 211 b and 211 c that manage the same information as that of the container 211 a may be transferred to the computers 302 b and 302 c , respectively.

[0041] Due to the difference between the edge device for generating the training model and the edge device for performing the inspection in terms of setting conditions such as imaging conditions, there is a possibility that the inspection cannot be accurately performed only by inputting the training model corresponding to the workpiece to be inspected into the edge device 300. In the present exemplary embodiment, not only the training model but also the imaging conditions for imaging of the training model are input from the integrated management apparatus 200 to the edge device 300. Therefore, when the training model is transferred to the edge device 300 for inference, imaging can be performed based on the same imaging conditions as those used when the training model was generated, so that the inference accuracy can be improved.

[0042] Reference FIG. 2A to FIG. 2F The effects of the present exemplary embodiment are described. A workpiece has a scratch 100, and a defect of the workpiece is to be inspected. FIG. 2A to FIG. 2F In each figure, the workpiece moves from top to bottom. For example, Figure 2A The results of imaging by the edge device 300a used to generate the training model are illustrated. Figure 2B The result of imaging by the edge device 300 b is illustrated, into which the training model is input but without inputting the imaging conditions. Figure 2C The result of imaging by the edge device 300 c is illustrated, into which the training model is input but without inputting the imaging conditions. Figure 2D , Figure 2E and Figure 2FResults of imaging by edge device 300a, edge device 300b, and edge device 300c are illustrated respectively, and the training model and imaging conditions are input into each of these edge devices. Figure 2A In the embodiment, imaging is performed based on appropriate conditions so that the scratch 100a of the workpiece can be detected. Figure 2B In , the exposure conditions are not suitable, and therefore the imaging results in blur, so that the scratch 100b cannot be accurately detected. Figure 2C In , the gain is not appropriate, and therefore the imaging results in overexposure, so that the scratch 100c cannot be detected. Figures 2D to 2F As shown in each of the figures, in the edge device 300 to which the training model and the imaging conditions are input, imaging is accurately performed so that the scratch 100 can be accurately detected.

[0043] The optimum conditions vary depending on the workpiece. For example, Figure 1 , a case will be described in which the imaging condition used when generating the training model is condition A, and the imaging condition set as the standard condition is condition B. In this case, the edge device 300a has the training model and condition A used when generating the training model, and therefore imaging can be performed based on the optimal condition. Meanwhile, in the edge device 300b, the imaging condition is condition B, and therefore, if only the training model is input, the optimal condition cannot be obtained, so that the inspection accuracy may be reduced compared to the edge device 300a. In contrast, in the present exemplary embodiment, the optimal conditions for inspection of the product and the training model for the workpiece are managed in association with each other, and both are input to the edge device 300. Therefore, the same inspection accuracy as that of the edge device 300a can be achieved in each edge device 300.

[0044] Next, we will refer to Figure 1 and Figure 3 Described is an execution process when the training model trained in the edge device 300 a is used in the other edge devices 300 b and 300 c.

[0045] like Figure 3 As shown, the execution process is divided into training process and deployment process.

[0046] First, the training process will be described. In step S101, the workpiece is imaged based on a first condition (imaging condition) using the photodetector 301a of the edge device 300a. The first condition is, for example, an exposure time of 1ms and a gain of twice. The image thus obtained is sent to the training execution unit 250 of the integrated management device 200.

[0047] In step S102, training is performed using the transmitted image, and a training model is generated. The training model may be generated by machine learning. When an image of a workpiece is input during inspection, the training model makes a determination regarding the presence / absence of a defect and regarding pass / fail, and outputs the result of the determination. In step S103, the training model is stored in the training model database 220. The first condition is also stored in the imaging condition database 230. In this process, the training model database 220 and the imaging condition database 230 are associated with each other in a relational database or the like, and are managed in the edge device environment providing unit 210.

[0048] As a specific algorithm of machine learning, algorithms such as the nearest neighbor algorithm, the naive Bayes algorithm, the decision tree, and the support vector machine can be used. In addition, deep learning that generates feature quantities and coupling weighting factors for training by itself by utilizing a neural network can be used. For example, a convolutional neural network (CNN) model can be used as a model for deep learning.

[0049] In the case where a plurality of training models are to be generated, the above steps S101 to S103 are repeated with different imaging conditions and / or workpieces.

[0050] Next, the deployment process will be described. First, in step S201, in order to deploy the training model of the edge device 300a to other edge devices 300b and 300c, the operator 700 uses a representative state transfer (REST) ​​application programming interface (API) or the like to access the integrated management device endpoint 240 to issue a command. Instead of access by the operator 700, the deployment of the training model of the edge device 300a to other edge devices 300b and 300c can be set by programming. For example, the deployment can be performed every weekend based on a schedule set using a script or the like.

[0051] In step S202, in response to a command from the integrated management device endpoint 240, the edge device environment providing unit 210 imports a set of training models and imaging conditions suitable for the edge device 300 into the container 211. In this process, the training model is reproduced from the training model database 220, and the imaging condition is reproduced from the imaging condition database 230 to be imported into the container 211. For the structure and orchestration of the container 211, a container orchestration system for performing deployment, scaling, and management of containerized applications can be used. As the container orchestration system, any type of structure can be used as long as similar operations can be performed.

[0052] A container for storing a training model and an inference container may be managed as separate containers in association with each other.

[0053] Next, in step S203 , the integrated management apparatus 200 transfers the containers 211 a , 211 b , and 211 c to the edge devices 300 a , 300 b , and 300 c , respectively.

[0054] Subsequently, in step S301, imaging is performed and reasoning is performed in each edge device 300. In the case where a predetermined reasoning accuracy is obtained in step S301, inspection is performed using the workpiece to be inspected. In the case where the predetermined reasoning accuracy is not obtained in step S301, steps S101 to S301 are performed again using different imaging conditions, and steps S101 to S301 are performed until the predetermined reasoning accuracy is achieved.

[0055] The execution process described above enables the training model trained in the edge device 300a to be used in the other edge devices 300b and 300c.

[0056] The situation of managing imaging conditions and training models in an integrated manner in a container is Figure 4 The container shown in is illustrative but not limited to Figure 4 The container shown in . Figure 4 As shown in , the system administrator executes the Deployment.yaml file for managing imaging conditions and training models via the API. Subsequently, the Deployment.yaml file is used and deployed to the edge device. Specifically, in Figure 4 In the present invention, imaging conditions including exposure time, gain, illumination intensity of an illumination device, and angle of an illumination device and a training model are managed in association with each other. Figure 1 The lighting equipment is not shown, but Figure 8 As shown in , the angle and lighting intensity are input into the lighting device included in the edge device. Subsequently, the above-mentioned imaging conditions and training models are input into the edge device, and reasoning is performed for the workpiece.

[0057] In the above description, the container 211 a is also transferred from the integrated management apparatus 200 to the edge device 300 a , but because the edge device 300 a is an edge device used when generating a training model, the transfer of the container 211 a may be omitted.

[0058] An example in which the training model and the imaging conditions are managed in a container and the container is delivered is described, but the training model may be managed in the container and the imaging conditions may be managed separately in association with the training model in the container.

[0059] In the present exemplary embodiment, the training model and the conditions used when generating the training model are managed in association with each other in the integrated management device 200. The training model and the conditions are transmitted to the edge device 300 to perform imaging and reasoning. Thus, the reasoning accuracy in the edge device 300 can be improved. In addition, the dead time before starting the training inspection can be reduced in the inspection of the workpiece.

[0060] (Second exemplary embodiment)

[0061] Reference Figure 5 An edge device management system according to a second exemplary embodiment is described. The system according to the second exemplary embodiment is different from the system according to the first exemplary embodiment in that the plurality of edge devices managed by the integrated management apparatus 200 includes an edge device that inspects a first workpiece and an edge device that inspects a second workpiece different from the first workpiece. Configurations other than the configuration described below are similar to those of the first exemplary embodiment, and thus descriptions thereof may be omitted.

[0062] like Figure 5 As shown in FIG. 1 , the integrated management apparatus 200 manages edge devices 300a and 300c that inspect a first workpiece (product A) and edge devices 300b and 300d that inspect a second workpiece (product B). A container suitable for the workpiece to be imaged by the edge device is delivered to each edge device 300. For example, in Figure 5 In the embodiment, containers 211a and 211c, each managing imaging conditions and training models corresponding to the first workpiece, are delivered to edge devices 300a and 300c. In addition, containers 211b and 211d, each managing imaging conditions and training models corresponding to the second workpiece, are delivered to edge devices 300b and 300d.

[0063] The method of generating a training model and the structure of management of the training model and imaging conditions are similar to those of the first exemplary embodiment and thus will not be described.

[0064] In the present exemplary embodiment, the time until the conversion is completed when the product to be inspected by the edge device is changed can be reduced compared to the case where only the training model is input. For example, there is a case where product A is inspected by the edge device for a predetermined time, and product B is inspected by the same edge device for a predetermined time. The optimal imaging condition varies among products, so if only the training model corresponding to the product to be inspected is input to the edge device, the imaging condition for the edge device takes time to become the optimal imaging condition. Therefore, time is required to complete the conversion each time the product to be inspected changes. In contrast, in the present exemplary embodiment, since the training model and the imaging condition are transmitted to the edge device, the time until the optimal imaging condition is reached can be reduced, so that the reasoning accuracy can be improved. In addition, each edge device can inspect different workpieces.

[0065] (Third Exemplary Embodiment)

[0066] Reference Figure 6 An edge device management system according to a third exemplary embodiment is described. The system according to the third exemplary embodiment is different from the system according to the second exemplary embodiment in that each edge device 300 includes a training execution unit 250. Configurations other than the configuration described below are similar to those of the second exemplary embodiment, and thus descriptions thereof may be omitted.

[0067] In the system according to the third exemplary embodiment, the edge devices 300a, 300b, and 300c include execution units 250a, 250b, and 250c, respectively. The training execution unit 250 included in the edge device 300 generates a training model. Specifically, the training model obtained from the photodetector 301 is input to the training execution unit 250, and the training model is generated in the training execution unit 250.

[0068] Subsequently, the training model generated in each edge device 300 is input to the training model database 220 and the imaging condition database 230 of the integrated management apparatus 200. The training model of each of the plurality of edge devices 300 and the imaging conditions used when generating the training model are collectively managed in the integrated management apparatus 200.

[0069] An example of a process according to the present exemplary embodiment will be described below. First, the operator 700 provides instructions regarding execution / non-execution of training in the training execution unit 250 and update of a training model via the integrated management device endpoint 240 .

[0070] Next, an image obtained by imaging by the photodetector 301 of the edge device 300 is transmitted to the training execution unit 250 , and training is performed.

[0071] The training model that has been trained to a certain extent and has obtained a predetermined accuracy is stored in the training model database 220 of the integrated management device 200. In this process, in the case where there is a training model stored before the training is performed, the training model can be updated to a newly generated training model. The predetermined accuracy can be set appropriately. The predetermined accuracy is, for example, an accuracy of 80% or more in the pass / fail determination in the inspection process.

[0072] At or about the same time as the training model is stored in the training model database 220, the imaging conditions used when generating the stored training model are also stored in the imaging condition database 230. Subsequently, the training model database 220 and the imaging condition database 230 are managed in association with the corresponding training model and the corresponding imaging condition, respectively.

[0073] In the present exemplary embodiment, a set of training models and imaging conditions is managed, and the training models and imaging conditions are delivered to the edge device 300, so that the inference accuracy can be improved. In the present exemplary embodiment, the training execution unit 250 is included in each edge device 300, and retraining is performed in the edge device 300, so that the accuracy of the training model can also be improved.

[0074] (Fourth Exemplary Embodiment)

[0075] Reference Figure 7 An edge device management system according to a fourth exemplary embodiment is described. The system according to the fourth exemplary embodiment differs from the system according to the first exemplary embodiment in the following aspects. In the system according to the fourth exemplary embodiment, data from the photodetector 301 is input from the edge device 300 to the training execution unit 250 via the preprocessing unit 303. In addition, data is input from the training execution unit 250 to the training model database 220 via the post-processing unit 304. In addition, data is input from the preprocessing unit 303b to the training execution unit 250b during the inference process. Configurations other than the configuration described below are similar to those of the second exemplary embodiment, so their descriptions may be omitted.

[0076] exist Figure 7 , the photodetector 301a to the post-processing unit 304a indicates a training process, and the photodetector 301b to the post-processing unit 304b indicates an inference process. The training process and the inference process differ in a configuration of receiving input data from the pre-processing unit 303.

[0077] The pre-processing unit 303 and the post-processing unit 304 may be included in the integrated management apparatus 200 , or may be included in the edge device 300 .

[0078] The pre-processing unit 303 performs trimming, target area identification, inversion and correction of the image data obtained by the photodetector 301. Examples of correction include averaging and correction of brightness and contrast. In addition, edge enhancement can be performed.

[0079] During training in the training execution unit 250 , hyper parameters of an algorithm for controlling machine learning may be additionally managed.

[0080] The post-processing unit 304 may include inspection pass / fail information and production line information.

[0081] In this exemplary embodiment, pre-processing and post-processing can be managed in combination in a system that manages training models and imaging conditions. Therefore, the reasoning accuracy can be further improved, and the reproducibility can be enhanced. In addition, additional training can also be performed during training. In addition, a post-processing unit 304 is added, and this can be utilized in annotation processing. Therefore, the system can be connected to various management systems such as supervisory control and data acquisition systems and manufacturing execution systems.

[0082] (Fifth Exemplary Embodiment)

[0083] Reference Figure 8 An edge device management system according to a fifth exemplary embodiment is described. The system according to the fifth exemplary embodiment is different from the system according to the first exemplary embodiment in that the edge device 300 includes a lighting device 305 and a robot 306, and the integrated management apparatus 200 manages not only imaging conditions but also conditions for movement of units other than the photodetector 301. Configurations other than the configuration described below are similar to those of the first exemplary embodiment, and thus descriptions thereof may be omitted.

[0084] Figure 8 The diagram shows a concept of a case where an edge device management system is used as an inspection application. The management system includes a photodetector 301 , a lighting device 305 , a robot 306 , and a production line 600 .

[0085] The edge device 300 inspects the workpiece flowing on the production line 600. The photodetector 301 of the edge device 300 images the workpiece. The lighting device 305 irradiates the imaging range of the photodetector 301 with light. Based on the image data from the edge device 300, the integrated management device 200 performs pass / fail determination of the workpiece. The robot 306 moves the workpiece determined to have a defect from the production line 600.

[0086] The inspection of the workpiece involves various conditions to be satisfied by the configuration, such as the operating speed of the production line 600, the lighting intensity, angle and color temperature of the lighting device 305, and the movable range, operating angle and angular velocity of the robot 306. At least one of these conditions is managed in the integrated management device 200. When the training model is transferred from the integrated management device 200 to the edge device 300, these conditions are also transferred in addition to the imaging conditions, so that imaging can be performed accurately. In addition, by integrally managing the movable range and operating speed of the robot 306, the dead time can be reduced in the process of inspecting the workpiece.

[0087] The matters described in the first exemplary embodiment to the fifth exemplary embodiment can be appropriately combined.

[0088] For a predetermined period of time, a first exemplary embodiment in which a plurality of edge devices are used to inspect the same workpiece may be adopted, and for other periods of time, a second exemplary embodiment in which each edge device inspects a different workpiece may be adopted. In other words, the edge device does not always inspect the same workpiece, and the workpiece and the conditions to be selected may be appropriately changed depending on the period of time.

[0089] Furthermore, in the first to fifth exemplary embodiments, the photodetector 301 is described as an image sensor, and the integrated management device 200 is described as managing a training model and imaging conditions, but the conditions are not limited to imaging conditions. Figure 7 Conditions such as image parameters in can be managed in association with the training model. In addition, in the case where the photodetector 301 is a distance measuring sensor, conditions such as a pulse interval for measuring the distance can be passed together with the training model.

[0090] In this exemplary embodiment, the inference accuracy in the edge device can be improved.

[0091] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments.The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. A system comprising: one or more edge devices; as well as an integrated management device configured to manage the one or more edge devices, Among them, edge devices include photodetectors, wherein the integrated management device manages a first training model and a first condition in association with each other, the first condition setting an imaging condition of a photodetector used when generating the first training model, the first training model having been trained to inspect a workpiece and providing an inspection result of the workpiece based on an image output from the photodetector having captured an image of the workpiece based on the first condition, and The integrated management device is configured to transmit a first training model and a first condition suitable for an edge device selected from the edge devices to the selected edge device.

2. The system according to claim 1, wherein: The integrated management device also manages the second training model and the second condition for generating the second training model in association with each other.

3. The system according to claim 2, in, The first training model and the second training model are training models based on the same objective, and The first condition and the second condition are different imaging conditions.

4. The system according to claim 2, in, The first training model and the second training model are training models based on different objectives, and Here, the first condition and the second condition are the same imaging condition.

5. The system according to claim 2, in, The first training model and the second training model are training models based on different objectives, and The first condition and the second condition are different imaging conditions.

6. The system according to claim 2, wherein: The first condition and the second condition are imaging conditions.

7. The system according to claim 3, in, The one or more edge devices include a first edge device and a second edge device, wherein the first training model and the first condition are transmitted from the integrated management device to the first edge device, and The second training model and the second condition are transmitted from the integrated management device to the second edge device.

8. The system according to claim 7, in, The photodetector is an image sensor, and The first training model is generated based on an image obtained from an image sensor of the first edge device.

9. The system according to claim 8, in, The integrated management device includes a first container for managing a first training model and a first condition, and The first container is delivered to at least one edge device among the one or more edge devices.

10. The system according to claim 9, wherein: The machine learning library is not included in the first container.

11. The system according to any one of claims 7 to 10, in, The integrated management device includes a training execution unit, and The image obtained from the first edge device is sent to the training execution unit, and the first training model is generated in the training execution unit.

12. The system according to claim 11, in, The integrated management device includes a first database and a second database. The first database manages a plurality of training models including a first training model and a second training model, and The second database manages a plurality of conditions including a first condition and a second condition.

13. The system according to claim 11, in, The training execution unit inputs the training model into the training model database via the post-processing unit, and Among other things, the post-processing unit manages the parameters used for the pass / fail determination.

14. The system of claim 1, wherein: The first training model and the first condition are managed in association with each other in the relational database.

15. The system of claim 1, further comprising a production line, wherein: The first condition includes the operating speed of the production line.

16. The system of claim 15, further comprising a robot, wherein the robot moves the workpiece determined to have a defect by the first training model.

17. A system comprising: two or more edge devices; as well as an integrated management device configured to manage the two or more edge devices, wherein the edge device includes a photodetector, and wherein the integrated management device manages a first training model and a first condition in association with each other, the first condition setting an imaging condition of a photodetector used when generating the first training model, the first training model having been trained to inspect a workpiece and providing an inspection result of the workpiece based on an image output from the photodetector having captured an image of the workpiece based on the first condition.

18. An edge device for use in the system according to claim 1 or claim 17, the edge device comprising: Photodetector, The edge device is configured to receive a first training model transmitted to the edge device and a condition set when generating the first training model.

19. The edge device according to claim 18, in, The photodetector is an image sensor, and Among them, the set conditions are imaging conditions.

20. The edge device according to claim 18 or 19, further comprising a robot, in, Edge devices image workpieces on the production line, and Therein, the robot moves a workpiece that is determined to have a defect by the first training model.

Images