Decomposition information generation method

Abstract

On the basis of a first deep learning model (22a) that learns two classified images of a decomposition target part (10a) and a decomposition non-target part (10b) of an object (10), a plurality of decomposition target parts are identified from a captured image of the object. And a decomposition information generation step in which at least one of the specified plurality of decomposition target parts is deleted as a non-decomposition target part using a first deletion step or a second deletion step, and the specified plurality of decomposition target parts are updated, thereby generating decomposition information. And a first deletion step for deleting, as a non-decomposition-target site, at least one of the plurality of identified decomposition-target sites on the basis of a second deep learning model (22b) that learns the two classified images and that has a lower detection rate of the decomposition-target site than the first deep learning model. And a second deletion step for deleting, on the basis of the plurality of specified decomposition target sites, at least one of the specified decomposition target sites as a non-decomposition target site on the basis of decomposition availability information.

Description

Technical Field

[0001] This disclosure relates to a method for generating decomposition information for use in the automatic decomposition of decomposition objects such as products. Background Technology

[0002] Global-scale environmental problems such as global warming and resource depletion are caused by massive consumption and waste-generating economic activities.

[0003] In this context, Japan implemented the Home Appliance Recycling Law in April 2001, aiming to build a resource-recycling society. Under this law, it is mandatory to recycle used home appliances (air conditioners, televisions, refrigerators, freezers, washing machines, clothes dryers, etc.). As a result, used appliances are manually disassembled in home appliance recycling plants, recycled into various materials. Alternatively, after being shredded into small pieces, they are sorted and recycled according to material type using magnetism, wind, or vibration, thus being reused as recycled materials.

[0004] Currently, manual disassembly is generally carried out for recycling. However, due to high labor costs or labor shortages, research and development are underway for automated disassembly devices that utilize cameras and robots for automatic identification. In the disassembly of used household appliances, the parts to be disassembled can sometimes be partially unidentifiable due to rust, deterioration, or deformation. Therefore, if design information is prepared in advance, determining the parts to be disassembled becomes easier. In this context, an information processing technology similar to that in Patent Document 1 is proposed, in which the product itself stores disassembly information, which is then retrieved at the disassembly plant for automatic disassembly.

[0005] Figure 14 This is a diagram representing the conventional information processing device described in Patent Document 1.

[0006] exist Figure 14 In the dismantling plant 104, a product 103 with dismantling information is prepared. This product 103 is obtained by equipping product 101 with a storage device 102 for storing the dismantling information. Next, in the dismantling plant 104, an information processing device controls an information reading operation performed by a reader that reads the dismantling information from product 103. Then, the dismantling program contained in the dismantling information in storage device 102 is sent to a dismantling robot. Next, the dismantling robot's actions are controlled according to the dismantling program to dismantle product 101 into various materials 105.

[0007] Prior art literature

[0008] Patent documents

[0009] Patent Document 1: JP 2002-346537 Summary of the Invention

[0010] The decomposition information generation method disclosed herein is based on a first deep learning model of two deep learning models that have learned images of decomposed object parts and decomposed non-object parts of an object, to determine multiple decomposed object parts from a photographic image obtained by photographing the object.

[0011] The first deletion step updates the determined plurality of decomposition object parts by deleting at least one of the first and second deletion steps as a non-decomposition object part. The first deletion step deletes at least one of the determined plurality of decomposition object parts as a non-decomposition object part based on the second deep learning model, which has a lower detection rate for the decomposition object parts compared to the first deep learning model. The second deletion step deletes at least one of the determined plurality of decomposition object parts as a non-decomposition object part based on the decomposition feasibility information obtained by the automatic decomposition device performing decomposition based on the determined plurality of decomposition object parts.

[0012] Decomposition information is generated based on the multiple updated decomposition object parts. Attached Figure Description

[0013] Figure 1 This is a structural block diagram of an automatic decomposition apparatus capable of implementing the decomposition information generation method in Embodiment 1 of this disclosure.

[0014] Figure 2 This is a schematic diagram of the automatic disassembly apparatus in Embodiment 1 of this disclosure.

[0015] Figure 3 This is a flowchart of the automatic disassembly operation of the automatic disassembly device in Embodiment 1 of this disclosure.

[0016] Figure 4 This is a structural block diagram of an automatic decomposition device including the decomposition information accumulation unit in Embodiment 1 of this disclosure.

[0017] Figure 5 This is a flowchart of the decomposition information generation method in Embodiment 1 of this disclosure.

[0018] Figure 6 This is a flowchart of automatic decomposition using the decomposition information in Embodiment 1 of this disclosure.

[0019] Figure 7 This is a flowchart of the decomposition information generation method in Embodiment 1 of this disclosure.

[0020] Figure 8This is a structural block diagram of the arithmetic unit of the automatic decomposition device in Embodiment 1 of this disclosure.

[0021] Figure 9 This is a flowchart of the decomposition information generation method in Embodiment 1 of this disclosure.

[0022] Figure 10 This is a flowchart of the decomposition information generation method in Embodiment 1 of this disclosure.

[0023] Figure 11 This is a flowchart of the decomposition information generation method in Embodiment 1 of this disclosure.

[0024] Figure 12 This is a structural block diagram of the arithmetic unit of the automatic decomposition device in Embodiment 2 of this disclosure.

[0025] Figure 13 This is a flowchart of the decomposition information generation method in Embodiment 2 of this disclosure.

[0026] Figure 14 This is a diagram showing the existing disassembly information processing device described in Patent Document 1. Detailed Implementation

[0027] In existing technologies, manufacturers need to pre-define and assign breakdown information to their products. However, in situations like the home appliance cycle where products from various manufacturers coexist, it is difficult to pre-plan and assign breakdown information in a standardized format across manufacturers.

[0028] The present disclosure addresses the aforementioned existing problems by providing a method for generating decomposition information that eliminates the need for manufacturers to pre-assign decomposition information to products, and generates highly accurate decomposition information based on recognition results or actual decomposition performance utilizing deep learning.

[0029] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0030] (Implementation Method 1)

[0031] In the following explanation, in Figures 1 to 7 The basic structure and operation of the automatic decomposition apparatus 1, which can implement the decomposition information generation method in Embodiment 1 of this disclosure, are explained below. Furthermore, in Figures 8-11 The text describes in detail the characteristic parts of the decomposition information generation method in Implementation 1.

[0032] first, Figure 1 This is a block diagram illustrating the basic structure of an automatic decomposition apparatus 1 capable of implementing the decomposition information generation method in Embodiment 1 of this disclosure.

[0033] like Figure 1As shown, the automatic disassembly device 1 includes a camera unit 2, a computing unit 20, a control unit 3, a robotic arm 4, and a disassembly hand 5.

[0034] The camera unit 2 is, for example, a camera that captures images of the object to be analyzed 10 based on instructions from the control unit 3. Examples of the object to be analyzed 10 include household appliances such as air conditioners, refrigerators, televisions, and washing machines.

[0035] The computing unit 20 includes an image processing unit 21, a deep learning model 22, a decomposition object location determination unit 23, and a decomposition information generation unit 50.

[0036] The image processing unit 21 performs image processing on the image data of the image captured by the camera unit 2.

[0037] The deep learning model 22 is generated by learning from images classified into two categories: object parts and non-object parts. The image processing unit 21 uses the deep learning model 22 to extract the object parts from the camera data.

[0038] Here, the term "disassembly target part" refers to the coordinates of the position of a fastening component, such as a screw, used to fix the parts constituting the used product 10 that is the disassembly target, i.e., the position of the screw hole; the coordinates of the part that is attracted or gripped in order to remove the parts that have been released from fixation; the coordinates of the part that should be cut off; and the coordinates of the part that should be released from restraint in order to disassemble the product or other disassembly target 10.

[0039] The decomposition target region determination unit 23 determines the decomposition target region based on the extracted decomposition target region. As will be described in detail later, the decomposition target region determination unit 23 performs selection based on deletion processes and other factors from the decomposition target regions extracted through image processing to ultimately determine the decomposition target region.

[0040] The decomposition information generation unit 50 generates decomposition information based on the determined decomposition object location.

[0041] The so-called decomposition information includes at least the identification information of the decomposition object 10, the location information of the decomposition object part, and the information of the decomposition object components of each decomposition object part. This information is then associated with the product model (in other words, the product group described later).

[0042] The control unit 3 comprehensively controls the camera unit 2, the robot arm 4, and the disassembly hand 5, and performs motion control on the robot arm 4 and the disassembly hand 5 based on the disassembly information generated by the disassembly information generation unit 50 of the computing unit 20.

[0043] The robotic arm 4 has a disassembly hand 5 at its front end. Using the tools installed on the disassembly hand 5, it performs actions such as releasing the fastening of fasteners that fix the components of the used product 10, which are the objects to be disassembled, such as screws; or adsorption or gripping actions to remove the unfastened components; cutting actions of components; and other actions to untie the constraints in order to disassemble the product or other objects.

[0044] use Figure 2 Overview diagram and Figure 3 The basic disassembly operation of the automatic disassembly device 1 in this embodiment 1 will be explained using a flowchart. Here, the steps of the disassembly operation are referred to... Figure 3 Other symbols for reference Figure 2 .

[0045] In step S1, the camera unit 2 takes a picture of the object to be decomposed 10 based on the instructions from the control unit 3 and obtains the image data.

[0046] Next, in step S2, the computation unit 20 determines the decomposition target region based on the acquired camera data through image processing. Here, the computation unit 20 uses a deep learning model 22 to extract the decomposition target region from the camera data using the image processing unit 21, and uses the decomposition target region determination unit 23 to determine the decomposition target region. The deep learning model 22 learns the decomposition target region (e.g., by pre-learning) Figure 2 10a) and decomposition of non-object parts (e.g. Figure 2 The images are generated from the two categories of 10b). The decomposition information generation unit 50 generates decomposition information based on the determined decomposition object parts.

[0047] Here, the term "decomposition of non-object parts" refers to parts that are not part of the object parts and are not the object of the decomposition action.

[0048] Next, in step S3, the control unit 3 sends instructions for the decomposition action to the robot arm 4 and the decomposition hand 5 based on the generated decomposition information.

[0049] Next, in step S4, the robot arm 4 and the disassembly hand 5 disassemble the disassembly target parts of the disassembly target 10.

[0050] The above describes the basic decomposition operation of the automatic decomposition device 1. However, in this embodiment 1, the automatic decomposition operation is effectively utilized to generate and accumulate decomposition information, and the decomposition accuracy of the automatic decomposition is improved by using the generated and accumulated decomposition information. The method is described below.

[0051] Figure 4 This is a block diagram illustrating the structure of the automatic decomposition apparatus 1A that generates and uses the decomposition information in the decomposition information generation method of Embodiment 1. Figure 4 In the middle, to and Figure 1 The same constituent elements use the same reference numerals, and descriptions are omitted.

[0052] exist Figure 4 The automatic decomposition device 1A includes: a product group determination unit 6 for determining product groups with the same product structure among many decomposition objects; a decomposition information generation unit 50 for generating decomposition information; and a decomposition information storage unit 30 for storing information on the parts of the decomposition objects that are associated with the product groups.

[0053] The product group determination unit 6, for example, includes a camera and an image processing unit that processes the image data captured by the camera. It reads the product model number from the product label affixed to the product on the disassembled object 10, or reads the model number based on the shape of the product's appearance, to determine product groups with the same product structure among many disassembled objects. Here, "product group" refers to products with the same product model number, meaning products with the same product structure.

[0054] The decomposition information storage unit 30 stores decomposition information including decomposition object parts that are associated with the product group.

[0055] The automatic decomposition device 1A generates decomposition information using image processing results of automatic decomposition, as follows. The automatic decomposition device 1A performs automatic decomposition based on data obtained by image processing of the captured image using the deep learning model 22 until the decomposition information is generated, without using the decomposition information itself. After generating the decomposition information, the automatic decomposition device 1A also uses the decomposition information to perform automatic decomposition.

[0056] use Figure 5 The flowchart below illustrates the decomposition information generation method in Embodiment 1.

[0057] In step S11, the product group determination unit 6 determines the product group of the object to be disassembled 10. Here, the determination method is not limited to reading the product model from the product label affixed to the product on the object to be disassembled 10, or reading it based on the shape of the product's appearance.

[0058] Next, in step S12, the camera unit 2 takes a picture of the object to be decomposed 10 based on the instructions from the control unit 3.

[0059] Next, in step S13, the arithmetic unit 20 performs image processing on the camera data using the image processing unit 21, determines the decomposition target location using the decomposition target location determination unit 23, and generates decomposition information using the decomposition information generation unit 50.

[0060] Next, in step S14, the control unit 3 sends decomposition action commands to the robot arm 4 and the decomposition hand 5 based on the decomposition information.

[0061] Next, in step S15, the robot arm 4 and the disassembly hand 5 disassemble the disassembly target part of the disassembly target 10.

[0062] Furthermore, following step S13, parallel to step S14, in step S16, decomposition information containing information about the decomposition target location determined in step S13 is accumulated. Therefore, the decomposition information generation unit 50 generates accumulated information in a given data format.

[0063] Next, in step S17, the decomposition information accumulation unit 30 accumulates the decomposition information from the decomposition information generation unit 50 as accumulated information. Regarding the given data format, the format is not limited as long as the automatic decomposition device 1A can operate.

[0064] For the product group that generates and accumulates integrated solution information as described above, the automatic decomposition device 1A is based on... Figure 6 The flowchart is used to implement automatic decomposition of the decomposition information using the decomposition information accumulation unit 30. Figure 6 The flowcharts differ in the following aspects: replacing Figure 5 In step S12, the image data of the object to be decomposed is acquired. In step S18, the decomposition information is read from the decomposition information storage unit 30 using the decomposition information generation unit 50, thereby determining the decomposition object location in step S13. With this configuration, even in products where determining decomposition locations based on image recognition is difficult, automatic decomposition can be performed with high precision. Furthermore, since the decomposition information has already been stored, therefore... Figure 6 The flowchart lacks the generation of decomposition information in step S16 and the accumulation of decomposition information in step S17, which is also consistent with... Figure 5 The flowcharts are different.

[0065] But it is also possible to... Figure 7 As shown in the flowchart, both step S12 (acquiring the image data of the object to be decomposed) and step S18 (reading out the decomposition information) are performed. This configuration avoids the situation where only one piece of decomposition information is accumulated, and no more decomposition information is accumulated. Furthermore, for the same decomposition object location, if steps S12 and S18 are inconsistent, either can be used as an example. As another example, as described later, when merging multiple saved decomposition information candidate files to generate decomposition information, the decomposition information with improved completeness through merging can be used. Alternatively, for example, if there is a screw but no decomposition object location, it will cause undesirable conditions; therefore, the side with the decomposition object location can be used.

[0066] Furthermore, there is no restriction on the order of steps S12 and S18 when performing them. Additionally, it can be combined with... Figure 7Similarly, the decomposition information generation in step S16 and the decomposition information accumulation in step S17 are implemented.

[0067] Here, in generating decomposition information, the data is based on images obtained from photographing the object to be decomposed, 10. Since the object to be decomposed 10 is a used product, there are discrepancies in its appearance, making it sometimes difficult to determine the parts to be decomposed. Figure 5 In step S13, the determination of the decomposed object part is assumed to utilize the deep learning model 22, but sometimes a single processing step cannot accurately determine the part. Therefore, in this embodiment 1, as... Figure 8 As shown, the deep learning model 22 in the computing unit 20 has two types of models: a high-sensitivity deep learning model 22a and a low-sensitivity deep learning model 22b.

[0068] High-sensitivity deep learning model 22a is a deep learning model that, among those learning to classify images into two categories—object parts and non-object parts—extracts more object parts but is prone to misclassifying non-object parts as object parts. Low-sensitivity deep learning model 22b is a deep learning model that, among those learning to classify images into two categories—object parts and non-object parts—has a lower detection rate for object parts than high-sensitivity deep learning model 22a; in other words, it is less likely to misclassify non-object parts as object parts but extracts fewer object parts than high-sensitivity deep learning model 22a. The difference between the two deep learning models 22a and 22b in terms of object part extraction accuracy and the ease of misclassifying non-object parts is shown; the magnitude of the difference is not limited.

[0069] use Figure 9 The flowchart illustrates the decomposition information generation method using the two deep learning models 22a and 22b.

[0070] exist Figure 9 In China, regarding Figure 7 The same steps are followed, using the same reference numerals, and descriptions are omitted. Furthermore, in the following description, the following terms are also used... Figure 4 Block diagram of the automatic decomposition device 1A.

[0071] After the product group is determined in step S11, in step S12, the camera unit 2 takes a picture of the decomposed object 10 based on the instruction from the control unit 3 and obtains the picture data.

[0072] Next, in step S23, the computation unit 20 uses the high-sensitivity deep learning model 22a and the decomposition object location determination unit 23 to perform pre-determination. Specifically, the image processing unit 21 performs image processing based on the acquired camera data using the high-sensitivity deep learning model 22a to extract the decomposition object locations. The decomposition object location determination unit 23 pre-determines the decomposition object locations.

[0073] Next, the computation unit 20 deletes the decomposition target regions. Specifically, using only the periphery of the region predetermined in step S23 as input, in step S25, the image processing unit 21 performs image processing using the low-sensitivity deep learning model 22b to extract the decomposition target regions. The decomposition target region determination unit 23 determines the decomposition target regions. The decomposition target region determination unit 23 compares the decomposition target regions determined by the low-sensitivity deep learning model 22b with the predetermined decomposition target regions. As a first deletion step, the decomposition target region determination unit 23 deletes decomposition target regions that were incorrectly determined by the high-sensitivity deep learning model 22a and regions that were determined by the low-sensitivity deep learning model 22b to be non-decomposition target regions other than decomposition target regions.

[0074] Next, proceed to step S13. In step S13, the decomposition object determination unit 23 updates the decomposition object location to formally perform the determination. The following is related to... Figure 7 The same steps.

[0075] Therefore, as the first deletion step, after the high-sensitivity deep learning model 22a has accurately identified all decomposition target parts and narrowed the exploration scope, it deletes decomposition target parts incorrectly identified by the high-sensitivity deep learning model 22a and parts that are identified by the low-sensitivity deep learning model 22b as non-decomposition target parts other than decomposition target parts. As a result, the decomposition information generation unit 50 can generate decomposition information with higher accuracy.

[0076] Furthermore, as a variation of implementation method 1, it can also be as follows: Figure 10 As shown in the flowchart, the calculation unit 20 performs deletion of the decomposition target parts based on the actual decomposition results. Specifically, during the decomposition operation in step S15, the decomposition information generation unit 50 records whether the decomposition target parts can be decomposed, as the actual decomposition results. In step S30, as the second deletion step, the decomposition information generation unit 50 deletes the decomposition target parts that cannot be decomposed. The decomposition target part determination unit 23 updates the decomposition target parts in step S13 to formally determine whether the decomposition target parts can be decomposed. Regarding whether the decomposition target parts can be decomposed, for example, the difference (change) between the image before and after the robot's decomposition operation is judged. If there is a difference, the decomposition is determined to be successful. Alternatively, a sensor attached to the robot's finger can be used to directly sense the removal of the decomposition target parts after the decomposition operation, and the determination is made accordingly.

[0077] Then proceed to steps S16 and S17.

[0078] Therefore, only the parts of the decomposition object that can be reliably decomposed can be accumulated as decomposition information, and the decomposition information generation unit 50 can generate decomposition information with higher accuracy.

[0079] Furthermore, as other variations of implementation method 1, it can also be as follows: Figure 11 As shown in the flowchart, steps S25 and S30, which are the two types of decomposition target part deletion processes described above, are combined. In this case, the decomposition target part determination unit 23 and... Figure 9 Similarly, the information of the decomposed object part deleted in step S25 is determined as the first decomposed object part in step S33.

[0080] After steps S14 and S15, the decomposition target part determination unit 23 and... Figure 10 Similarly, the information of the decomposed object part deleted in step S30 is identified as the second decomposed object part in step S34.

[0081] Therefore, even if the scope of the target area for the decomposition action in step S15 is narrowed down in the first decomposition target area determination in step S33, and if a part other than the target area is mistakenly identified as a target area in the first decomposition target area determination in step S33, it can be corrected and updated based on the actual decomposition results in the second decomposition target area determination in step S34. After the second decomposition target area is determined, the decomposition information generation unit 50 can generate decomposition information with higher accuracy, which is better.

[0082] (Implementation Method 2)

[0083] Figure 12 This is a block diagram illustrating the automatic disassembly apparatus 1B in Embodiment 2 of this disclosure. (Compared to the diagram illustrating Embodiment 1...) Figure 4 In comparison, the part of the decomposed information storage unit 30 that has a candidate file storage unit 31 is different; the others are the same. Figure 4 Similarly, detailed explanations are omitted.

[0084] In Implementation 1, decomposition information is accumulated based on the data from a single camera shot. However, since the object is a used product, the deformation is significant, and it is also anticipated that the coordinates of the decomposed object parts generated as decomposition information may deviate. To address this, decomposition information from multiple cameras is stored as candidate files in the candidate file storage unit 31. When the number of cameras storing decomposition information in the candidate file storage unit 31 meets a threshold number, automatic decomposition is performed in the next processing object using decomposition information generated by the decomposition information generation unit 50 by merging the multiple candidate files stored in the candidate file storage unit 31.

[0085] In Figure 13 FIG. 2 shows a flowchart of the decomposition information generation method in the second embodiment.

[0086] First, in step S11, a product group is determined.

[0087] Next, in step S40, the number of candidate files NCF of the decomposition information of the corresponding product group is compared with the file merging threshold N. When NCF < N, similar to the first embodiment, in steps S12, S23, S25, S33, S14, S15, S30, and S34, decomposition information is generated based on the captured data. At this time, in the first embodiment, the decomposition information generation unit 50 generates decomposition information in step S16. In the second embodiment, the decomposition information generation unit 50 generates candidate files for decomposition information in step S41 after step S34 and saves them in the candidate file storage unit 31. Here, in the Figure 13 flowchart, steps S40 and later are described based on the Figure 11 flowchart, but it can be based on either the Figure 9 , Figure 10 whichever.

[0088] On the other hand, in step S40, when NCF = N, the decomposition information generation unit 50 generates decomposition information by merging N candidate files in step S42, and reads out the decomposition information generated in step S18, thereby connecting to the subsequent decomposition operation.

[0089] Here, when merging in step S42, the decomposition information generation unit 50 obtains the average value of the decomposition target part. Thereby, the deviation of the coordinates of the decomposition target part of each candidate file is reduced. However, it is not limited to the average value, and the median value or the like can also be taken, and there is no limitation on the merging method.

[0090] In addition, here, after decomposing N products of the same product group, N candidate files are merged, but it is also possible to merge each time one product of the same product group is decomposed and update each time. N is an integer of 2 or more.

[0091] Here, if the file merging threshold N of the candidate files is set to be 2 or more and 15 or less, the deviation of each individual can be reduced, and there is no need to save an excessive number of candidate files, so it is better.

[0092] Furthermore, the decomposition information generation unit 50 can also merge the parts whose position coordinates are closest to those of the decomposition object parts determined during merging and the decomposition object parts in the existing decomposition information candidate files as the same decomposition object parts. In this case, the decomposition information generation unit 50 can also merge multiple decomposition object parts whose position coordinates are closest and located inside a ball with a radius of, for example, a radius of 20 mm or less than twice the width of the head of the decomposition object component such as a screw as the same decomposition object parts. Therefore, since the same decomposition object parts with different position coordinates in multiple candidate files are considered the same, and they are not confused with other nearby decomposition object parts, it is even better.

[0093] According to the decomposition information generation method described in embodiments 1 and 2 above, decomposition information can be generated that reduces the identification error caused by deformation or stains unique to the used product that is the object of decomposition.

[0094] That is, as described above, the decomposition information generation method according to embodiments 1 and 2 can generate highly accurate decomposition information based on the recognition results or actual decomposition performance using deep learning, without requiring the manufacturer to pre-assign decomposition information to the product. For example, it can also generate information on decomposition object parts that are sometimes difficult to determine in recognition based on camera information. As a result, in a recycling factory that processes used products from various manufacturers, information on decomposition object parts that are difficult to determine can be generated in the recycling factory without the manufacturer pre-assigning decomposition information to the product.

[0095] In embodiments 1 and 2, each component that implements each process or function can be constructed by dedicated hardware, or, for components that can be implemented by software, they can be implemented by executing a program. For example, each component can be implemented by reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory through a program execution unit such as a computer's CPU.

[0096] At this point, the computer executing the program can be a single entity or multiple entities. That is, each process or function can be implemented through centralized processing by a single device or a single system, or it can be implemented through distributed processing by multiple devices or multiple systems.

[0097] Furthermore, by appropriately combining any of the various embodiments or variations, their respective effects can be achieved. Moreover, it is possible to combine embodiments with each other, or combinations of examples, or combinations of embodiments and examples, and also to combine features from different embodiments or examples.

[0098] As described above, the decomposition information generation method according to one aspect of this disclosure can generate highly accurate decomposition information based on the recognition results or actual decomposition performance using deep learning, without requiring the manufacturer to pre-assign decomposition information to the product. For example, it can also generate information on decomposition object parts that are sometimes difficult to determine in recognition based on camera information. As a result, for example, in a recycling factory that processes used products from various manufacturers, information on difficult-to-determine decomposition object parts can be generated in the recycling factory without the manufacturer pre-assigning decomposition information to the product.

[0099] Industrial availability

[0100] By using the decomposition information generation method described in this disclosure, in recycling plants that process used products from various manufacturers, such as air conditioners, refrigerators, televisions, washing machines, and other household appliances, information on difficult-to-determine decomposition parts can be created in the recycling plant without manufacturers pre-assigning decomposition information to the products. This allows for more precise processing of used products based on automatic decomposition, saving manpower and promoting more efficient resource utilization of used household appliances.

[0101] Explanation of reference numerals in the attached figures

[0102] 1. Automatic decomposition devices, 1A and 1B

[0103] 2. Camera Department

[0104] 3. Control Department

[0105] 4 robotic arms

[0106] 5. Disassembly Hand

[0107] 10. Decompose the object

[0108] 10a Decomposition of object parts

[0109] 10b Decompose non-object parts

[0110] 20 Computing Department

[0111] 21 Image Processing Department

[0112] 22 Deep Learning Models

[0113] 22a High-sensitivity deep learning model (first deep learning model)

[0114] 22b Low-sensitivity deep learning model (second deep learning model)

[0115] 23 Determination of the part of the decomposed object

[0116] 30. Decomposed Information Accumulation Department

[0117] 31. Candidate Document Storage Department

[0118] 50. Decomposed Information Generation Department

[0119] 101 products

[0120] 102 Storage device

[0121] 103 Goods with dismantling information

[0122] 104 Dismantling Plant

[0123] 105 Materials.

Claims

1. A method for generating decomposed information, Based on the first deep learning model of two deep learning models that have learned the decomposed object parts and the decomposed non-object parts of an object, multiple decomposed object parts are determined from the photographic image obtained by photographing the object. The determined plurality of decomposition object parts are updated by deleting at least one of the determined plurality of decomposition object parts as a non-decomposition object part through a deletion step of at least one of the first deletion step and the second deletion step, wherein, The first deletion step deletes at least one of the determined multiple decomposition object parts as a non-decomposition object part based on the second deep learning model, which has a lower detection rate for the decomposition object part compared to the first deep learning model. The second deletion step deletes at least one of the determined multiple decomposition object parts as a non-decomposition object part based on the decomposition information obtained by the automatic decomposition device based on the decomposition action of the determined multiple decomposition object parts. Decomposition information is generated based on the updated multiple decomposition object parts.

2. The method for generating decomposed information according to claim 1, wherein, The first deep learning model is a high-sensitivity deep learning model that has pre-learned images of two categories: the decomposed object parts and the decomposed non-object parts of the object. It is more accurate than the second deep learning model in identifying the decomposed object parts, but it is prone to misclassifying the decomposed non-object parts as the decomposed object parts. The second deep learning model is a low-sensitivity deep learning model that has pre-learned images of the two categories of the object: the decomposed object parts and the decomposed non-object parts. It has fewer misclassifications than the first deep learning model but is more likely to detect fewer decomposed object parts.

3. The method for generating decomposed information according to claim 1, wherein, The generation of the decomposition information includes: Save the candidate files containing the decomposition information of the identified multiple decomposition object parts. When the number of saved candidate files for decomposition information reaches a threshold, the multiple saved candidate files for decomposition information are merged to generate the decomposition information.

4. The method for generating decomposed information according to claim 3, wherein, The merging of the decomposition information candidate files includes: obtaining the average value of the corresponding multiple decomposition object parts contained in the multiple decomposition information candidate files respectively.

5. The method for generating decomposed information according to claim 3 or 4, wherein, Set the threshold to be above 2 and below 15.

6. The method for generating decomposed information according to claim 3 or 4, wherein, The merging of the decomposition information candidate files includes: determining the decomposition object components within the object, and merging the decomposition object parts located inside a sphere within a radius equal to twice the width of the decomposition object component from among the multiple decomposition object parts included in the multiple decomposition candidate files.

7. The method for generating decomposed information according to claim 5, wherein, The merging of the decomposition information candidate files includes: determining the decomposition object components within the object, and merging the decomposition object parts located inside a sphere within a radius equal to twice the width of the decomposition object component from among the multiple decomposition object parts included in the multiple decomposition candidate files.

Images