Battery detection program and method, and information processing system

Abstract

To enable a battery included in a waste product and a waste to be detected while achieving both accuracy and comprehensiveness.SOLUTION: The battery detection method includes the steps of: (A) detecting an image part at which any of a plurality of predetermined kinds of waste products appears in an input transmission X-ray image obtained by photographing at least either a waste product or a waste; (B) making a first battery detection unit learned in advance so as to detect a battery corresponding to the kind of the waste product appearing at the detected image part and incorporated in the kind of the waste product from a transmission X-ray image of the kind of the waste product detect the battery appearing at the detected image part; and (C) making a second battery unit learned in advance so as to detect a battery from a transmission X-ray image of at least the waste product or the battery itself detect batteries appearing in areas other than the area of the detected image part in the input transmission X-ray image.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a technique for detecting batteries contained in waste products and waste materials. [Background technology]

[0002] In recycling plants, there are 9,000 fires a year caused by batteries mixed in with waste products or waste materials, and most of these fires are caused by batteries being mistakenly put into crushers. Therefore, technology to automatically detect batteries in advance is desired. In this application, primary batteries and secondary batteries are referred to as batteries.

[0003] In response to this, for example, Non-Patent Document 1 discloses a technology in which waste products are photographed with transmitted X-rays and batteries are detected using a deep learning program. In Non-Patent Document 1, only one deep learning program that has been trained with all the training data is used.

[0004] In general, in detection using artificial intelligence (AI), there is a trade-off between accuracy (the proportion of those detected by AI that are truly batteries) and comprehensiveness (the proportion of all batteries that are detected by AI). In order to prioritize safety at recycling plants, it is preferable to increase comprehensiveness and eliminate missed detection of batteries, but doing so will result in a decrease in accuracy and an increase in false positives. Currently, waste products that are determined to contain batteries are manually disassembled to remove the batteries, so there is a problem that low accuracy increases the time required for manual work. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] W. Sterkens, et. al., "Detection and recognition of batteries on X-Ray images of waste electrical and electronic equipment using deep learning", Resources, Conservation and Recycling, Volume 168, May 2021, 105246 Summary of the Invention [Problem to be solved by the invention]

[0006] Therefore, according to one aspect, an object of the present invention is to provide a technique for detecting batteries contained in discarded products or waste with both accuracy and comprehensiveness. [Means for solving the problem]

[0007] A battery detection method according to a first aspect of the present invention includes the steps of (A) detecting an image portion in which any of a predetermined number of types of waste products appears in an input transmission X-ray image of at least one of waste products and waste material; (B) causing a first battery detection unit, which corresponds to the type of waste product appearing in the detected image portion and has been previously trained to detect a battery built into the waste product of that type from a transmission X-ray image of the waste product of that type, to detect the battery appearing in the detected image portion; and (C) causing a second battery detection unit, which has been previously trained to detect a battery from at least a transmission X-ray image of the waste product or the battery itself, to detect a battery appearing in an area of ​​the input transmission X-ray image other than the area of ​​the detected image portion.

[0008] An information processing system according to a second aspect of the present invention includes (A) a means for detecting an image portion in which any of a predetermined number of types of waste products appears in an input transmission X-ray image of at least one of waste products and waste material; (B) a first battery detection unit of a number of the above-mentioned types that has been trained in advance to detect a battery built into any of the above-mentioned number of types of waste products from a transmission X-ray image of that type of waste product; (C) a means for causing the first battery detection unit corresponding to the type of waste product appearing in the detected image portion to detect a battery appearing in the detected image portion; (D) a second battery detection unit that has been trained in advance to detect a battery from a transmission X-ray image of at least waste material or the battery itself; and (E) a means for causing the second battery detection unit to detect a battery appearing in an area of ​​the input transmission X-ray image other than the area of ​​the detected image portion. Effect of the Invention

[0009] According to one aspect, it is possible to detect batteries contained in discarded products or waste materials with both accuracy and comprehensiveness. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an overview of a system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating an example of a functional configuration of the control device. [Diagram 3] FIG. 3 is a diagram showing a process flow executed in the system according to the first embodiment of the present invention. [Figure 4] FIG. 4 is a diagram showing a processing flow of image processing. [Diagram 5] FIG. 5 is a diagram showing an example of a pre-processed transmitted X-ray image. [Figure 6] FIG. 6 is a diagram illustrating an example of a processing result of the object detection unit. [Figure 7] FIG. 7 is a diagram illustrating a processing result of the first battery detection unit. [Figure 8]FIG. 8 is a schematic diagram showing the result of masking the image area where discarded products are detected and detecting batteries in the other areas. [Figure 9] FIG. 9 is a diagram showing a processing flow of image processing. [Figure 10] FIG. 10 is a diagram showing the verification results of the embodiment of the present invention. [Figure 11] FIG. 11 is a diagram illustrating an example of a functional configuration of a control device according to the second embodiment. As illustrated in FIG. [Figure 12] FIG. 12 is a diagram illustrating a process flow of image processing according to the second embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of a functional configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] [Embodiment 1] Figure 1 shows an example of the configuration of a system for sorting discarded products containing batteries and the batteries themselves.

[0012] The system according to the present embodiment includes a radiographic X-ray apparatus 100, a NAS (Network Attached Storage) 200, a control device 300, a sorting device 400, and a belt conveyor (not shown). In Fig. 1, solid arrows indicate the movement of objects, and dotted arrows indicate the transmission of data.

[0013] Waste products to be determined whether they contain batteries (for example, products in which a battery is integrated with components other than the battery) and waste products in which a single battery is mixed with metal, resin, glass, etc. are placed on a belt conveyor and move, and when passing through the transmission X-ray imaging device 100, a transmission X-ray image is taken by the transmission X-ray imaging device 100. The taken transmission X-ray images are stored in the NAS 200 in sequence, and the control device 300 reads the transmission X-ray images from the NAS 200 in sequence and executes the process described below to determine whether a battery is contained in the waste product or waste. The control device 300 issues an instruction to the sorting device 400 based on the battery detection result, and the sorting device 400 discharges waste products and waste products containing batteries to the side with batteries, for example, for battery removal work by workers, in response to the instruction from the control device 300, and discharges waste products and waste products not containing batteries to the side without batteries, for example, for normal waste disposal processing.

[0014] Next, an example of the functional configuration of the control device 300 that executes the main processing in this embodiment will be described with reference to FIG.

[0015] The control device 300 includes an image acquisition unit 310, a pre-processing unit 320, a first data storage unit 330, an object detection unit 340, a second data storage unit 350, a battery detection control unit 360, a plurality of first battery detection units 370, a second battery detection unit 380, and an instruction output unit 390.

[0016] The image acquisition unit 310 acquires transmitted X-ray images from the NAS 200, for example, periodically, and outputs them to the pre-processing unit 320. The pre-processing unit 320 performs pre-processing such as resizing the transmitted X-ray images and inverting the grayscale of the images for the following processing, and stores the images in the first data storage unit 330. Note that the inversion of the grayscale is performed to improve the accuracy of battery detection in the subsequent processing.

[0017] The object detection unit 340 executes a process of detecting an image portion in which any of a plurality of predetermined types of waste products appears in the preprocessed transmitted X-ray image stored in the first data storage unit 330. For example, 13 items, namely, a handy cleaner, a digital camera, an electronic cigarette, an electric toothbrush, a feature phone, a mobile battery, a portable DVD (Digital Versatile Disc) player, a portable speaker, a robot vacuum cleaner, a shaver, a smartphone, a tablet, and a charging box for a wireless earphone, are adopted as the types of waste products to be detected. The object detection unit 340 is, for example, a single shot detector (SSD: Single Shot multibox Detector). When used, the single shot detector identifies the positions of a plurality of objects in an image, surrounds them with a rectangle (i.e., a bounding box), and further identifies the object for each rectangle. For details, see Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, CY, & Berg, AC (2016, October). SSD: Single shot multibox detector. In European conference on computer vision (pp. 21-37). Springer, Cham., etc. In this embodiment, an SSD is used that is trained using training data in which a rectangle surrounding each waste product sample and its product name are added in a transmission X-ray image taken of the above-mentioned 13 waste product samples by the transmission X-ray photography device 100. Note that there are cases where the 13 waste products can be detected by adjusting the weight or performing transfer learning on an existing SSD. The processing result of the object detection unit 340 is stored in the second data storage unit 350.

[0018] The battery detection control unit 360 causes the first battery detection unit 370 and the second battery detection unit 380 to execute processing for battery detection using data stored in the first data storage unit 330 and the second data storage unit 350. The battery detection control unit 360 stores data in the middle of processing in the second data storage unit 350 and outputs the battery detection result to the instruction output unit 390. The instruction output unit 390 outputs control data to the sorting device 400 based on the battery detection result so that the waste products with batteries are discharged to the side with batteries and the waste products without batteries are discharged to the side without batteries. For example, the data converts the coordinates in the transmitted X-ray image into coordinates on the belt conveyor, and discharges the waste products located at the coordinates to the side with batteries if the waste products have batteries, and to the side without batteries if the waste products do not have batteries.

[0019] Each of the first battery detection units 370 specializes in a different type of waste product, and detects the battery built into the waste product in the transmission X-ray image of the waste product. When the above-mentioned 13 types of waste products are handled, 13 first battery detection units 370 are used. The first battery detection units 370 are also, for example, SSDs, and are SSDs that have learned using teacher data in which a rectangle surrounding a built-in battery is added in a transmission X-ray image of a waste product sample of the type in which the first battery detection unit 370 specializes, which is taken by the transmission X-ray imaging device 100. Note that there are cases where the built-in battery can be detected by adjusting the weight or performing transfer learning on an existing SSD. The processing results of each first battery detection unit 370 are output to the battery detection control unit 360.

[0020] The second battery detection unit 380 is for detecting batteries overlooked by the first battery detection unit 370, and is not a battery built into a specific type of waste product like the first battery detection unit 370, but is a general-purpose detection unit that detects batteries that appear in a transmission X-ray image. The second battery detection unit 380 is also, for example, an SSD, and is an SSD that has been trained using teacher data in which a rectangle surrounding a battery is added in a transmission X-ray image in which the above-mentioned 13 waste product samples, a single battery, and waste in which a single battery is mixed with metals and the like appear. Note that there are cases in which a battery can be detected by adjusting the weight or performing transfer learning on an existing SSD. Although the SSD is trained to detect built-in batteries for the 13 waste product samples, this is to enable detection of batteries built into waste products other than the 13 items, and if waste product samples other than the 13 items can be obtained, teacher data in which a rectangle surrounding a battery is added in a transmission X-ray image in which the waste product appears may be prepared and trained. The processing result of the second battery detection unit 380 is output to the battery detection control unit 360.

[0021] Next, the processing contents of the system shown in Fig. 1 will be described with reference to Fig. 3 to Fig. 9. The processing of Fig. 3 is performed, for example, every time waste products or the like are moved on a belt conveyer by the range that can be photographed at one time by the transmission X-ray imaging device 100.

[0022] First, the transmission X-ray imaging device 100 takes a transmission X-ray image of a waste product or waste material and stores it in the NAS 200 (step S1). For example, every time a new transmission X-ray image is stored in the NAS 200, the control device 300 reads the transmission X-ray image and executes image processing, which will be described in detail below (step S3). Then, the sorting device 400 sorts waste products and the like according to control data based on the battery detection result in the control device 300 (step S5). As described above, if there is a battery, the product is discharged to the side with a battery, and if there is no battery, the product is discharged to the side without a battery.

[0023] Next, image processing will be described with reference to FIGS.

[0024] First, the image acquisition unit 310 reads out a transmitted X-ray image from the NAS 200, and the preprocessing unit 320 performs preprocessing on the transmitted X-ray image, such as resizing to a predetermined size and inverting black and white, and stores the processing result in the first data storage unit 330 (FIG. 4: step S11). For example, it is assumed that a preprocessed transmitted X-ray image as shown in FIG. 5 is obtained. In this example, the rectangle on the left is an example of a portable DVD player, the rectangle on the top right is an example of waste, and the rectangle on the bottom right is an example of an electric toothbrush. The portable DVD player and the electric toothbrush are included in the 13 items mentioned above. It is assumed that the waste contains a battery removed from a smartphone.

[0025] Next, the object detection unit 340 executes an object detection process to detect an image portion in which any of the 13 types of waste products appears in the pre-processed transmitted X-ray image, and stores the process result in the second data storage unit 350 (step S13). As described above, the portable DVD player and the electric toothbrush are included in the 13 items, so as shown in FIG. 6, an image portion A in which the portable DVD player appears and an image portion B in which the electric toothbrush appears are detected. For the bounding box surrounding the detected image portion, data on the reference position (for example, the coordinates of the upper left vertex) and the height and width of the rectangle are obtained, and the name of the product is also specified. More specifically, the product name and its image portion with a reliability score equal to or higher than a threshold are specified.

[0026] Then, the battery detection control unit 360 refers to the processing result of the object detection stored in the second data storage unit 350 and determines whether or not a predetermined discarded product (here, 13 types of discarded products) has been detected (step S14). If none of the predetermined discarded products has been detected, the process proceeds to the process of FIG. 9 via terminal A.

[0027] On the other hand, if at least one predetermined discarded product is detected, the battery detection control unit 360 sets the number of detected discarded products to N (step S15) and initializes a counter n to 1 (step S17). In the example of FIG. 6, N=2.

[0028] Then, the battery detection control unit 360 cuts out the image portion where the n-th discarded product appears, and resizes it to a predetermined size (step S19). Then, the battery detection control unit 360 outputs the resized image portion to the first battery detection unit 370 corresponding to the type of the n-th discarded product (the product name in the above example), and the first battery detection unit 370 specialized for the type of the n-th discarded product executes battery detection processing for the resized image portion (step S21). For example, in the example shown in FIG. 6, as shown in FIG. 7, for the image portion A, the first battery detection unit 370 specialized for the portable DVD player outputs a bounding box a surrounding the area where the battery appears, and for the image portion B, the first battery detection unit 370 specialized for the electric toothbrush outputs a bounding box b surrounding the area where the battery appears. Note that in order to show the transition of processing, areas other than the image portions A and B are also shown in FIG. 7, but in reality, a bounding box is shown for each of the image portions A and B. 7 shows an example in which bounding boxes a and b are displayed, but since there are cases in which the battery has already been removed, it may be output that there is no area in which the battery is present. In this embodiment, it is not necessary to specify the area in which the battery is present, so it is also possible to output only the presence or absence of a battery.

[0029] When the battery detection control unit 360 receives the output from the first battery detection unit 370 , it stores it in the second data storage unit 350 .

[0030] Then, the battery detection control unit 360 determines whether or not n=N (step S23). If n has not reached N, the battery detection control unit 360 increments n by 1 (step S25), and the process returns to step S19.

[0031] On the other hand, if n=N, the battery detection control unit 360 executes a process of painting the detected image portion in the pre-processed transmitted X-ray image with a predetermined background color to mask it (step S27). In step S21, for a plurality of predetermined types of discarded products, a highly accurate battery detection process is executed by the first battery detection unit 370 specialized for them, so that the image portion detected in step S13 is painted and masked with the background color to avoid erroneous detection by the second battery detection unit 380. In the example of FIG. 6, as shown in FIG. 8, the areas of the image portions A and B are masked with the background color. As a result, the image portions A and B are substantially excluded from the processing target area of ​​the second battery detection unit 380.

[0032] 9 through terminal A, the battery detection control unit 360 outputs the pre-processed transmitted X-ray image to the second battery detection unit 380 when the transmitted X-ray image or waste product after the processing of step S27 is not detected, and the second battery detection unit 380 executes the battery detection process for the entire input transmitted X-ray image (step S29). As described above, the second battery detection unit 380 executes the battery detection process to detect batteries that could not be detected by the multiple first battery detection units 370, and in particular detects individual batteries mixed in waste and batteries built into waste products other than the 13 types of waste products. This ensures comprehensiveness.

[0033] As shown in FIG. 8, a bounding box c enclosing the area in the upper right corner where the battery mixed in the waste appears is output. For example, data on the reference position (for example, the coordinates of the upper left vertex) and the height and width of the rectangle are output. If there is no area where a battery appears, a message indicating that there is no area where a battery appears is output. When the battery detection control unit 360 receives the output from the second battery detection unit 380, it stores the output in the second data storage unit 350.

[0034] Then, the battery detection control unit 360 outputs the battery detection results of the first battery detection unit 370 and the second battery detection unit 380 to the instruction output unit 390. The instruction output unit 390 generates the above-mentioned control data based on the battery detection results and outputs it to the sorting device 400 (step S31). Then, the process returns to the calling process. As described above, for example, the coordinates of the image part where the waste product appears or the area where the battery appears in the transmitted X-ray image are converted to coordinates on the belt conveyor, and the waste product, etc. located at the coordinates is discharged to the side with the battery if the waste product, etc. has a battery, and is discharged to the side without the battery if the waste product, etc. does not have a battery, and the data is generated and output.

[0035] This makes it possible to improve the accuracy of battery detection by the first battery detection section 370, while at the same time enabling the second battery detection section 380 to perform battery detection that also ensures comprehensiveness.

[0036] [Modification A of the First Embodiment] Although an example has been shown above in which a first battery detection unit 370 is provided for each of 13 types of discarded products, the embodiment of the present invention is not limited to those that correspond to multiple types of discarded products determined by classification based on such product names.

[0037] For example, features of a transmission X-ray image of a waste product to be handled may be extracted by some method, and the first battery detection unit 370 may be provided for each of a plurality of types of waste products determined by classification based on the features.

[0038] When classification based on product names is used, the number of first battery detection units 370 increases as the number of items to be handled increases, which causes a problem that it becomes difficult to implement the first battery detection units 370 in a computer. On the other hand, the characteristics in a transmitted X-ray image may be similar even for waste products with different product names, and there may be cases where there is no problem in terms of battery detection accuracy even if multiple types of waste products with different product names are handled as one type.

[0039] That is, a feature extraction process may be performed on the waste products that appear in the transmitted X-ray image, and the waste products may be clustered using multiple types of features obtained by the feature extraction process with a number of clusters within a range in which accuracy can be maintained, and a first battery detection unit 370 may be provided for each cluster.

[0040] In such a case, the object detection unit 340 specifies which cluster the object belongs to when detecting the object.

[0041] In addition, in the clustering, a set of waste products having a specific characteristic in a transmission X-ray image may be recognized as one cluster. In other words, for example, one type of waste product (or several types of waste products in some cases) among multiple types of waste products may be regarded as a set of waste products having a specific characteristic.

[0042] Furthermore, since items with common characteristics in the transmission X-ray image are classified into the same cluster, items classified into a certain cluster may not only be waste products but also waste with the same characteristics. For example, if the whole or part of the waste other than the image parts A and B in Fig. 6 has a predetermined characteristic, it may be specified as the image part C. In other words, multiple types of waste products may be multiple types of waste items determined by clustering.

[0043] [Variation B of the First Embodiment] Although an example in which SSDs are used has been shown for the object detection unit 340, the first battery detection unit 370, and the second battery detection unit 380, it is needless to say that this is not limited to SSDs. Other object detection models may also be used.

[0044] For example, well-known models such as R-CNN (Region-Convolutional Neural Network), YOLO (You Only Live Once), DCN (Deformed Convolutional Networks), DETR (End-to-End Object Detection with Transformers), and HOG (Histogram of oriented gradient) may be used. Furthermore, various derived models of these may be used.

[0045] Also, different models may be adopted for the object detection unit 340, the first battery detection unit 370, and the second battery detection unit 380, respectively.

[0046] [Verification results of the first embodiment] Battery detection was performed on 150 other transmitted X-ray images using an example in which the above-described embodiment was implemented and a comparison program which is a second battery detection unit 380 obtained by creating and learning teacher data using the same number and type of transmitted X-ray images as those used for learning in constructing the example.

[0047] The detection results were evaluated using three indices: Precision, which relates to the number of false positives, Recall, which relates to the number of missed detections, and Accuracy, which is a comprehensive evaluation index. Each index takes a value between 0 and 1, with higher values ​​indicating higher performance. Precision=TP / (TP+FP) Recall = TP / (TP+FN) Accuracy=(TP+TN) / (TP+TN+FP+FN) Here, TP indicates the number of true positives (correct detections), FP indicates the number of false positives (false positives), FN indicates the number of false negatives (failed detections), and TN indicates the number of true negatives (correctly missed detections).

[0048] For the verification, two cases were set in which the threshold (Tf) of the reliability score for battery detection was set to a high value (0.6) and a low value (0.001). The verification results are shown in FIG. 10. Under both conditions, the evaluation index of the embodiment was almost equal to or higher than that of the comparison program, and the effectiveness of the embodiment was confirmed. In particular, at the high threshold (0.6), the comparison program had a high Precision but a low Recall. On the other hand, at the low threshold (0.001), the comparison program had a high Recall and a very low Precision. This shows the trade-off relationship between "few false positives" and "few missed detections" in detection using general deep learning. On the other hand, in the embodiment, it was confirmed that by adopting a high threshold (0.6), it was possible to achieve both Precision and Recall at a high level. The trade-off relationship between "few false positives" and "few missed detections" was overcome, making it possible to achieve both at a high level. Moreover, when comparing Accuracy, which is a comprehensive evaluation index, the working examples far surpass the comparative programs at both thresholds.

[0049] From the above, it can be seen that the above-described embodiment is highly effective in applications such as battery detection in discarded products.

[0050] [Embodiment 2] In the first embodiment, a configuration adapted to one specific situation is shown, but it is preferable in terms of accuracy to change the learning contents of the object detection unit 340, the first battery detection unit 370, the second battery detection unit 380, etc. in the control device 300 depending on, for example, the type of X-ray radiography device used, the type of belt conveyor on which waste products, etc. are carried, etc. In addition, the thickness (also called layer thickness) of the waste, etc. carried on the belt conveyor, the main components of the waste, etc. (for example, non-burnable garbage, plastic garbage, etc.), etc. may change over time.

[0051] In order to appropriately handle each of these cases, in this embodiment, the configuration of the control device 300 is changed to control device 3000 as shown in Fig. 11. Note that functions with the same reference symbols as those shown in Fig. 2 will be described as having the same functions.

[0052] Specifically, the control device 3000 includes an image acquisition unit 310, a preprocessing unit 320, a classification unit 325, a category A processing unit 4000A, a category B processing unit 4000B, and a category C processing unit 4000C. Here, the category represents a classification specified by a combination of, for example, the type of transmission X-ray imaging device, the width of the belt conveyor, the main components of the waste, the layer thickness of the waste, and the like, and specifically, the characteristics appear in the transmission X-ray image.

[0053] The categories are as follows: Category A: X-ray equipment α from X company, with a conveyor belt width of a mm, waste mainly non-burnable garbage, and a waste layer thickness of approximately b mm. Category B: Y company's X-ray equipment β, the width of the conveyor belt is cmm, the waste is mainly plastic waste, and the thickness of the waste layer is approximately dmm. Category C: Z company's X-ray equipment γ, belt conveyor width fmm, waste mainly a mixture of plastic waste and non-burnable waste, waste layer thickness is assumed to be approximately gmm.

[0054] That is, the classification unit 325 is a trained model that is trained in advance with teacher data whose categories are known in advance so that it can determine which category the output (preprocessed transmitted X-ray image) of the preprocessing unit 320 belongs to. The classification unit 325 outputs the preprocessed transmitted X-ray image to a processing unit (such as the category A processing unit 4000A, category B processing unit 4000B, or category C processing unit 4000C in FIG. 11) corresponding to the category to which the preprocessed transmitted X-ray image belongs.

[0055] The processing unit of each category is configured to appropriately process the preprocessed transmitted X-ray images belonging to the category, and includes a first data storage unit 330, an object detection unit 340, a second data storage unit 350, a battery detection control unit 360, an instruction output unit 390, a first battery detection unit 370, and a second battery detection unit 380. As described above, the object detection unit 340, the first battery detection unit 370, and the second battery detection unit 380 have learned about the preprocessed transmitted X-ray images belonging to the category.

[0056] Next, the difference between the processing flow of the control device 3000 shown in FIG. 11 and FIG. 4 will be described with reference to FIG.

[0057] 12, step S111 is added between step S11 and step S13, and the classification unit 325 executes classification processing to determine which category the preprocessed transmission X-ray image belongs to (step S111), and outputs the preprocessed transmission X-ray image to a processing unit corresponding to the determined category. The following processing is performed in the processing unit corresponding to the determined category.

[0058] In this way, battery detection can be performed appropriately for each category in terms of accuracy and the like.

[0059] The number of categories is an integer equal to or greater than 2, and may be added later. In that case, the classification unit 325 is trained so as to be able to handle the categorization after the addition.

[0060] Although the embodiment of the present invention has been described above, the present invention is not limited thereto. For example, the functional configuration examples of the control device 300 shown in FIG. 2 and the control device 3000 shown in FIG. 11 are only examples, and may differ from the configuration examples of the programs. In addition, the processing flows shown in FIG. 3, FIG. 9, and FIG. 11 are also only examples, and the processing order may be changed or multiple steps may be executed simultaneously as long as the same processing results can be obtained. For example, when multiple types of waste products appear in a transmitted X-ray image, different first battery detection units 370 may be operated in parallel. In addition, the modified example of the first embodiment can also be applied to the second embodiment.

[0061] The control device 300 or 3000 described above is, for example, a computer device as shown in FIG. 13, in which a memory 2501, a CPU 2503, a hard disk drive (HDD) 2505, a display control unit 2507 connected to a display device 2509, a drive device 2513 for a removable disk 2511, an input device 2515, and a communication control unit 2517 for connecting to a network are connected by a bus 2519. An operating system (OS) and an application program for carrying out the processing in this embodiment are stored in the HDD 2505, and are read from the HDD 2505 to the memory 2501 when executed by the CPU 2503. The CPU 2503 controls the display control unit 2507, the communication control unit 2517, and the drive device 2513 according to the processing contents of the application program to perform a predetermined operation. Data during processing is mainly stored in the memory 2501, but may be stored in the HDD 2505. In an embodiment of the present invention, an application program for carrying out the above-mentioned processes is distributed by being stored on a computer-readable removable disk 2511, and is installed on the HDD 2505 from the drive device 2513. It may also be installed on the HDD 2505 via a network such as the Internet and a communication control unit 2517. Such a computer device realizes the various functions described above by organic cooperation between hardware such as the above-mentioned CPU 2503 and memory 2501 and programs such as the OS and application programs.

[0062] Furthermore, the object detection unit 340, the first battery detection unit 370, and the second battery detection unit 380 may be implemented using a plurality of GPUs (Graphics Processing Units) instead of the CPU 2503.

[0063] Furthermore, instead of implementing all the functions of the control device 300 or 3000 in one computer device, the functions may be shared among multiple computer devices. For example, the object detection unit 340, the first battery detection unit 370, and the second battery detection unit 380 may be implemented in different computers connected to a network. In addition, they may be implemented on the cloud, and only the parts that are utilized may be implemented in the control device 300 at the recycle plant. Note that whether they are implemented in one computer device or in multiple computer devices, the whole is referred to as an information processing system.

[0064] The above-described embodiment can be summarized as follows.

[0065] A battery detection method according to a first aspect of an embodiment of the present invention includes the steps of (A) detecting an image portion in which any of a predetermined number of types of waste products appears in an input transmission X-ray image (e.g., a transmission X-ray image after preprocessing) of at least one of a waste product and waste material, (B) causing a first battery detection unit, which corresponds to the type of waste product appearing in the detected image portion and has been trained in advance to detect a battery built into the waste product of that type from the transmission X-ray image of the waste product of that type, to detect the battery appearing in the detected image portion, and (C) causing a second battery detection unit, which has been trained in advance to detect a battery from at least a transmission X-ray image of the waste product or the battery itself, to detect a battery appearing in an area other than the area of ​​the detected image portion in the input transmission X-ray image.

[0066] By ensuring accuracy with the first battery detection unit and comprehensiveness with the second battery detection unit, it becomes possible to detect batteries contained in discarded products or waste materials while achieving both accuracy and comprehensiveness.

[0067] In addition, the first battery detection unit described above may detect the battery by learning (e.g., by machine learning) transmitted X-ray images of the waste products of the type it is responsible for, in which the area of ​​the battery built into the waste products of that type is specified.

[0068] The second battery detection unit may detect the battery by learning a transmission X-ray image of the waste or the battery itself, in which the battery area is specified, and a transmission X-ray image of each of the above-mentioned multiple types of waste products, in which the battery area built in the waste product is specified, or may detect the battery by learning a transmission X-ray image of the waste or the battery itself, in which the battery area is specified, and a transmission X-ray image of a waste product other than the above-mentioned multiple types, in which the battery area built in the waste product is specified. This makes it possible to detect batteries contained in waste that cannot be detected by the first battery detection unit, and batteries built in waste products other than the predetermined multiple types. Note that both the transmission X-ray image of each of the above-mentioned multiple types of waste products, in which the battery area built in the waste product is specified, and the transmission X-ray image of a waste product other than the above-mentioned multiple types, in which the battery area built in the waste product is specified, may be used for learning.

[0069] Furthermore, the detected image portion may be masked with a specific color in the input transmission X-ray image to allow the second battery detection unit to detect the battery in an area other than the detected image portion, thereby making it possible to suppress erroneous detection by the second battery detection unit.

[0070] Furthermore, the step of detecting the image portion described above may be performed by an object detection unit that learns the transmission X-ray image in which the area of ​​the waste product of each of the multiple types is specified, and detects the type of waste product appearing in the input transmission X-ray image and the image portion that is the area in which the waste product appears. The object detection unit can be implemented using various techniques for detecting objects appearing in an image, and may be implemented using such deep learning techniques.

[0071] In addition, the above-mentioned multiple types may be classified based on the name of the waste product, or based on the characteristics of the waste product in a radiographic image. If the number of items increases, the latter method allows the number of types to be reduced more than the number of items. For example, if different types of waste products have common characteristics, they may be classified into one type of waste product.

[0072] Furthermore, the above-mentioned multiple types of waste products may be multiple types of waste products determined by classification based on the names of the waste products, multiple types of waste products determined by classification based on the characteristics of the waste products in a transmission X-ray image, or multiple types of waste items determined by classification based on the characteristics of the waste products and waste in a transmission X-ray image. In this way, if not only the waste products themselves but also collections of waste products are included in the multiple types of waste products, they can be appropriately processed by the first battery detection unit that has been adapted and learned. More specifically, a certain type of waste item may be one type of waste product, multiple types of waste products, or a collection of waste.

[0073] The battery detection method according to the first aspect may further include a step of determining which of a plurality of predetermined categories the input transmission X-ray image belongs to based on the characteristics of the input transmission X-ray image. In this case, the above-mentioned detection step is executed according to the category to which the input transmission X-ray image belongs, and the first battery detection unit and the second battery detection unit for the category to which the input transmission X-ray image belongs are used. This allows the battery detection method to be adapted to various environments and to respond to changes in the environment.

[0074] An information processing system according to a second aspect of this embodiment includes (A) a means for detecting an image portion in which any of a predetermined number of types of waste products appears in an input transmission X-ray image of at least one of waste products and waste material; (B) a first battery detection unit of a number of the above-mentioned types that has been trained in advance to detect a battery built into any of the above-mentioned number of types of waste products from a transmission X-ray image of that type of waste product; (C) a means for causing the first battery detection unit corresponding to the type of waste product appearing in the detected image portion to detect a battery appearing in the detected image portion; (D) a second battery detection unit that has been trained in advance to detect a battery from a transmission X-ray image of at least waste material or the battery itself; and (E) a means for causing the second battery detection unit to detect a battery appearing in an area other than the area of ​​the detected image portion in the input transmission X-ray image.

[0075] A battery detection method according to a third aspect of this embodiment includes the steps of (A) detecting an image portion in which any of a predetermined number of types of waste items appear in an input transmission X-ray image of at least one of waste products and waste, (B) causing a first battery detection unit, which corresponds to the type of waste item appearing in the detected image portion and has been previously trained to detect a battery built into the type of waste item from a transmission X-ray image of the type of waste item, to detect the battery appearing in the detected image portion, and (C) causing a second battery detection unit, which has been previously trained to detect a battery from at least a transmission X-ray image of the waste item or the battery itself, to detect a battery appearing in an area of ​​the input transmission X-ray image other than the area of ​​the detected image portion.

[0076] An information processing system according to a fourth aspect of this embodiment includes (A) a means for detecting an image portion in which any of a predetermined number of types of waste items appear in an input transmission X-ray image of at least one of waste products and waste materials; (B) a first battery detection unit of a number of the above-mentioned types that has been trained in advance to detect a battery built into any of the above-mentioned number of types of waste items from a transmission X-ray image of that type of waste item; (C) a means for causing the first battery detection unit corresponding to the type of waste item appearing in the detected image portion to detect a battery appearing in the detected image portion; (D) a second battery detection unit that has been trained in advance to detect a battery from a transmission X-ray image of at least the waste material or the battery itself; and (E) a means for causing the second battery detection unit to detect a battery appearing in an area of ​​the input transmission X-ray image other than the area of ​​the detected image portion.

[0077] A program for causing a processor to execute the above-mentioned processes can be created, and the program is stored in a computer-readable storage medium or storage device, such as a flexible disk, an optical disk such as a CD-ROM (Read Only Memory), a magneto-optical disk, a semiconductor memory (e.g., a ROM), a hard disk, etc. Data during processing is temporarily stored in a storage device such as a RAM (Random Access Memory). [Explanation of symbols]

[0078] 100 X-ray machine 200 NAS 300 Control device 400 Distributor 310 Image acquisition unit 320 Pre-processing unit 330 First data storage unit 340 Object detection unit 350 Second data storage unit 360 Battery detection unit 370 First battery detection unit 380 Second battery detection unit 390 Instruction output section 325 Classification section

Claims

1. A step of detecting an image portion in which any of a plurality of predetermined types of waste products appears in an input transmission X-ray image in which at least any of the waste products and waste materials is captured; A step of detecting a battery appearing in the detected image portion by a first battery detection unit which corresponds to the type of waste product appearing in the detected image portion and which has been previously trained to detect a battery built in the waste product of the type from a transmitted X-ray image of the waste product of the type; a step of causing a second battery detection unit, which has been trained in advance to detect a battery from at least a transmission X-ray image of the waste or the battery itself, to detect a battery appearing in an area other than the area of ​​the detected image portion in the input transmission X-ray image; A program for causing a computer to execute the above.

2. The second battery detection unit is A device that detects a battery by learning a transmission X-ray image of the waste or the battery itself, in which the battery area is designated, and a transmission X-ray image of each of the multiple types of waste products, in which the battery area built into the waste product is designated, or The device detects batteries by learning transmitted X-ray images of waste or batteries themselves, in which the battery area is designated, and transmitted X-ray images of waste products other than the multiple types, in which the battery area built into the waste products is designated. The program according to claim 1.

3. In the input transmitted X-ray image, the detected image portion is masked with a specific color, so that the second battery detection unit detects a battery in an area other than the area of ​​the detected image portion. The program according to claim 1.

4. The step of detecting the image portion comprises: An object detection unit learns a transmission X-ray image in which an area of ​​the waste product of each of the plurality of types is specified, and detects the type of waste product appearing in the input transmission X-ray image and an image portion in which the waste product appears. The program according to claim 1.

5. The multiple types of waste products are multiple types of waste products determined by classification based on the names of the waste products, multiple types of waste products determined by classification based on the characteristics of the waste products in a transmission X-ray image, or multiple types of waste items determined by classification based on the characteristics of the waste products and waste in a transmission X-ray image. The program according to claim 1.

6. A step of determining which of a plurality of predetermined categories the input transmission X-ray image belongs to based on the characteristics of the input transmission X-ray image. Further comprising: The detecting step is performed according to a category to which the input transmission X-ray image belongs; The first battery detection unit and the second battery detection unit for a category to which the input transmission X-ray image belongs are used. The program according to claim 1.

7. A step of detecting an image portion in which any of a plurality of predetermined types of waste products appears in an input transmission X-ray image in which at least any of the waste products and waste materials is captured; a step of detecting a battery appearing in the detected image portion by a first battery detection unit which corresponds to the type of waste product appearing in the detected image portion and which has been previously trained to detect a battery built in the waste product of the type from a transmitted X-ray image of the waste product of the type; a step of causing a second battery detection unit, which has been trained in advance to detect a battery from at least a transmission X-ray image of the waste or the battery itself, to detect a battery appearing in an area other than the area of ​​the detected image portion in the input transmission X-ray image; 4. A battery detection method implemented in a computer.

8. A means for detecting an image portion in which any of a plurality of predetermined types of waste products appears in an input transmission X-ray image in which at least any of waste products and waste materials is photographed; a first battery detection unit for detecting a battery built in any one of the plurality of types of waste products from a transmission X-ray image of the waste product; a means for causing the first battery detection unit corresponding to the type of waste product appearing in the detected image portion to detect a battery appearing in the detected image portion; A second battery detection unit that is trained in advance to detect a battery from at least a transmission X-ray image of the waste or the battery itself; a means for causing the second battery detection unit to detect a battery appearing in an area other than the area of ​​the detected image portion in the input transmitted X-ray image; An information processing system having the above configuration.

9. A step of detecting an image portion in which any of a plurality of predetermined types of waste items appears in an input transmission X-ray image in which at least any of the waste products and waste materials is captured; a step of detecting a battery appearing in the detected image portion by a first battery detection unit which corresponds to the type of waste item appearing in the detected image portion and which has been previously trained to detect a battery built in the waste item of the type from a transmitted X-ray image of the waste item of the type; a step of causing a second battery detection unit, which has been trained in advance to detect a battery from at least a transmission X-ray image of the waste or the battery itself, to detect a battery appearing in an area other than the area of ​​the detected image portion in the input transmission X-ray image; A program for causing a computer to execute the above.

10. A step of detecting an image portion in which any of a plurality of predetermined types of waste items appears in an input transmission X-ray image in which at least any of the waste products and waste materials is captured; a step of detecting a battery appearing in the detected image portion by a first battery detection unit which corresponds to the type of waste item appearing in the detected image portion and which has been previously trained to detect a battery built in the waste item of the type from a transmitted X-ray image of the waste item of the type; a step of causing a second battery detection unit, which has been trained in advance to detect a battery from at least a transmission X-ray image of the waste or the battery itself, to detect a battery appearing in an area other than the area of ​​the detected image portion in the input transmission X-ray image; a battery detection method executed by a computer;

11. a means for detecting an image portion in which any of a plurality of predetermined types of waste items appears in an input transmission X-ray image in which at least any of a waste product and a waste material is photographed; a first battery detection unit for detecting a battery contained in any one of the plurality of types of waste from a transmission X-ray image of the waste; a means for causing the first battery detection unit corresponding to the type of waste item appearing in the detected image portion to detect a battery appearing in the detected image portion; A second battery detection unit that is trained in advance to detect a battery from at least a transmission X-ray image of the waste or the battery itself; a means for causing the second battery detection unit to detect a battery appearing in an area other than the area of ​​the detected image portion in the input transmitted X-ray image; An information processing system having the above configuration.

Images