Computer program, information processing device, information processing method, and waste selection system

Abstract

To provide a computer program, an information processing device, an information processing method, and a waste selection system.SOLUTION: A computer program implements processes of: acquiring an X-ray image of waste; inputting the acquired X-ray image into each of plural types of learning models for detecting a lithium-ion battery from the X-ray image so as to execute detection processing using each learning model; determining whether the lithium-ion battery is included in the waste by using a detection result by each learning model; and making a computer execute processing for outputting information based on a determination result.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a computer program, an information processing method, an information processing device, and a waste sorting system. [Background technology]

[0002] In recent years, a technology has been proposed in which waste transported by a belt conveyor or the like is imaged by an imaging device and specific waste is selected from the obtained images. For example, Patent Document 1 discloses a system that irradiates X-rays from multiple directions, processes the obtained X-ray images from multiple directions to generate three-dimensional data, and detects objects to be removed contained in the waste based on the generated three-dimensional data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-21894 Summary of the Invention [Problem to be solved by the invention]

[0004] However, since waste contains various types and sizes of waste in addition to the items to be removed, it is expected that it will be difficult to detect the desired items to be removed even if three-dimensional data is generated.

[0005] The present disclosure has been made in consideration of the above circumstances, and aims to provide a computer program, an information processing device, an information processing method, and a waste sorting system that can accurately determine whether waste contains lithium ion batteries by using X-ray images. [Means for solving the problem]

[0006] The computer program disclosed herein is a computer program for causing a computer to execute a process of acquiring X-ray images of waste, inputting the acquired X-ray images into each of a number of learning models for detecting lithium-ion batteries from X-ray images, executing a detection process using each learning model, determining whether the waste contains lithium-ion batteries using the detection results from each learning model, and outputting information based on the determination results. Effect of the Invention

[0007] According to the present disclosure, it is possible to accurately determine whether or not waste contains lithium ion batteries by using X-ray images. [Brief description of the drawings]

[0008] [Figure 1] 1 is a schematic diagram showing a configuration example of a waste sorting system according to a first embodiment. FIG. [Diagram 2] FIG. 2 is a block diagram illustrating an internal configuration of an information processing device. [Diagram 3] FIG. 2 is a schematic diagram illustrating an example of the configuration of a learning model. [Figure 4] 4 is a flowchart showing a procedure of processes executed in the waste sorting system according to the first embodiment. [Diagram 5] 13 is a schematic diagram showing an example of displaying a detection result in the second embodiment. FIG. [Figure 6] FIG. 13 is a schematic diagram showing an example of the configuration of a learning model for judgment in embodiment 3. [Figure 7] FIG. 13 is an explanatory diagram illustrating an overview of a process executed by an information processing device according to a fourth embodiment. [Figure 8] 13 is a schematic diagram showing an example of a display when it is determined that a lithium ion battery is included in an X-ray image. FIG. [Figure 9] FIG. 13 is a schematic diagram showing a configuration example of a waste sorting system according to a sixth embodiment. [Figure 10] 13 is a flowchart showing a procedure of processes executed in the waste sorting system according to the sixth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the present invention will be described in detail with reference to the drawings showing embodiments thereof. (Embodiment 1) Fig. 1 is a schematic diagram showing a configuration example of a waste sorting system according to embodiment 1. The waste sorting system according to embodiment 1 includes a conveyor 1, an X-ray imaging device 2, a sorting device 3, and an information processing device 4 (see Fig. 2).

[0010] The conveyor 1 is a transport device such as a belt conveyor or roller conveyor, and transports waste brought in from an entrance to a predetermined destination. The waste transported by the conveyor 1 is, for example, ordinary waste or bulky waste discharged from general households. The waste may be in a state in which it is contained in a garbage bag, or may be in a state in which it has been removed from a garbage bag. The destination of the transport by the conveyor 1 is a crusher that crushes the waste, or a pit for temporarily storing the waste. Alternatively, the destination of the waste may be an incinerator that incinerates the waste, etc.

[0011] Waste may contain objects that should be removed beforehand. One example of an object to be removed is lithium-ion batteries. If lithium-ion batteries are put into a crusher and crushed, they may ignite and cause a fire. Although lithium-ion batteries are objects that should be removed from waste, lithium-ion batteries may be found mixed in with ordinary garbage and bulky garbage discharged from general households, either alone or installed in electronic devices and home appliances. Electronic devices and home appliances that contain lithium-ion batteries and may be disposed of as waste include mobile batteries, wireless earphones, wireless speakers, digital cameras, IC recorders, electric toothbrushes, electronic cigarettes, and robot vacuum cleaners.

[0012] In this embodiment, in order to detect such waste on the conveyor 1, the waste transported by the conveyor 1 is imaged by an X-ray imaging device 2, and the X-ray image of the waste obtained by the X-ray imaging device 2 is analyzed by an information processing device 4 to determine whether the waste contains lithium-ion batteries.

[0013] The X-ray imaging device 2 is an imaging device that uses transmitted X-rays to image waste transported by the conveyor 1. The X-ray imaging device 2 is provided midway along the transport path of the conveyor 1 (for example, at a position close to the entrance).

[0014] The X-ray imaging device 2 includes an X-ray source 21, a shielding box 22, a detector 23, and the like. The X-ray source 21 is installed above the conveyor 1. The X-ray source 21 irradiates a predetermined irradiation area on the conveyor 1 with X-rays from above the conveyor 1. The shielding box 22 is installed so as to surround the irradiation area. The shielding box 22 is made of a material with high shielding ability against X-rays, and prevents leakage of X-rays outside the irradiation area. The detector 23 is installed below the conveyor 1. The detector 23 is an image sensor such as a CCD (Charge-Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The detector 23 detects X-rays (transmitted X-rays that have penetrated the waste) irradiated by the X-ray source 21 when the waste passes through the irradiation area, thereby generating an X-ray image of the waste. The X-ray imaging device 2 outputs the X-ray image of the waste generated by the detector 23 to the information processing device 4.

[0015] The information processing device 4 is a computer that performs various information processing, and is connected to the outside of the X-ray imaging device 2. Alternatively, the information processing device 4 may be built into the X-ray imaging device 2, or may be installed as an external server. The information processing device 4 acquires an X-ray image of the waste from the X-ray imaging device 2, and determines whether or not the waste contains lithium ion batteries based on the acquired X-ray image of the waste. The determination method will be described in detail later. The information processing device 4 outputs the determination result to the sorting device 3.

[0016] The sorting device 3 includes a separator 31 for sorting the destination of the waste. In this embodiment, the sorting device 3 drives the separator 31 based on the determination result notified from the information processing device 4, and changes the position of the separator 31 to sort the destination of the waste to a crusher or a collection pit.

[0017] For example, when the sorting device 3 obtains from the information processing device 4 a determination result that the waste contains lithium ion batteries, it changes the position of the separator 31 from the default position to a collection position (the position shown by the dashed line in the figure) so that the waste is transported to the collection pit. As a result, the waste containing lithium ion batteries is removed from the transport path and stored in the collection pit. The lithium ion batteries are manually removed from the waste stored in the collection pit by workers. The removed lithium ion batteries are handed over to a recycling business operator, etc.

[0018] On the other hand, when the sorting device 3 acquires from the information processing device 4 a determination result that the waste does not contain lithium ion batteries, it stops the position of the separator 31 at the default position (the position shown by the solid line in the figure) so that the waste is transported to the crusher. As a result, the waste that does not contain lithium ion batteries is transported to the crusher and crushed by the crusher.

[0019] 2 is a block diagram illustrating the internal configuration of the information processing device 4. The information processing device 4 includes a control unit 41, a storage unit 42, a communication unit 43, an operation unit 44, a display unit 45, and the like.

[0020] The control unit 41 is composed of a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The CPU included in the control unit 41 loads various computer programs stored in the built-in ROM and the storage unit 42 onto the RAM and executes them, thereby causing the entire device to function as the information processing device 4 of the present disclosure.

[0021] The control unit 41 is not limited to the above configuration, and may be any processing circuit or arithmetic circuit equipped with a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), a volatile or non-volatile memory, etc. The control unit 41 may also be equipped with a timer that measures the elapsed time from when a measurement start instruction is given to when a measurement end instruction is given, a counter that counts numbers, a clock that outputs date and time information, etc.

[0022] The storage unit 42 includes a storage device such as an SSD (Solid State Drive) or a hard disk drive. The storage unit 42 stores various computer programs executed by the control unit 41, data used when executing the computer programs, and the like. The computer programs stored in the storage unit 42 include a judgment processing program PG1 that judges whether or not lithium ion batteries are contained in the waste transported by the conveyor 1 described above. The judgment processing program PG1 is a computer program that causes a computer (information processing device 4) to execute a process of detecting lithium ion batteries from an X-ray image using a plurality of types of learning models LM11 to LM13, judging whether or not lithium ion batteries are contained in the waste using these detection results, and outputting information based on the judgment results.

[0023] In this embodiment, the judgment processing program PG1 is a single computer program, but it may be a group of programs constructed by multiple computer programs. The judgment processing program PG1 may partially use an existing library or simulator. The judgment processing program PG1 may be distributed to multiple computers and executed by the multiple computers in cooperation with each other.

[0024] A computer program including the judgment processing program PG1 is provided by a non-transitory recording medium (program product) RM on which the computer program is readably recorded. The recording medium RM is a portable memory such as a CD-ROM, a Universal Serial Bus (USB) memory, or a Secure Digital (SD) card. The control unit 41 reads the computer program from the recording medium RM using a reading device (not shown), and stores the read computer program in the storage unit 42. Alternatively, the computer program including the judgment processing program PG1 may be provided by communication. In this case, the control unit 41 acquires the computer program including the judgment processing program PG1 by communication via the communication unit 43, and stores the acquired computer program in the storage unit 42.

[0025] The storage unit 42 stores a plurality of types of learning models LM11 to LM13 used in the judgment processing program PG1. The learning models LM11 to LM13 are each a trained learning model that has been trained to detect a lithium ion battery included in an X-ray image when the X-ray image is input and output a detection result. The learning models LM11 to LM13 are each a learning model with a different detection algorithm, such as R-CNN (Region-based Convolutional Neural Network), YOLO (You Only Look Once), and SSD (Single Shot Multibox Detector). The storage unit 42 stores information describing the model, such as the layer structure of each model, the number and connection relationships of nodes constituting each layer, and weight coefficients and biases between nodes, as well as trained parameters.

[0026] The learning models LM11 to LM13 are not limited to R-CNN, YOLO, and SSD, and may be any learning models with different detection algorithms. For example, the learning models LM11 to LM13 may be three types of learning models selected from learning models including SegNet, U-Net (U-Shaped Network), FCN (Fully Convolutional Network), and PSPNet (Pyramid Scene Parsing Network). In addition, an existing detection method such as template matching may be used instead of a part of the learning models LM11 to LM13.

[0027] In this embodiment, a configuration is described in which three types of learning models LM11 to LM13 are used to detect a lithium ion battery, but the number of learning models used for detection is not limited to three, and may be two or four or more.

[0028] The communication unit 43 includes a communication interface for transmitting and receiving various data. The communication interface included in the communication unit 43 is, for example, a communication interface conforming to a LAN communication standard used in WiFi (registered trademark) or Ethernet (registered trademark). When data to be transmitted is input from the control unit 41, the communication unit 43 transmits the data to be transmitted to a specified destination. When data transmitted from an external device is received, the communication unit 43 outputs the received data to the control unit 41.

[0029] The operation unit 44 includes operation devices such as a touch panel, a keyboard, and switches, and receives various operations and data input by an operator, etc. The control unit 41 performs appropriate control based on various operation information provided by the operation unit 44, and stores the input data in the memory unit 42 as necessary.

[0030] The display unit 45 includes a display device such as a liquid crystal display, an organic EL (Electro-Luminescence) display, etc. The display unit 45 displays information to be notified to an operator or the like in response to an instruction from the control unit 41.

[0031] The information processing device 4 may be configured to receive an operation through an externally connected computer and output information to be notified to the external computer. In this case, the information processing device 4 may not include the operation unit 44 and the display unit 45.

[0032] In this embodiment, the information processing device 4 is a single computer, but may be a computer system configured with multiple computers and peripheral devices. Alternatively, the information processing device 4 may be a virtual machine whose entity is virtualized, or may be a cloud. As long as the information processing device 4 can execute the same process, it may be a microcomputer, a tablet, or a smartphone.

[0033] FIG. 3 is a schematic diagram for explaining a configuration example of the learning model LM11. The learning model LM11 includes an input layer, an intermediate layer, and an output layer. An X-ray image to be processed is input to the input layer. The intermediate layer includes a convolution layer, a pooling layer, a fully connected layer, and the like. In the calculation by the intermediate layer, a process of extracting features from the input image, a process of selecting candidates for a region that is likely to be a detection object (lithium ion battery) included in the input image, a process of identifying the detection object in the selected region, and the like are executed. The output layer refers to the calculation result by the intermediate layer and outputs the detection result. The detection result includes a bounding box that surrounds the detected object and an identification name (class name) that identifies the object in the bounding box. In this embodiment, since the object to be detected is one of lithium ion batteries, the identification name may be omitted. In addition, the detection result may include information on the prediction reliability (certainty) of the detected object.

[0034] The learning model LM11 is generated by preparing a dataset including an X-ray image of waste containing a lithium ion battery, a class name indicating that the object included in the X-ray image is a lithium ion battery, and coordinate values ​​of a bounding box surrounding the object, and using the dataset as training data to perform machine learning with a predetermined algorithm. The machine learning may be performed inside the information processing device 4, or may be performed on an external server communicably connected to the information processing device 4. In the latter case, after the machine learning is performed on the external server, the trained learning model LM11 may be acquired by communication. The machine learning algorithm itself is publicly known, and therefore a detailed description thereof will be omitted.

[0035] The X-ray images used in the training data preferably include images of lithium ion batteries of various types, shapes, sizes, and orientations. For this reason, the number of X-ray images used in the training data may be increased by preparing X-ray images of waste that does not contain lithium ion batteries and X-ray images of lithium ion batteries, and synthesizing the X-ray images of the lithium ion batteries with various sizes and orientations.

[0036] Although an example of the configuration of the learning model LM11 has been described with reference to FIG. 3, the configurations of the learning models LM12 and LM13 are similar to that of the learning model LM11, and therefore the description thereof will be omitted.

[0037] 4 is a flow chart showing the procedure of processing executed in the waste sorting system according to the first embodiment. The X-ray imaging device 2 captures an image of the top of the conveyor 1 (step S101) and generates an X-ray image of the waste (step S102). Specifically, the X-ray source 21 irradiates a predetermined irradiation area on the conveyor 1 with X-rays, and the detector 23 detects the X-rays transmitted by the waste, thereby generating an X-ray image of the waste. The X-ray imaging device 2 outputs the generated X-ray image of the waste to the information processing device 4 (step S103).

[0038] The information processing device 4 acquires the X-ray image of the waste outputted from the X-ray imaging device 2 (step S104). The information processing device 4 communicates with the X-ray imaging device 2 via the communication unit 43, thereby acquiring the X-ray image.

[0039] The control unit 41 of the information processing device 4 inputs the X-ray image acquired in step S104 to each of the learning models LM11 to LM13, and executes calculations by the learning models LM11 to LM13 individually (step S105). In step S105, calculations by the multiple learning models LM11 to LM13 may be executed simultaneously in parallel, or may be executed sequentially for each model. The control unit 41 acquires detection results of each of the learning models LM11 to LM13 by executing calculations by the learning models LM11 to LM13 (step S106).

[0040] The control unit 41 uses the acquired detection results to determine whether or not the waste contains lithium ion batteries (step S107). For example, the control unit 41 determines whether or not the waste contains lithium ion batteries by taking a majority vote on the detection results from a plurality of types of learning models LM11 to LM13. In this embodiment, three types of learning models LM11 to LM13 are used, so if two or more types of models have obtained detection results indicating that a lithium ion battery has been detected, the control unit 41 determines that the waste contains lithium ion batteries. Conversely, if fewer than two types of models have obtained detection results indicating that a lithium ion battery has been detected, the control unit 41 determines that the waste does not contain lithium ion batteries.

[0041] Alternatively, the control unit 41 may determine whether or not the waste contains a lithium ion battery by taking the logical sum of the detection results from the multiple types of learning models LM11 to LM13. That is, the control unit 41 may determine that the waste contains a lithium ion battery if there is one or more models that have obtained a detection result indicating that a lithium ion battery has been detected, and may determine that the waste does not contain a lithium ion battery if there is none.

[0042] The information processing device 4 outputs the result of the determination as to whether or not the waste contains lithium ion batteries to the sorting device 3 (step S108).

[0043] The sorting device 3 acquires the judgment result output from the information processing device 4 (step S109). The sorting device 3 drives the separator and changes the position of the separator 31 in accordance with the judgment result by the information processing device 4, thereby sorting the destination of the waste to a crusher or a collection pit (step S110). That is, when the sorting device 3 obtains a judgment result that the waste contains lithium ion batteries, it changes the position of the separator 31 so that the destination of the waste is the collection pit. Also, when the sorting device 3 obtains a judgment result that the waste does not contain lithium ion batteries, it changes the position of the separator 31 so that the destination of the waste is the crusher.

[0044] As described above, in the first embodiment, the detection results from a plurality of types of learning models LM11 to LM13 are used, so that it is possible to accurately determine whether or not the waste contains lithium ion batteries.

[0045] In the first embodiment, a configuration is adopted in which a plurality of learning models LM11 to LM13, each of which is different in model itself, are prepared, and a lithium ion battery to be detected is detected using these learning models LM11 to LM13. Alternatively, a plurality of learning models, each of which is the same in model itself but has different detection sensitivity, may be prepared, and a lithium ion battery to be detected may be detected using these learning models. For example, all of the learning models LM11 to LM13 are learning models using R-CNN, but the learning model LM11 may be a model learned using the entire X-ray image, the learning model LM12 may be a model having a relatively high detection sensitivity for lithium ion batteries when the size of waste present in the surroundings is relatively small (or large), and the learning model LM13 may be a model having a relatively high detection sensitivity for lithium ion batteries when the orientation of the lithium ion battery is vertical (or horizontal, or diagonal), etc.

[0046] In the first embodiment, the learning models LM11 to LM13 used for detecting a lithium ion battery are predefined models. Alternatively, the learning models LM11 to LM13 may be models selected by an operator. For example, the information processing device 4 may be configured to display a selection screen on the display unit 45, which includes an explanation of the detection sensitivity, strengths and weaknesses, etc., of each learning model, and to accept the selection of the learning models LM11 to LM13 to be used for detection via the operation unit 44.

[0047] (Embodiment 2) In the second embodiment, a configuration will be described in which the detection results of the learning models LM11 to LM13 are displayed on the display unit 45 and whether the detection results are correct or not is accepted. The overall configuration of the waste sorting system and the configuration of each device that makes up the waste sorting system are the same as those in the first embodiment, so the description thereof will be omitted.

[0048] Fig. 5 is a schematic diagram showing an example of a display of detection results in the second embodiment. Fig. 5 shows an example of an interface screen that displays the detection results by a plurality of learning models LM11 to LM13 and accepts the correctness of the detection results by each model. For example, when the detection results by the plurality of learning models LM11 to LM13 are divided into those that detect a lithium ion battery and those that do not (when at least one detection result is different from the others), the information processing device 4 displays an interface screen as shown in Fig. 5 on the display unit 45 and accepts the correctness of the detection results of each model through the operation unit 44. The interface screen in Fig. 5 shows an example in which it is accepted that the detection results of the learning models LM11 and LM12 are correct and that the detection result of the learning model LM13 is incorrect.

[0049] When the information processing device 4 receives the result of success or failure for each model through the interface screen, regardless of the detection results of the learning models LM11 to LM13, it outputs information based on the result of success or failure received from the operator to the sorting device 3. The sorting device 3 drives the separator 31 according to the information output from the information processing device 4, and changes the position of the separator 31 to sort the waste to a crusher or a recovery pit.

[0050] Furthermore, the X-ray image used for detection and the information on whether the detection was successful or not received via the interface screen shown in FIG. 5 may be stored as training data, and the learning models LM11 to LM13 may be re-trained using the stored training data.

[0051] As described above, in the second embodiment, when the detection results by the multiple types of learning models LM11 to LM13 are divided into correct and incorrect answers, the judgment by the operator is accepted, so that the presence or absence of a lithium ion battery can be judged more accurately.

[0052] (Embodiment 3) In the third embodiment, a configuration will be described in which a learning model is used to determine whether or not a lithium ion battery is included in an X-ray image. The overall configuration of the waste sorting system and the configuration of each device that makes up the waste sorting system are the same as those in the first embodiment, so the description thereof will be omitted.

[0053] 6 is a schematic diagram showing a configuration example of a learning model LM20 for determination in the third embodiment. When the detection results by the learning models LM11 to LM13 are input, the learning model LM20 for determination is trained to output information indicating whether or not a lithium ion battery is included in the X-ray image. For example, a CNN (Convolutional Neural Networks), an RNN (Recurrent Neural Networks), an LSTM (Long Short Term Memory), an autoencoder, or the like is used for the learning model LM20.

[0054] The learning model LM20 includes an input layer, an intermediate layer, and an output layer. Each layer includes a plurality of nodes. The input layer includes, for example, three nodes, and the detection results by the learning models LM11 to LM13 are input to each node. The intermediate layer includes one or more layers. Each node in the intermediate layer is connected to a node in the layer provided before and after the node. The activity of each node and the strength of the connection between the nodes (weight coefficient) are determined during the learning process. The node in each layer outputs a value calculated based on the weight coefficient and activity to the next node. The output layer calculates a probability using a softmax function based on the value input from the intermediate layer. The output layer outputs the calculated probability as a judgment result by the learning model LM20. For example, the output layer may output the probability (P1) that a lithium ion battery is included in the X-ray image and the probability (P2) that a lithium ion battery is not included in the X-ray image. Such a learning model LM20 is generated by using a dataset containing X-ray images and correct answer information indicating whether the X-ray images contain lithium-ion batteries as training data and learning with an existing learning algorithm.

[0055] When an X-ray image including waste is acquired from the X-ray imaging device 2, the control unit 41 of the information processing device 4 performs a process of detecting lithium ion batteries included in the X-ray image using the detection learning models LM11 to LM13, similar to the first embodiment. Next, the control unit 41 inputs the detection results by the learning models LM11 to LM13 to the judgment learning model LM20, and executes a calculation by the learning model LM20. The control unit 41 acquires the calculation result by the learning model LM20, and judges whether or not a lithium ion battery is included in the X-ray image.

[0056] For example, when the probability P1 that the X-ray image contains a lithium ion battery is greater than a threshold TH1, the control unit 41 determines that the X-ray image contains a lithium ion battery, and when the probability P2 that the X-ray image does not contain a lithium ion battery is greater than a threshold TH2, the control unit 41 determines that the X-ray image does not contain a lithium ion battery. The thresholds TH1 and TH2 are constants that are set in advance. The thresholds TH1 and TH2 may be the same value or different values.

[0057] As described above, in the third embodiment, the learning model LM20 for judgment is used in addition to the learning models LM11 to LM13 for detection, so that the presence or absence of a lithium ion battery can be determined more accurately.

[0058] (Embodiment 4) In the fourth embodiment, a configuration will be described in which weighting is performed on a plurality of types of learning models LM11 to LM13, and whether or not a lithium ion battery is included in an X-ray image is determined based on the detection results and weighting by the learning models LM11 to LM13. The overall configuration of the waste sorting system and the configuration of each device that makes up the waste sorting system are the same as those in the first embodiment, so the description thereof will be omitted.

[0059] 7 is an explanatory diagram for explaining an outline of the process executed by the information processing device 4 according to the embodiment 4. The multiple types of learning models LM11 to LM13 used in the embodiment 4 are all, for example, learning models based on YOLO, but the learning model LM11 is a learning model with high detection sensitivity when the size of the waste containing lithium batteries is relatively large, the learning model LM12 is a learning model with high detection sensitivity when the size of the waste containing lithium ion batteries is medium, and the learning model LM13 is a learning model with high detection sensitivity when the size of the waste containing lithium ion batteries is relatively small.

[0060] Alternatively, learning model LM11 may be a learning model with high detection sensitivity when the size of waste included in the background is relatively large, learning model LM12 may be a learning model with high detection sensitivity when the size of waste included in the background is medium, and learning model LM13 may be a learning model with high detection sensitivity when the size of waste included in the background is relatively small.

[0061] Furthermore, learning model LM11 may be a model that has high detection sensitivity when the orientation of the lithium ion battery in the X-ray image is vertical, learning model LM12 may be a model that has high detection sensitivity when the orientation of the lithium ion battery in the X-ray image is horizontal, and learning model LM13 may be a model that has high detection sensitivity when the orientation of the lithium ion battery in the X-ray image is diagonal.

[0062] When the control unit 41 of the information processing device 4 acquires an X-ray image including waste from the X-ray imaging device 2, the control unit 41 inputs the acquired X-ray image to each of the learning models LM11 to LM13, performs calculations using each of the learning models LM11 to LM13, and acquires a detection result. The detection result by each of the learning models LM11 to LM13 is represented by, for example, the probability P1 to P3 that a lithium ion battery is included in the X-ray image.

[0063] The control unit 41 specifies the size of the waste in the X-ray image, and sets weights for each of the detection results P1 to P3 according to the specified size. The size of the waste in the X-ray image may be specified, for example, by preparing a learning model for detecting waste from an X-ray image and using this learning model. Alternatively, the size of the waste may be specified by setting a shading value (gray value) of a background area where no waste is present, counting pixels having that shading value (gray value range) to determine the area of ​​the background area, and subtracting it from the original X-ray image.

[0064] The control unit 41 determines the weights W1 to W3 so that when the size of the identified waste is relatively large, the weight W1 for the detection result P1 is large and the weights for the detection results P2 and P3 are small. The same applies when the size of the identified waste is medium or relatively small, and the weights W1 to W3 may be determined so that the weights W2 and W3 for the detection results P2 and P3 are relatively large.

[0065] The control unit 41, for example, compares the magnitude of the products (P1×W1, P2×W2, P3×W3) of each detection result P1 to P3 and each weighting W1 to W3, and if the largest of these products exceeds a threshold value, it determines that the X-ray image contains a lithium ion battery.

[0066] As described above, in the fourth embodiment, a plurality of learning models LM1 to LM3 having different detection sensitivities can be used to determine whether or not a lithium ion battery is included in an X-ray image.

[0067] In the fourth embodiment, the control unit 41 is configured to determine whether or not a lithium ion battery is included in the X-ray image based on the detection results P1 to P3 by the learning models LM11 to LM13 and the weights W1 to W3, but the determination may be performed using the learning model LM20 for determination. In this case, the learning model LM20 for determination may be trained to output information indicating the determination result of whether or not a lithium ion battery is included in the X-ray image when the detection results P1 to P3 by the learning models LM11 to LM13 and the weights W1 to W3 are input. Alternatively, the learning model LM20 for determination may be trained to output information indicating the determination result of whether or not a lithium ion battery is included in the X-ray image when the detection results P1 to P3 by the learning models LM11 to LM13 and the feature amounts extracted from the X-ray image (such as the size of the waste in the X-ray image, the size of the waste in the background, and the orientation of the lithium ion battery) are input.

[0068] (Embodiment 5) In the fifth embodiment, a configuration for marking the position of waste containing lithium ion batteries on an X-ray image will be described. The overall configuration of the waste sorting system and the configuration of each device that makes up the waste sorting system are the same as those in the first embodiment, so the description thereof will be omitted.

[0069] Fig. 8 is a schematic diagram showing a display example when it is determined that a lithium ion battery is included in an X-ray image. When the control unit 41 of the information processing device 4 determines that a lithium ion battery is included in an X-ray image based on the detection results of the learning models LM11 to LM13, the control unit 41 displays a marker M on the display unit 45 to mark the position of the lithium ion battery on the X-ray image. In the example of Fig. 8, the marker M is shown as a black circle, but the color and shape of the marker M may be appropriately changed. Furthermore, as shown in the example of Fig. 8, text information indicating that a lithium ion battery is included in the X-ray image may also be displayed.

[0070] As described above, in embodiment 5, if a lithium ion battery is contained in an X-ray image, it is marked with a marker M on the X-ray image, allowing an operator to grasp the position of the waste containing the lithium ion battery on the screen.

[0071] (Embodiment 6) In the sixth embodiment, a configuration will be described in which, when it is determined that a lithium ion battery is included in an X-ray image, the position of the lithium ion battery is indicated by a light emitting device such as a laser pointer.

[0072] 9 is a schematic diagram showing a configuration example of a waste sorting system according to embodiment 6. The waste sorting system according to embodiment 6 includes a light irradiation device 5 in addition to a conveyor 1, an X-ray imaging device 2, and an information processing device 4. The conveyor 1, the X-ray imaging device 2, and the information processing device 4 are the same as those in embodiment 1, so their explanation will be omitted.

[0073] The light irradiation device 5 is, for example, a laser pointer, and is a device for indicating the position of an object to an operator by emitting light. The light irradiation device 5 is installed above the conveyor 1. The light irradiation device 5 includes, for example, a plurality of light emitting units 51 in the transport direction and width direction of the conveyor 1, and indicates the position of an object to an operator by emitting light from any one of the light emitting units 51 onto the conveyor 1.

[0074] For this reason, it is assumed that the distance (geometric positional relationship) between the X-ray imaging device 2 and each light-emitting unit 51, and the transport speed of the conveyor 1 are known, and this information is stored in the storage unit 42 of the information processing device 4. When the control unit 41 of the information processing device 4 determines that a lithium-ion battery is included in the X-ray image, it calculates the timing at which the lithium-ion battery will pass directly below each light-emitting unit 51 based on the position of the lithium-ion battery in the X-ray image and the information stored in the storage unit 42. The control unit 41 drives each light-emitting unit 51 at the calculated timing to emit light, thereby indicating the position of the lithium-ion battery to the operator.

[0075] 10 is a flowchart showing the procedure of processing executed in the waste sorting system according to embodiment 6. The control unit 41 of the information processing device 4 acquires an X-ray image in the same procedure as in embodiment 1, and determines whether or not a lithium ion battery is included in the acquired X-ray image (steps S601 to S604).

[0076] The control unit 41 determines whether or not the waste contains a lithium ion battery (step S605), and if it determines that the waste contains a lithium ion battery (S605: YES), it calculates the timing at which the lithium ion battery in the waste will pass directly below the light-emitting unit 51 based on the position of the lithium ion battery in the X-ray image and the information stored in the memory unit 42 (step S606).

[0077] The control unit 41 drives the light emitting unit 51 of the light irradiation device 5 at the calculated light emission timing (step S607). The control unit 41 outputs a control signal for driving the light emitting unit 51 to the light irradiation device 5, thereby causing the light emitting unit 51 to emit light at a timing when a lithium ion battery in the waste passes directly below the light emitting unit 51. The control unit 41 causes the multiple light emitting units 51 arranged in the transport direction to emit light in sequence from the upstream side to the downstream side, thereby tracking the movement of the lithium ion battery and indicating the position of the lithium ion battery.

[0078] The light irradiation device 5 irradiates light from the light emitting section 51 in accordance with a control signal from the information processing device 4, thereby indicating the location of the lithium ion battery in the waste to the worker (step S608).

[0079] As described above, in the sixth embodiment, the positions of lithium ion batteries contained in waste can be indicated to an operator by emitting light from the light irradiation device 5. The operator can remove the waste at the position indicated by the light irradiation device 5 from the conveyor 1, and can prevent the waste containing lithium ion batteries from being fed into the crusher.

[0080] In this embodiment, a method for determining whether or not a waste contains a lithium ion battery based on an X-ray image of the waste has been described. The method described in this embodiment is not limited to lithium ion batteries, but can also be applied to any object to be removed, such as dry batteries, metal pieces, gas cylinders, etc.

[0081] Furthermore, in this embodiment, the system is configured to determine whether or not lithium ion batteries are contained in waste transported by the conveyor 1, but the system may also be configured to determine whether or not lithium ion batteries are contained in waste temporarily placed on a platform, pit, or the like. If it is determined that the waste placed on a platform, pit, or the like does not contain lithium ion batteries, the waste may be transported en bloc to a crusher or incinerator. If it is determined that the waste placed on a platform, pit, or the like contains lithium ion batteries, the lithium ion batteries may be removed from the waste, and then the waste may be transported en bloc to a crusher or incinerator.

[0082] The embodiments disclosed herein are illustrative in all respects and should not be considered as limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims.

[0083] The matters described in each embodiment can be combined with each other. In addition, the independent claims and dependent claims described in the claims can be combined with each other in any and all combinations regardless of the citation format. Furthermore, the claims use a format in which a claim cites two or more other claims (multiple claim format), but this is not limited to this. The claims may also be written in a format in which a multiple claim cites at least one other multiple claim (multi-multi claim). [Explanation of symbols]

[0084] 1 Conveyor 2 X-ray imaging device 3 sorting device 4. Information processing equipment 41 Control section 42 Storage section 43 Communications Department 44 Control section 45 Display section 5 Light irradiation device 51 Light emitting part PG1 Decision Processing Program LM11~LM13 Learning Model LM20 Learning Model

Claims

1. Obtaining x-ray images of the waste; inputting the acquired X-ray image into each of a plurality of learning models for detecting lithium ion batteries from X-ray images, and executing a detection process using each learning model; Using the detection results of each learning model, it is determined whether the waste contains a lithium ion battery; Output information based on the judgment result A computer program for causing a computer to execute a process.

2. Taking a majority vote on the detection results obtained by the multiple learning models; Based on the result of the majority vote, it is determined whether the waste contains lithium-ion batteries.

2. The computer program product according to claim 1, for causing a computer to execute a process.

3. Display multiple detection results, Accept the result of the detection 2. The computer program product according to claim 1, for causing a computer to execute a process.

4. A logical sum is taken for the detection results obtained by the plurality of learning models; Based on the logical sum of the detection results, it is determined whether the waste contains a lithium ion battery.

2. The computer program product according to claim 1, for causing a computer to execute a process.

5. When the detection results obtained by the plurality of learning models are input, a judgment learning model that has been trained to output information indicating whether or not the waste contains a lithium ion battery is used to judge whether or not the waste contains a lithium ion battery.

2. The computer program product according to claim 1, for causing a computer to execute a process.

6. The plurality of types of learning models include learning models having different detection sensitivities for lithium ion batteries in the entire X-ray image, detection sensitivities for lithium ion batteries due to the size of surrounding waste, or detection sensitivities for the orientation of lithium ion batteries.

2. The computer program product of claim 1.

7. The plurality of learning models are trained using a synthetic image obtained by synthesizing an X-ray image of a lithium ion battery and an X-ray image of waste that does not contain a lithium ion battery as training data.

2. The computer program product of claim 1.

8. If it is determined that the waste contains a lithium ion battery, a mark indicating the location of the lithium ion battery is displayed on the X-ray image.

2. The computer program product according to claim 1, for causing a computer to execute a process.

9. When it is determined that the waste contains a lithium ion battery, the operation of a light irradiation device arranged to indicate the location of the lithium ion battery is controlled. The computer program product according to claim 1, for causing a computer to carry out a process.

10. At least one processor; The processor, Obtaining x-ray images of the waste; inputting the acquired X-ray image into each of a plurality of learning models for detecting lithium ion batteries from X-ray images, and executing a detection process using each learning model; Using the detection results of each learning model, it is determined whether the waste contains a lithium ion battery; Output information based on the judgment result Information processing device.

11. Obtaining x-ray images of the waste; inputting the acquired X-ray image into each of a plurality of learning models for detecting lithium ion batteries from X-ray images, and executing a detection process using each learning model; Using the detection results of each learning model, it is determined whether the waste contains a lithium ion battery; Output information based on the judgment result An information processing method in which processing is carried out by a computer.

12. an X-ray imaging device that irradiates X-rays onto waste material and generates an X-ray image of the waste material; An information processing device in which the computer program according to any one of claims 1 to 9 is implemented; A waste sorting system comprising:

13. A sorting device for sorting specific waste contained in the waste from other waste, The information processing device includes: When it is determined that the waste contains lithium ion batteries, the operation of the sorting device is controlled to separate the lithium ion batteries from other waste.

13. A waste sorting system according to claim 12.

14. The waste disposal method further includes a light irradiation device that irradiates light to indicate the location of a specific waste contained in the waste, The information processing device includes: When it is determined that the waste contains a lithium ion battery, the operation of the light irradiation device is controlled to indicate the location of the lithium ion battery.

13. A waste sorting system according to claim 12.

Images