Screw piece assembly error detection device, screw piece assembly error detection method, and computer program

Abstract

To provide a screw piece assembly error detection device which can detect assembly errors of a plurality of screw pieces constituting screws of a twin-screw extruder.SOLUTION: A screw piece assembly error detection device includes: an acquisition part for acquiring appearance data indicating appearances of a plurality of screw pieces which are arrayed of two screws or the two screws; and a calculation part for calculating a difference between the plurality of screw pieces constituting the first screw and the plurality of screw pieces constituting the second screw, on the basis of the acquired appearance data, and detects assembly errors of the screw pieces of the respective screws.SELECTED DRAWING: Figure 9

Description

Technical Field

[0001] The present invention relates to a screw piece assembly error detection device, a screw piece assembly error detection method, and a computer program.

Background Art

[0002] A twin-screw extruder includes two screws that mesh and rotate. The screws are configured by combining and integrating multiple types of screw pieces. Checking for incorrect combinations of screw pieces is done visually. Also, checking for incorrect combinations of screw pieces is performed based on the feel when the two screws are combined and rotated.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, since it is a manual check, there is a possibility of overlooking an incorrect combination. In a twin-screw extruder, there is a problem that the screw may be damaged when an incorrect screw is attached to the device and operated.

[0005] An object of the present disclosure is to provide a screw piece assembly error detection device, a screw piece assembly error detection method, and a computer program that can detect incorrect combinations of a plurality of screw pieces constituting the screw of a twin-screw extruder.

Means for Solving the Problems

[0006] A screw piece assembly error detection device relating to one aspect of the present disclosure is a screw piece assembly error detection device that compares the configurations of two screws for a twin-screw extruder, which are made by combining a plurality of screw pieces, and detects an assembly error of the screw pieces, comprising: an acquisition unit that acquires appearance data representing the appearance of the plurality of screw pieces arranged for each of the two screws or the appearance of the two screws; and a calculation unit that calculates the difference between the plurality of screw pieces constituting the first screw and the plurality of screw pieces constituting the second screw based on the acquired appearance data.

[0007] A screw piece assembly error detection method relating to one aspect of the present disclosure is a screw piece assembly error detection method that compares the configurations of two screws for a twin-screw extruder, which are made by combining a plurality of screw pieces, and detects an error in the assembly of the screw pieces, wherein appearance data representing the arrangement of the plurality of screw pieces or the appearance of the two screws is acquired, and the difference between the plurality of screw pieces constituting the first screw and the plurality of screw pieces constituting the second screw is calculated based on the acquired appearance data.

[0008] A computer program relating to one aspect of this disclosure is a computer program that causes a computer to perform a process to compare the configurations of two screws for a twin-screw extruder, which are made up of a plurality of screw pieces, and to detect an error in the assembly of the screw pieces, and to obtain appearance data representing the arrangement of the plurality of screw pieces of each of the two screws or the appearance of the two screws, and to cause the computer to perform a process to calculate the difference between the plurality of screw pieces constituting the first screw and the plurality of screw pieces constituting the second screw based on the obtained appearance data. [Effects of the Invention]

[0009] According to this disclosure, it is possible to detect incorrect assembly of multiple screw pieces that make up the screw of a twin-screw extruder. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic diagram showing an example of the configuration of a screw piece assembly error detection device according to Embodiment 1. [Figure 2] This is a schematic diagram showing an example of a screw configuration. [Figure 3] This flowchart shows the processing procedure for the screw piece assembly error detection method according to Embodiment 1. [Figure 4] This is a schematic diagram showing a screw image. [Figure 5] This is a conceptual diagram illustrating a method for extracting images of the screw portion. [Figure 6] This is a conceptual diagram illustrating a method for detecting incorrect assembly of screw pieces. [Figure 7] This is a schematic diagram showing the display screen for assembly error detection results. [Figure 8] This is an explanatory diagram showing an example of the results of incorrect assembly detection. [Figure 9] This is a schematic diagram showing an example configuration of a screw piece assembly error detection device according to Embodiment 2. [Figure 10] This is a conceptual diagram showing the object detection learning model according to Embodiment 2. [Figure 11] This flowchart shows the processing procedure for the screw piece assembly error detection method according to Embodiment 2. [Figure 12] This is a conceptual diagram showing the object detection results. [Figure 13] This is a table showing the object detection results. [Figure 14] This is a schematic diagram illustrating another example of how to import appearance data. [Figure 15] This is a schematic diagram showing an example configuration of a screw piece assembly error detection device according to Embodiment 3. [Figure 16] This is a conceptual diagram showing a type recognition detection learning model according to Embodiment 3. [Figure 17] This flowchart shows the processing procedure for the screw piece assembly error detection method according to Embodiment 3.

Mode for Carrying Out the Invention

[0011] Specific examples of the screw piece combination error detection device, the screw piece combination error detection method, and the computer program according to the embodiment of the present invention will be described below with reference to the drawings. It should be noted that the present invention is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims. Also, at least a part of the following embodiments and modification examples may be arbitrarily combined.

[0012] Hereinafter, the present invention will be specifically described based on the drawings showing its embodiments. (Embodiment 1) FIG. 1 is a schematic diagram showing a configuration example of the screw piece combination error detection device 1 according to Embodiment 1. The screw piece combination error detection device 1 according to Embodiment 1 is a computer that executes arithmetic processing for detecting an error in the combination of a plurality of screw pieces 3a constituting two screws 3 for a twin-screw extruder A.

[0013] The twin-screw extruder A includes a cylinder having a heater, two screws 3 provided so as to be drivable in the rotational direction within the cylinder for melting, plasticizing, and kneading the raw material resin, and a rotation motor for rotating the screws 3.

[0014] FIG. 2 is a schematic diagram showing a configuration example of the screw 3. The screw 3 is configured as a single screw 3 by combining and integrating a plurality of types of screw pieces 3a. For example, a forward flight piece having a flight screw shape for transporting the raw material in the forward direction, a reverse flight piece for transporting the raw material in the reverse direction, a kneading piece for kneading the raw material, etc. are arranged and combined in an order and position according to the characteristics of the raw material, whereby the screw 3 is configured.

[0015] As a hardware configuration, the screw piece combination error detection device 1 includes an arithmetic unit 11, a storage unit 12, a communication unit 13, a display unit 14, and an operation unit 15.

[0016] The arithmetic unit 11 is a processor having arithmetic circuits such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), internal storage devices such as ROM (Read Only Memory) and RAM (Random Access Memory), and I / O terminals. The arithmetic unit 11 may also include one or more arithmetic circuits such as a GPU (Graphics Processing Unit), TPU (Tensor Processing Unit), or AI chip (AI semiconductor) specialized for image processing related to object detection and image recognition. The arithmetic unit 11 detects incorrect assembly of screw pieces 3a by reading and executing the computer program 16 stored in the storage unit 12. Each functional part of the screw piece incorrect assembly detection device 1 may be implemented in software, or some or all of it may be implemented in hardware.

[0017] The storage unit 12 is, for example, a storage device such as a hard disk, an EEPROM (Electrically Erasable Programmable ROM), or flash memory. The storage unit 12 stores various programs executed by the arithmetic unit 11, and various data necessary for processing by the arithmetic unit 11. In this embodiment, the storage unit 12 stores at least the computer program 16 executed by the arithmetic unit 11.

[0018] The computer program 16 may be written to the storage unit 12 during the manufacturing stage of the screw piece assembly error detection device 1, or it may be distributed via a network from another information processing device. The screw piece assembly error detection device 1 acquires the computer program 16 via communication and writes it to the storage unit 12. The computer program 16 may also be recorded in a readable manner on a recording medium 10 such as a semiconductor memory such as flash memory, an optical disc, a magneto-optical disc, or a magnetic disc. The screw piece assembly error detection device 1 reads the computer program 16 and stores it in the storage unit 12.

[0019] The communication unit 13 is a communication circuit that transmits and receives data to and from the measuring device 2 via wired or wireless connection. The communication unit 13 may also be a circuit that reads data from the memory of the measuring device 2 or from a storage medium removed from the measuring device 2. The communication unit 13 functions as an acquisition unit that acquires appearance data representing the appearance of multiple screw pieces 3a arranged on each of the two screws 3 or the appearance of the two screws 3.

[0020] The measuring device 2 is a device that can obtain appearance data showing the appearance of multiple arranged screw pieces 3a or screws 3. The communication device is, for example, a camera or video camera that images the screws 3. The communication device may also be a smartphone, tablet terminal, or laptop PC (Personal Computer) having a light-receiving lens and an image sensor. The measuring device 2, which has an imaging function, transmits image data showing the appearance of the two screws 3 as appearance data to the screw piece assembly error detection device 1. The image data is data that includes the pixel values ​​of multiple pixels arranged in the horizontal and vertical directions.

[0021] The method for imaging the long screw 3 is not particularly limited. The entire two screws 3 may be imaged in a single image, or the images obtained by imaging one end and the other end of the two screws 3 separately may be combined to create a single image containing the entire two screws 3. Alternatively, the two screws 3 may be scanned from one end to the other, and the image may be combined to create a single image containing the entire two screws 3. The image of the two screws 3 may be a video.

[0022] The measuring device 2 may be a distance measuring sensor that measures the distance between multiple points on the surface of the screw 3 to obtain point cloud data. Distance measuring sensors include infrared sensors such as LiDAR. The infrared sensor comprises a light-emitting element that emits infrared light and a light-receiving element that receives the infrared light that is emitted from the screw 3 and reflected. The light-emitting element is, for example, an infrared laser such as a vertical cavity surface-emitting laser (VCSEL), which emits a dot pattern arranged vertically and horizontally onto the screw 3. The light-receiving element is, for example, a CMOS image sensor. The infrared sensor calculates the distance to the screw 3 based on the round-trip time from when it is emitted towards the screw 3 until it is reflected back. The measuring device 2 calculates the distance to each dot pattern and transmits the point cloud data, which is three-dimensional information of the two screws 3, as appearance data to the screw piece assembly error detection device 1. The point cloud data is, for example, a representation of many points on the surface of the two screws 3 in three-dimensional coordinates.

[0023] The screw piece assembly error detection device 1 acquires image data or point cloud data, which is appearance data transmitted from the measuring device 2, via the communication unit 13 and stores it in the storage unit 12. The calculation unit 11 can convert the point cloud data into voxel data. The calculation unit 11 can also convert the point cloud data or voxel data into two-dimensional image data. To simplify the explanation, the following example describes how to estimate the screw configuration using two-dimensional image data. Hereafter, the two-dimensional images of the two screws 3 obtained by imaging or distance measurement will be called screw images, and the two-dimensional image data will be called screw image data.

[0024] The display unit 14 is a display device such as a liquid crystal display panel or an organic EL display panel. The display unit 14 displays the screw piece assembly error detection result, etc., according to the control of the calculation unit 11.

[0025] The operation unit 15 is an input device such as a touch panel, mouse, operation buttons, or keyboard that accepts input from a user using the screw piece assembly error detection device 1.

[0026] The screw piece assembly error detection device 1 described above may be a multicomputer system comprising multiple computers, or it may be a virtual machine virtually constructed by software. Furthermore, part or all of the screw piece assembly error detection device 1 may be configured as a cloud server.

[0027] Figure 3 is a flowchart showing the processing procedure for the screw piece assembly error detection method according to Embodiment 1, Figure 4 is a schematic diagram showing a screw image, Figure 5 is a conceptual diagram showing a method for extracting a partial image of a screw, and Figure 6 is a conceptual diagram showing a method for detecting an assembly error of a screw piece 3a. The calculation unit 11 of the screw piece assembly error detection device 1 acquires screw image data from the measuring device 2 via the communication unit 13 (step S111). As shown in Figure 4, the screw image includes images of two screws 3. In the example shown in Figure 4, the screw image is an image obtained by imaging two screws 3 supported by two bases, and the image of the bases is also included in the background. The screw image data is data representing the appearance of multiple arranged screw pieces 3a that constitute each of the two screws 3, or the appearance of the two screws 3.

[0028] Next, the calculation unit 11 receives input for a first screw range, which indicates the range of the first screw 3 in the screw image (step S112). The user of the screw piece assembly error detection device 1 can specify the first screw range in the screw image by operating the operation unit 15. The first screw range is the range that is subject to detection of assembly errors in the screw piece 3a, in particular the range that surrounds the image portion of the first screw 3. For example, in Figure 5 "Screw Range Input", the range enclosed by the dashed rectangular frame is the first screw range.

[0029] Next, the calculation unit 11 detects the second screw range (step S113). For example, the calculation unit 11 detects the range in which images similar to the first screw range exist as the second screw range. The second screw range is the range in which incorrect assembly of the screw piece 3a is detected, and in particular, the range surrounding the image portion of the second screw 3. For example, in Figure 5 "Screw Range Detection", the range enclosed by the lower dashed rectangular frame is the second screw range.

[0030] The calculation unit 11 may be configured to detect both the first screw range and the second screw range by image processing such as pattern matching.

[0031] Hereafter, the image portion of the first screw 3 will be referred to as the first screw portion image, and the image portion of the second screw 3 will be referred to as the second screw portion image. Furthermore, the first screw portion image and the second screw portion image will be collectively referred to as the screw portion image as appropriate.

[0032] Next, the calculation unit 11 performs longitudinal alignment of the first screw portion image and the second screw portion image (step S114). As shown in Figure 5 "Screw Range Detection", the longitudinal positions of the first screw portion image and the second screw portion image are misaligned. Through the process in step S114, the first screw portion image and the second screw portion image are aligned, as shown in Figure 5 "Alignment of Screw Portion Images". Longitudinal alignment is performed on a pixel-by-pixel basis.

[0033] Furthermore, the calculation unit 11 performs image processing such as scaling, rotation, and distortion correction on the first screw portion image and the second screw portion image to make the outline and size of the first screw portion image and the outline and size of the second screw portion image substantially the same.

[0034] Furthermore, the calculation unit 11 can perform longitudinal translation, rotation, scaling, and reduction of the screw portion image by affine transformation of the first or second screw portion image. If distortion correction of the screw portion image is also necessary, the calculation unit 11 can perform nonlinear transformation of the first or second screw portion image.

[0035] Next, the calculation unit 11 removes the background image of the first screw image included in the first screw range and the background image of the second screw image included in the second screw range (step S115). As shown in Figure 5, "Alignment of Screw Part Images, etc.", the first screw range and the second screw range include background images, such as an image of the base. As shown in Figure 5, "Background Removal", the processing in step S115 removes the extra background images other than the first screw part image and the second screw part image. The background of the screw part image after background image removal is, for example, a black image with a brightness value of zero.

[0036] Next, the calculation unit 11 calculates the difference between the screw portion images (step S116). The calculation unit 11 calculates the magnitude of the difference between the brightness value of the first screw portion image and the brightness value of the second screw portion image on a pixel-by-pixel basis. Hereinafter, an image having the difference as a pixel value will be called a screw difference image.

[0037] The central diagram in Figure 6 shows schematic diagrams of the first and second screw portion images, while the upper diagram in Figure 6 shows a schematic diagram of the screw difference image. In the example shown in Figure 6, the screw pieces 3a in the central part of the two screws 3 are different. It can be seen that the difference in screw pieces 3a in the central part of the screw portion image is reflected in the screw difference image. Areas in the screw portion image where there is no difference in brightness become black areas in the screw difference image, and areas in the screw portion image where there is a difference in brightness become high-brightness areas in the screw difference image.

[0038] The calculation unit 11 then calculates the average brightness value of the difference image as a matching score (index value) for each position in the longitudinal direction (step S117). For example, if the XY coordinate position of a pixel in the difference image is (x, y) and the pixel value (size of the difference) at that pixel position is D(x, y), then the matching score at any position x=x0 in the longitudinal direction is expressed as ΣD(x0, y) / (y0+1). "Σ" is the sum of the pixel values ​​D(x0,0), D(x0,1), ... D(x0,y0) up to y=0, 1, 2, ... y0.

[0039] The lower part of Figure 6 is a graph showing the matching score. The horizontal axis represents the longitudinal position (X coordinate) of the screw difference image, and the vertical axis represents the matching score. It can be seen that the matching score is high in the areas where there is an assembly error in screw piece 3a.

[0040] Note that the average pixel value is just one example of a matching score. The method for calculating the matching score is not particularly limited, as long as it represents the difference between the first screw portion image and the second screw portion image.

[0041] Next, the calculation unit 11 determines whether or not there is an assembly error in the screw piece 3a (step S118). For example, the calculation unit 11 determines whether or not there are any locations where the matching score is above a predetermined threshold. The calculation unit 11 may also determine whether or not there are any regions where the moving average value of the matching score in the longitudinal direction is above a predetermined threshold. The calculation unit 11 may also detect anomalies in the matching score in the longitudinal direction of the screw difference image and determine whether or not there is an assembly error based on the presence or absence of anomalies. The calculation unit 11 may also determine the difference using an object detection learning model or image processing, regardless of the matching score.

[0042] If it is determined that there are no assembly errors (step S118: NO), the calculation unit 11 finishes processing. If it is determined that there are assembly errors (step S118: YES), the calculation unit 11 displays the location of the assembly error in the screw piece 3a on the display unit 14 (step S119).

[0043] Figure 7 is a schematic diagram showing the assembly error detection result display screen 131, and Figure 8 is an explanatory diagram showing an example of the assembly error detection result. In step S119, the calculation unit 11 displays the assembly error detection result display screen 131 on the display unit 14. The assembly error detection result display screen 131 includes a first screw portion image, a second screw portion image, and a screw difference image. The calculation unit 11 may display the first screw portion image, the second screw portion image, and the screw difference image side by side vertically so that their longitudinal positions coincide.

[0044] Furthermore, the calculation unit 11 displays an image 131a indicating the location of an incorrectly assembled screw piece 3a. An incorrectly assembled screw piece 3a is, for example, a region where the moving average of the matching score is greater than or equal to a threshold. The image 131a is, for example, a rectangular frame image that surrounds the first screw portion image, the second screw portion image, and the screw difference image in that region.

[0045] The rectangular frame image is just an example; it could be a circular image, an elliptical image, an arrow image indicating an incorrectly assembled screw piece 3a, or the like. The calculation unit 11 may also be configured to display the incorrectly assembled screw piece 3a by changing the color of the first screw portion image and the second screw portion image at the incorrectly assembled screw piece 3a. Alternatively, the calculation unit 11 may display the incorrectly assembled screw piece 3a by displaying a numerical value representing its position in the longitudinal direction.

[0046] Furthermore, the misassembly detection result display screen 131 may include a graph of the matching score. The horizontal axis of the graph indicates the position in the longitudinal direction of the screw difference image, and the vertical axis indicates the matching score.

[0047] Furthermore, the screw piece assembly error detection device 1 may also include a speaker or light-emitting device that notifies the user of the assembly error by sound or light when an assembly error of the screw piece 3a is detected. The screw piece assembly error detection device 1 may also be configured to transmit notification data for notifying the user of the assembly error of the screw piece 3a to the user's communication terminal or the control device of the twin-screw extruder A.

[0048] According to the screw piece assembly error detection device 1 of the embodiment 1 configured in this way, it is possible to detect assembly errors of screw pieces 3a constituting the screw 3 by comparing images of the screw portions of the two screws 3 mounted on the twin-screw extruder A. Assembly errors of screw pieces 3a can be detected without relying on manual intervention.

[0049] Furthermore, the screw piece assembly error detection device 1 can visually display the location of the screw piece 3a assembly error using the detection result display screen 131. Specifically, the first screw portion image and the second screw portion image are displayed side by side, and the location of the screw piece 3a assembly error can be indicated by the pointed-out image 131a.

[0050] In this first embodiment, an example was described in which incorrect assembly of screw pieces 3a is detected based on visual data obtained by imaging or measuring the distance of the screw 3. However, incorrect assembly of screw pieces 3a may also be detected using visual data obtained by imaging or measuring the distance of the multiple screw pieces 3a constituting the first screw 3 and the multiple screw pieces 3a constituting the second screw 3, which are arranged before assembly (see Figure 14).

[0051] Furthermore, although this first embodiment describes an example of processing two-dimensional screw image data, the system may be configured to detect assembly errors in screw piece 3a using three-dimensional data of screw 3. Three-dimensional data of screw 3 can be obtained based on images captured from multiple different positions and angles using a camera or the like. Alternatively, three-dimensional data of screw 3 can be obtained from point cloud data obtained by distance measurement. By comparing the three-dimensional data of the first screw 3 with the three-dimensional data of the second screw 3, three-dimensional difference data can be calculated, and a matching score can be calculated to detect assembly errors in screw piece 3a.

[0052] Furthermore, in this embodiment 1, an example was described in which the difference between the brightness value of the first screw portion image and the brightness value of the second screw portion image is calculated to detect an assembly error of the screw piece 3a, but the detection process is not limited to this. For example, the calculation unit 11 may determine the types of the multiple screw pieces 3a that make up each of the two screws 3 by pattern matching processing, and determine whether or not there is a difference between the screw pieces 3a that make up the two screws 3, thereby determining an assembly error of the screw piece 3a.

[0053] Alternatively, the calculation unit 11 may calculate the feature quantities of multiple screw pieces 3a that make up each of the two screws 3 by pattern matching processing, and determine whether there are any differences in the feature quantities of the screw pieces 3a that make up the two screws 3, thereby determining if there is an error in assembling the screw pieces 3a.

[0054] (Embodiment 2) The screw piece assembly error detection device 1 according to Embodiment 2 differs from Embodiment 1 in that it uses a learning model to detect assembly errors in the screw piece 3a. The other components of the screw piece assembly error detection device 1 are the same as those of the screw piece assembly error detection device 1 according to Embodiment 1, so the same reference numerals are used for the same parts, and detailed explanations are omitted.

[0055] Figure 9 is a schematic diagram showing an example of the configuration of the screw piece assembly error detection device 1 according to Embodiment 2. The storage unit 12 of the screw piece assembly error detection device 1 according to Embodiment 2 stores the object detection learning model 17.

[0056] Figure 10 is a conceptual diagram showing an object detection learning model 17 according to Embodiment 2. The object detection learning model 17 includes, for example, a convolutional neural network (CNN) trained by deep learning. The object detection learning model 17 has an input layer 17a into which screw image data is input, an intermediate layer 17b that extracts features from the screw image data, and an output layer 17c that outputs inference results related to the detected object. The object detection learning model 17 is, for example, a YOLO model.

[0057] Each layer of the object detection learning model 17 has multiple nodes. The nodes in each layer are connected by edges. Each layer has an activation function (response function), and the edges have weights. The value output from the nodes of each layer is calculated from the value of the node in the previous layer, the weight of the edge, and the activation function of the layer. The weight of the edge can be changed through learning.

[0058] The input layer 17a of the object detection learning model 17 has multiple nodes that accept input of screw image data, that is, the pixel values ​​of each pixel that make up the image of the surface of screw 3, and passes the input pixel values ​​to the intermediate layer 17b. The intermediate layer 17b has multiple sets of convolutional layers (CONV layers), a pooling layer, and a fully connected layer. The convolutional layer performs filtering on the values ​​output from the nodes of the previous layer and extracts a feature map. The pooling layer reduces the feature map output from the convolutional layer to obtain a new feature map.

[0059] The output layer 17c has a node that outputs the final inference results for objects detected from the screw image. The inference results include the center coordinates and dimensions of the bounding box surrounding the object, an object detection score indicating the likelihood that the image enclosed by the bounding box is an image of an object, and a class score indicating the likelihood that the object belongs to a specific class.

[0060] More specifically, the normalized screw image input to input layer 17a is divided into multiple grids, and the bounding box position, size, object score, and class score are determined for each grid. The bounding box position in a single grid is expressed, for example, as a relative position to the top-left vertex or center of the grid.

[0061] If the screw image is divided into S×S grids, with B bounding boxes detected per grid and C object classes, then the output obtained from output layer 17c will be, for example, an S×S×(B×5+C) dimensional tensor.

[0062] The output from output layer 17c contains multiple overlapping bounding boxes. However, post-processing to remove these overlapping bounding boxes, such as NMS (Non-Maximum Suppression), allows us to obtain the most likely bounding box position and size surrounding one or more objects in the screw image, as well as the object detection score and class score.

[0063] The method for generating the object detection learning model 17 is described below. First, training data is prepared, which includes multiple screw image data and annotation files for each screw image data. The annotation files are training data that show the correct values ​​to be assigned to the corresponding screw images. Specifically, the annotation files are data that shows the center coordinate position, vertical and horizontal size, and class of the bounding box surrounding the image of the screw piece 3a included in the corresponding screw image. The class indicates a group classified according to the type of screw piece 3a. The type of screw piece 3a is determined by the L / D ratio, leads, lead angle, number of threads, etc. of the screw piece 3a.

[0064] When screw images from training data are input to a CNN, an object detection learning model 17 can be generated by optimizing the weight coefficients of the neural network using methods such as backpropagation and gradient descent, so that the error (value of a predetermined loss function or error function) between the data output from the CNN and the data shown in the training data is minimized.

[0065] While YOLO was described as an example of an object detection learning model 17, the object detection learning model 17 may also be constructed using R-CNN, Fast R-CNN, Faster R-CNN, or other CNNs. Furthermore, an object detection learning model 17 using algorithms such as decision trees, random forests, and SVM (Support Vector Machine) may also be used. Moreover, the object detection learning model 17 may be constructed by combining multiple of the above-mentioned algorithms.

[0066] Figure 11 is a flowchart showing the processing procedure for the screw piece assembly error detection method according to Embodiment 2. The calculation unit 11 of the screw piece assembly error detection device 1 acquires appearance data from the measuring device 2 via the communication unit 13 (step S211). The appearance data is data representing the appearance of multiple arranged screw pieces 3a that constitute two screws 3, or the appearance of two screws 3. Here, the calculation unit 11 is described as using two-dimensional screw image data based on the appearance data.

[0067] The calculation unit 11 inputs the screw image data to the object detection learning model 17 and outputs a feature map (step S212). The feature map includes information for identifying the position and class of objects contained in the screw image.

[0068] The calculation unit 11 identifies the arrangement position and class of the multiple screw pieces 3a that constitute each of the two screws 3 based on the feature map (step S213). Specifically, the coordinates of the center position of the bounding box surrounding each of the multiple screw pieces 3a, the length and width dimensions, the object detection score, the class score, etc., are obtained from the feature map. Based on the object detection score and the class score, the calculation unit 11 identifies the position and size of the bounding box and the class of each of the multiple screw pieces 3a. For example, the class with the highest class score and an object detection score above a threshold is identified.

[0069] Figure 12 is a conceptual diagram showing the object detection results, and Figure 13 is a table showing the object detection results. In Figure 12, the rectangular frame is the bounding box, and the numbers "1", "2", ... are numbers (No.) used to identify each of the identified screw pieces 3a. The center position of the bounding box is represented by the X and Y coordinates. "FF" indicates the type of screw piece 3a (forward flight piece) belonging to a particular class.

[0070] The calculation unit 11 can determine whether an object belongs to the first screw 3 or the second screw 3 by referring to the value of the center Y coordinate of the bounding box. For example, as shown in Figures 12 and 13, a bounding box with a center Y coordinate of approximately 300 surrounds the screw piece 3a of the first screw 3. A bounding box with a center Y coordinate of approximately 150 surrounds the screw piece 3a of the second screw 3.

[0071] Next, the calculation unit 11 compares the arrangement position and class of each of the multiple screw pieces 3a constituting the first screw 3 with the arrangement position and class of each of the multiple screw pieces 3a constituting the second screw 3 to determine whether there is an assembly error in the screw pieces 3a (step S214). Specifically, the calculation unit 11 determines whether there are screw pieces 3a of different types that have the same arrangement position (x coordinate) in the longitudinal direction of the screw image. For example, the calculation unit 11 determines whether the class of screw piece 3a enclosed by a bounding box (No. 1) with a center X coordinate of 100 is the same as the class of screw piece 3a enclosed by a bounding box (No. 15) with the same center X coordinate of 100. If the classes are different, the calculation unit 11 determines that there is an assembly error in the screw pieces 3a.

[0072] If it is determined that there are no assembly errors (step S214: NO), the calculation unit 11 finishes processing. If it is determined that there are assembly errors (step S214: YES), the calculation unit 11 displays the location of the assembly error in the screw piece 3a on the display unit 14 (step S215). The display unit 14 displays the location of screw pieces 3a that have the same alignment position in the longitudinal direction of the screw image but are of different classes. The method of displaying the assembly error location is the same as in Embodiment 1.

[0073] According to the screw piece assembly error detection device 1 of the embodiment 2 configured in this way, it is possible to estimate the arrangement position and type (class) of each of the multiple screw pieces 3a that constitute the screw 3 for the twin-screw extruder A. The screw piece assembly error detection device 1 can then determine whether or not there is an assembly error among the screw pieces 3a that constitute the two screws 3 mounted on the twin-screw extruder A, and can display the location of the assembly error. This makes it possible to detect screw piece assembly errors without relying on manual intervention.

[0074] Furthermore, by using a machine learning model, the position and type of screw piece 3a constituting screw 3 can be estimated with high accuracy.

[0075] In this second embodiment, an example was described in which an incorrect assembly of screw pieces 3a is detected based on visual data obtained by imaging or measuring the distance of the screw 3. However, visual data obtained by imaging or measuring the distance of multiple screw pieces 3a arranged before assembly may also be used.

[0076] Figure 14 is a schematic diagram showing another example of a method for acquiring external data. The calculation unit 11 acquires screw image data obtained by imaging multiple screw pieces 3a arranged with spaces in between, as shown in Figure 11. By inputting the screw image data into the object detection learning model 17, the calculation unit 11 can detect each of the multiple screw pieces 3a and recognize the arrangement position and type of the screws 3. In the case of a screw image obtained by imaging an assembled screw 3, there is a risk that the boundaries between screw pieces 3a may not be correctly determined. However, by imaging screw pieces 3a arranged with gaps between them before assembly, screw pieces 3a can be identified more accurately, and incorrect assembly of screw pieces 3a can be detected.

[0077] Furthermore, each of the multiple screw pieces 3a may be provided with an image-recognizable mark. The calculation unit 11 can correctly recognize each of the multiple screw pieces 3a based on the image of the mark included in the screw image. The system may be configured to provide different marks on each type of screw piece 3a. This allows for more accurate detection of the arrangement position and type of screw pieces 3a that constitute the screw 3.

[0078] Furthermore, although this embodiment 2 describes an example of processing two-dimensional screw image data, array data may also be generated using three-dimensional data of screw 3. Three-dimensional data of screw 3 can be obtained based on images captured from multiple different positions and angles using a camera or the like. Alternatively, three-dimensional data of screw 3 can be obtained from point cloud data obtained by distance measurement. By using a learning model or three-dimensional data matching, the calculation unit 11 can similarly detect assembly errors of screw piece 3a.

[0079] (Embodiment 3) The screw piece assembly error detection device 1 according to Embodiment 3 differs from Embodiment 2 in its method of identifying the arrangement and type of screw pieces 3a. The other components of the screw piece assembly error detection device 1 are the same as those of the screw piece assembly error detection device 1 according to Embodiments 1 and 2, so the same reference numerals are used for the same parts, and detailed explanations are omitted.

[0080] Figure 15 is a schematic diagram showing an example configuration of the screw piece assembly error detection device 1 according to Embodiment 3. The storage unit 12 of the screw piece assembly error detection device 1 according to Embodiment 3 stores a type recognition learning model 18 instead of an object detection learning model 17.

[0081] Figure 16 is a conceptual diagram showing a type recognition detection learning model according to Embodiment 3. The type recognition detection learning model is an image recognition model. The type recognition learning model 18 includes, for example, a convolutional neural network (CNN) trained by deep learning. The configuration of the neural network itself is the same as that of the object detection learning model 17. The type recognition learning model 18 has an input layer 18a into which piece image data, which is data of the image portion of screw piece 3a (hereinafter referred to as piece image), an intermediate layer 18b that extracts feature quantities from the piece image data, and an output layer 18c that outputs accuracy data indicating the probability that the piece image belongs to each of several classes.

[0082] The method for generating the type recognition learning model 18 is described below. First, training data is prepared, which includes multiple piece image data and training data indicating the class to which each piece image belongs. The class indicates a group classified according to the type of screw piece 3a. Then, the calculation unit 11 can generate the type recognition learning model 18 by optimizing the weight coefficients of the neural network using methods such as backpropagation and gradient descent, so that when the screw images of the training data are input to the CNN, the error (value of a predetermined loss function or error function) between the data output from the CNN and the data indicated by the training data is minimized.

[0083] Figure 17 is a flowchart showing the processing procedure for the screw piece assembly error detection method according to Embodiment 3. The calculation unit 11 of the screw piece assembly error detection device 1 acquires appearance data from the measuring device 2 via the communication unit 13 (step S311) and extracts piece appearance data corresponding to the piece image (step S312).

[0084] Similar to embodiments 1 and 2, the calculation unit 11 uses two-dimensional screw image data based on appearance data. In embodiment 3, it is desirable to use image data obtained by imaging a plurality of screw pieces 3a arranged with spaces in between. Piece appearance data is piece image data corresponding to the screw piece 3a portion in the piece image. The calculation unit 11 extracts piece images by processing such as pattern matching. Alternatively, piece images may be extracted using a machine learning (object detection) model.

[0085] At this stage, it is not necessary to identify the type of screw piece 3a; it is sufficient to extract images similar to the image of screw piece 3a. The calculation unit 11 recognizes the position of the extracted piece image. In other words, the calculation unit 11 detects the arrangement of multiple screw pieces 3a.

[0086] Next, the calculation unit 11 inputs the extracted piece images into the type recognition learning model 18 and outputs accuracy data belonging to each of the multiple classes (step S313). Based on the accuracy data output from the type recognition learning model 18, the calculation unit 11 identifies the classes of the multiple screw pieces 3a that constitute the first screw 3 and the second screw 3 (step S314).

[0087] Hereinafter, in the same manner as in Embodiments 1 and 2, the calculation unit 11 compares the arrangement position and class of each of the multiple screw pieces 3a constituting the first screw 3 with the arrangement position and class of each of the multiple screw pieces 3a constituting the second screw 3 to determine whether or not there is an assembly error in the screw pieces 3a (step S315). If the calculation unit 11 determines that there is no assembly error (step S315: NO), the calculation unit 11 ends its processing. If it determines that there is an assembly error (step S315: YES), the calculation unit 11 displays the location of the assembly error in the screw pieces 3a on the display unit 14 (step S316).

[0088] The screw piece assembly error detection device 1 according to Embodiment 3, like Embodiments 1 and 2, can estimate the arrangement position and type of each of the multiple screw pieces 3a that constitute the screw 3 for the twin-screw extruder A, and can detect assembly errors of the screw pieces 3a.

[0089] In this embodiment 2-3, an example was described in which the arrangement position and type of the multiple screw pieces 3a constituting the screw 3 are estimated mainly using a learning model, but it may also be configured to be estimated by rule-based image processing.

[0090] For example, the screw piece assembly error detection device 1 may be configured to estimate the arrangement position and type of screw pieces 3a by template matching processing. The storage unit 12 of the screw piece assembly error detection device 1 stores template images of multiple types of screw pieces 3a in association with the types of screw pieces 3a. The calculation unit 11 identifies the position and type of each of the multiple screw pieces 3a included in the screw image by performing template matching processing using the template images stored in the storage unit 12. The processing after identifying the arrangement position and type of the multiple screw pieces 3a is the same as in the embodiment 2-3 described above.

[0091] The screw piece assembly error detection device 1 may be configured to estimate the arrangement position and type of screw pieces 3a based on feature quantities. The storage unit 12 of the screw piece assembly error detection device 1 stores the feature quantities of multiple types of screw pieces 3a in association with the types of screw pieces 3a. Various feature quantities can be considered, such as edges, changes in intensity, or the positional relationships between characteristic points. The calculation unit 11 identifies the position and type of each of the multiple screw pieces 3a included in the screw image by comparing them with the feature quantities stored in the storage unit 12. The processing after identifying the arrangement position and type of the multiple screw pieces 3a is the same as in the embodiment 2-3 described above. [Explanation of symbols]

[0092] 1: Detection device 2: Measuring device 3: Screw 3a: Screw piece 10: Recording media 11: Arithmetic section 12: Storage section 13: Communications Department 14:Display section 15:Operation section 16: Computer Programs 17: Object detection learning model 18: Species Recognition Learning Model 131: Detection result display screen 131a: Image pointed out A: Twin-screw extruder

Claims

1. A screw piece assembly error detection device that compares the configurations of two screws for a twin-screw extruder, which are made by combining multiple screw pieces, and detects errors in the assembly of the screw pieces, An acquisition unit that acquires appearance data representing the appearance of the multiple screw pieces arranged in each of the two screws or the appearance of the two screws, Based on the acquired appearance data, a calculation unit calculates the difference between the plurality of screw pieces constituting the first screw and the plurality of screw pieces constituting the second screw. A screw piece assembly error detection device equipped with the following features.

2. The aforementioned arithmetic unit, The difference between the appearance data of the first screw and the appearance data of the second screw is calculated. An index value indicating the magnitude of the difference at multiple positions in the longitudinal direction of the screw is calculated. Based on the calculated index value, it is determined whether or not the screw piece is assembled incorrectly. Screw piece assembly error detection device according to claim 1.

3. The system includes a display unit that shows the position of the screw piece that has been determined to be incorrectly assembled based on the aforementioned index value. The screw piece assembly error detection device according to claim 2.

4. The aforementioned arithmetic unit, The arrangement position and type of each of the plurality of screw pieces constituting the first screw and the arrangement position and type of each of the plurality of screw pieces constituting the second screw are estimated. The arrangement and type of each of the plurality of screw pieces constituting the first screw are compared with the arrangement and type of each of the plurality of screw pieces constituting the second screw. Screw piece assembly error detection device according to claim 1.

5. The aforementioned arithmetic unit, When the aforementioned appearance data is input, the object detection learning model, which has been trained to output data indicating the arrangement position and type of each of the multiple screw pieces, receives the appearance data acquired by the acquisition unit and outputs data indicating the arrangement position and type of each of the multiple screw pieces. Screw piece assembly error detection device according to claim 4.

6. The aforementioned arithmetic unit, By performing rule-based image processing on the aforementioned appearance data, data indicating the arrangement position and type of each of the multiple screw pieces is estimated. Screw piece assembly error detection device according to claim 4.

7. The system includes a display unit that shows the positions of the screw pieces, which are the same in arrangement but of different types. Screw piece assembly error detection device according to any one of claims 4 to 6.

8. The aforementioned appearance data includes image data obtained by imaging the plurality of screw pieces or the screw, or point cloud data obtained by measuring the distances between multiple points on the surface of the plurality of screw pieces or the screw. Screw piece assembly error detection device according to any one of claims 1 to 7.

9. A screw piece assembly error detection method for detecting a screw piece assembly error by comparing the configurations of two screws for a twin-screw extruder, which are made by combining multiple screw pieces, and detecting an error in the assembly of the screw pieces, Obtain appearance data representing the appearance of the multiple screw pieces arranged in each of the two screws, or the appearance of the two screws themselves. Based on the acquired visual data, the difference between the plurality of screw pieces constituting the first screw and the plurality of screw pieces constituting the second screw is calculated. Method for detecting incorrect screw piece assembly.

10. A computer program for causing a computer to perform a process to compare the configurations of two screws for a twin-screw extruder, which are made up of multiple screw pieces, and to detect errors in the assembly of the screw pieces, Obtain appearance data representing the appearance of the multiple screw pieces arranged in each of the two screws, or the appearance of the two screws themselves. Based on the acquired visual data, the difference between the plurality of screw pieces constituting the first screw and the plurality of screw pieces constituting the second screw is calculated. A computer program that causes the computer to perform a process.

Images