Abnormality determination device, abnormality determination system, and abnormality determination method

Abstract

To detect an anomaly of a member simply at low cost.SOLUTION: An anomaly determination apparatus includes: an object information acquisition unit which acquires object information, regarding a conveyed object with an unfixed member temporarily disposed therein, from a reader that reads object information indicating a type of the object displayed on a display device; an image acquisition unit which acquires an image from an imaging apparatus that images the inside of the object in a predetermined position, on the basis of a result detected by a position detection apparatus which detects that the object has reached the predetermined position; and a determination unit which evaluates, for each object according to the object information, a state of the member included in the image on the basis of a trained model constructed by machine learning, to determine anomaly of the member temporarily disposed in the object.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an abnormality determination device, an abnormality determination system, and an abnormality determination method. [Background technology]

[0002] In some cases, objects such as vehicles are transported along a production line with components such as metal sheets temporarily installed without being fixed in place. At this time, the components may become misaligned.

[0003] As a technique for identifying the position of a detection target, there is a technique for installing a camera inside a vehicle to detect the riding position of an occupant and control the operation of an airbag (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2019 / 207625 Summary of the Invention [Problem to be solved by the invention]

[0005] However, for example, to detect misalignment of components within an object being transported on a manufacturing line using the technology of Patent Document 1, complex inspections using multiple cameras, sensors, etc. Inspections using multiple cameras, sensors, etc. increase the manufacturing costs of the object.

[0006] An object of the present disclosure is to provide an abnormality determination device, an abnormality determination system, and an abnormality determination method that can inexpensively and easily detect abnormalities in components. [Means for solving the problem]

[0007] The abnormality determination device according to the present disclosure is configured to detect an abnormality in a state where an unfixed member is temporarily installed inside. and production line Regarding the object being transported, A device for displaying the operation status of the production line, the device being fixed at a predetermined position within the production line. An object information acquiring unit acquires object information from a reading device that reads object information indicating the type of the object displayed on the display device, and a position detecting unit detects that the object has reached a predetermined position based on the detection result of the position detecting unit. Arrived an image acquisition unit that acquires an image from an imaging device that images the inside of the object; and a determination unit that evaluates the state of the component included in the image based on a trained model constructed by machine learning for each of the objects according to the object information, and performs an abnormality determination for the component temporarily placed inside the object. The position detection device detects that the object has reached a plurality of different transport positions, and the imaging device captures images of the inside of the object at the plurality of different transport positions. . [Effects of the Invention]

[0008] According to the abnormality determination device, abnormality determination system, and abnormality determination method disclosed herein, abnormalities in components can be detected inexpensively and simply. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of the configuration of an abnormality determination system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of the abnormality determination system according to the embodiment. [Figure 3] FIG. 3 is a block diagram illustrating an example of a functional configuration of the abnormality determination device according to the embodiment. [Figure 4] FIG. 4 is a schematic diagram illustrating an example of how the abnormality determination system according to the embodiment determines an abnormality. [Figure 5] FIG. 5 is a schematic diagram illustrating an example of how the abnormality determination system according to the embodiment determines an abnormality. [Figure 6] FIG. 6 is a schematic diagram illustrating an example of how the abnormality determination system according to the embodiment determines an abnormality. [Figure 7] FIG. 7 is a flowchart illustrating an example of a procedure of an abnormality determination process performed by the abnormality determination system according to the embodiment. [Figure 8]FIG. 8 is a block diagram illustrating an example of a functional configuration of an abnormality determination device according to a modified example of the embodiment. [Figure 9] FIG. 9 is a flowchart showing an example of a procedure of an abnormality determination process performed by an abnormality determination system according to a modified example of the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of an abnormality determination device, an abnormality determination system, and an abnormality determination method according to the present disclosure will be described with reference to the drawings.

[0011] (Example of anomaly detection system configuration) 1 is a schematic diagram showing an example of the configuration of an abnormality determination system 1 according to an embodiment. The abnormality determination system 1 performs abnormality determination on a member such as a sheet metal SH temporarily installed on a vehicle VH being transported in a production line.

[0012] In the production line, the vehicles VH are transported on a transport path such as a belt conveyor CNV, etc. A display device 60 that displays various statuses of the production line is suspended from the ceiling of the production line.

[0013] As shown in FIG. 1, the abnormality determination system 1 includes an abnormality determination device 10, a reading device 30, an imaging device 40, and position detection devices 50 (50a, 50b, 50c).

[0014] The abnormality determination device 10 is configured as a computer equipped with, for example, a CPU (Central Processing Unit) as described below, and performs abnormality determination on a mail sheet SH temporarily installed in the vehicle VH based on information from the reading device 30 and images from the imaging device 40.

[0015] The reading device 30 is a reader or the like equipped with a scanning function such as a camera or a laser sensor, and reads the vehicle model of the vehicle VH to be determined from the display device 60. Alternatively, the reading device 30 may be a recording device or the like connected directly or indirectly to the display device 60, and configured to record the video signal output from the display device 60 using a screenshot function or the like.

[0016] Display device 60 is a liquid crystal display (LCD) or an organic electro-luminescence (EL) display, and is suspended from, for example, the ceiling in the production line. Display device 60 displays various information related to the production line.

[0017] The information displayed on the display device 60 may include, for example, the type of vehicle VH currently being transported on the production line, the operating status of the production line, the quality of the vehicle VH, etc. The type of vehicle VH is displayed in the form of, for example, a number, symbol, model, or the like that can identify the type of vehicle VH.

[0018] 1, the model number indicating the model of the vehicle VH is displayed on the display device 60. The display device 60 also displays that the production line is operating normally and that the quality of the vehicle VH is normal.

[0019] The reading device 30 reads, as object information, information that identifies the type of vehicle VH, such as the vehicle model number, displayed on the display device 60. The reading device 30 generates the vehicle model information as object information, for example, in the form of image data or the like.

[0020] The imaging device 40 is a digital camera or the like that has a built-in imaging element such as a CCD (Charge Coupled Device) or a CIS (CMOS Image Sensor). The imaging device 40 can generate moving or still images including a plurality of frame images captured at a predetermined frame rate, and captures, for example, images of the interior of the vehicle VH that is being placed on a belt conveyor CNV and being transported.

[0021] Components such as Melting Sheet SH are placed inside the vehicle VH being transported on the belt conveyor CNV. Melting Sheet SH is a heat-sealed vibration-damping material for vehicle bodies made of a base material such as asphalt. Melting Sheet SH is installed on the floor and deck surfaces of the vehicle VH, for example, to provide vibration, sound, heat, water, and dust protection for the vehicle body.

[0022] 1, multiple melt sheets SH are temporarily installed inside the vehicle VH without being fixed. In the subsequent manufacturing process, these melt sheets SH are heat-sealed and fixed onto the floor panel of the vehicle VH, the floor surface of the deck, etc.

[0023] Here, the Mel Sheet SH is temporarily installed with different models, numbers, orientations, and locations for each vehicle model, etc. Therefore, due to human error, etc., the Mel Sheet SH may be temporarily installed in an incorrect state.

[0024] Furthermore, the mail sheet SH may tear during handling, etc. Furthermore, between the time the mail sheet SH is temporarily installed and the time it is fixed in the desired position inside the vehicle VH by heat sealing or the like, the mail sheet SH may become misaligned due to the vehicle VH being transported on the belt conveyor CNV, etc.

[0025] The abnormality determination device 10 determines whether or not there is an abnormality in the temporarily installed and unfixed mail sheet SH.

[0026] The belt conveyor CNV is provided with, for example, a plurality of position detection devices 50a, 50b, and 50c, which are installed in this order from upstream to downstream. These position detection devices 50a, 50b, and 50c are, for example, limit switches, and detect when the vehicle VH being transported reaches a predetermined position on the belt conveyor CNV.

[0027] The imaging device 40 described above captures an image of the interior of the vehicle VH at each predetermined position each time the vehicle VH reaches a predetermined position corresponding to each of the position detection devices 50a, 50b, and 50c.

[0028] Next, the hardware configuration of the abnormality determination system 1 will be described with reference to FIG.

[0029] Fig. 2 is a diagram showing an example of the hardware configuration of the abnormality determination system 1 according to the embodiment. As shown in Fig. 2, the abnormality determination device 10 of the abnormality determination system 1 is configured as a computer including a CPU 14, a RAM (Random Access Memory) 13, a ROM (Read Only Memory) 12, etc. A reading device 30, an imaging device 40, and position detection devices 50 (50a, 50b, 50c) are connected to the abnormality determination device 10, and a display device 60 is also connected via a network NT or the like.

[0030] More specifically, the abnormality determination device 10 includes a CPU 14, a RAM 13, a ROM 12, an auxiliary storage device 11, an interface (I / F) 15, an input device 17, an output device 18, and a communication interface (I / F) 19. The CPU 14, the RAM 13, the ROM 12, the auxiliary storage device 11, the I / F 15, the input device 17, the output device 18, and the communication I / F 19 are connected to one another by an internal bus.

[0031] The CPU 14 is a computing device that controls the overall operation of the abnormality determination device 10. The RAM 13 is a volatile storage device that is used as a work area for the CPU 14. The ROM 12 is a non-volatile storage device that stores programs and the like that realize various processes performed by the CPU 14.

[0032] The CPU 14, RAM 13, and ROM 12 may be configured as an SoC (System on a Chip) mounted on a single board, for example.

[0033] The I / F 15 is connected to the reading device 30, the imaging device 40, and the position detection device 50 via a bus, and is configured to enable the exchange of various signals between the abnormality judgment device 10 and the reading device 30, the imaging device 40, and the position detection device 50.

[0034] The signals transmitted from the abnormality determination device 10 to the reading device 30 and the imaging device 40 include, for example, control signals for controlling the operations of the reading device 30 and the imaging device 40 .

[0035] The signal that the abnormality determination device 10 receives from the reading device 30 is, for example, a signal containing object information such as the vehicle model read by the reading device 30 from the display device 60. The signal that the abnormality determination device 10 receives from the imaging device 40 is, for example, a signal containing information on multiple images of the vehicle VH captured by the imaging device 40 at different positions.

[0036] The signal that the abnormality determination device 10 receives from the position detection device 50 is, for example, a signal that includes position information of the vehicle VH detected by the position detection device 50.

[0037] The communication I / F 19 connects the abnormality determination device 10 and the display device 60 via a network NT, either wired or wirelessly. The network NT to which the abnormality determination device 10 and the display device 60 are connected is, for example, a local area network (LAN) in a factory that has a production line for the vehicle VH, or a wide area network (WAN) such as the Internet.

[0038] As a result, when the abnormality determination device 10 detects an abnormality in the mail seat SH of the vehicle VH, it is possible to transmit an instruction to display a warning to the display device 60 via the communication I / F 19.

[0039] The auxiliary storage device 11 is a hard disk drive (HDD), a solid state drive (SSD), an embedded multi media card (eMMC), a microSD card, or the like, and stores various types of data.

[0040] The input device 17 includes a keyboard, a mouse, etc., and is configured to enable a worker involved in the work of determining whether a component such as the mercury sheet SH is abnormal to input various instructions to the abnormality determination device 10.

[0041] The output device 18 includes a display device such as an LCD or an organic EL display, a printer, etc., and is configured to be able to output abnormality determination results for components inside the vehicle VH, such as the mail seat SH. The output device 18 may also include other components such as a speaker, a light, etc.

[0042] The input device 17 and the output device 18 may be integrally configured, such as a touch panel, or may be connected as external devices to the abnormality determination device 10, which is a computer or the like.

[0043] On the other hand, at least one of the reading device 30 and the imaging device 40 may be built into the abnormality determination device 10 and configured integrally with the abnormality determination device 10.

[0044] Next, the functional configuration of the abnormality determination device 10 will be described with reference to FIG.

[0045] Fig. 3 is a block diagram showing an example of the functional configuration of the abnormality determination device 10 according to the embodiment. As shown in Fig. 3, the abnormality determination device 10 includes, as functional units, a vehicle model information acquisition unit 101, a vehicle model information conversion unit 102, an image acquisition unit 103, a determination unit 104, a position information acquisition unit 105, a command unit 106, an input unit 107, an output unit 108, a communication unit 109, and a storage unit 110.

[0046] These functional units are realized in the abnormality determination device 10 by, for example, expanding a program stored in the ROM 12 or the like into the RAM 13 and executing the program by the CPU 14.

[0047] The vehicle model information acquisition unit 101, which serves as an object information acquisition unit, acquires vehicle model information of the vehicle VH being transported on the production line, read by the reading device 30, for example, in image data format, from the reading device 30. The vehicle model information acquisition unit 101 is realized, for example, by the I / F 15 that operates under the control of the above-mentioned CPU 14.

[0048] The vehicle model information conversion unit 102, which serves as an object information conversion unit, converts the vehicle model information in image data format into character data using, for example, an optical character recognition (OCR) function. OCR is a function that converts image data read by, for example, a scanner into character data.

[0049] The vehicle model information conversion unit 102 is realized by, for example, the above-mentioned CPU 14 executing a program.

[0050] The image acquisition unit 103 acquires information on a plurality of images of the vehicle VH at different positions captured by the imaging device 40. The image acquisition unit 103 is realized by, for example, the I / F 15 that operates under the control of the CPU 14 described above.

[0051] The determination unit 104 identifies the vehicle VH currently being transported on the production line based on the vehicle model information obtained by the vehicle model information acquisition unit 101 and the vehicle model information conversion unit 102. The determination unit 104 also analyzes the image captured by the imaging device 40, and refers to the image information 111 stored in the memory unit 110 to perform an abnormality determination on the mail sheet SH of the vehicle VH included in the captured image according to the vehicle model identified as described above.

[0052] Furthermore, if the determination unit 104 detects an abnormality in a member such as the mail sheet SH, it transmits a warning display instruction to the display device 60 via the communication unit 109 to notify the display device 60 of the abnormality.

[0053] The determination unit 104 is realized by, for example, the above-mentioned CPU 14 executing a program.

[0054] The position information acquisition unit 105 acquires the position information of the vehicle VH, which is the detection result of the position detection device 50, from the position detection device 50. The position information acquisition unit 105 is realized by, for example, the I / F 15 that operates under the control of the CPU 14 described above.

[0055] The command unit 106 controls the reading of vehicle model information by the reading device 30. Furthermore, based on the detection result of the position detection device 50, the command unit 106 causes the imaging device 40 to capture images of the interior of the vehicle VH each time the vehicle VH reaches one of a plurality of predetermined positions.

[0056] The command unit 106 is realized by, for example, the above-mentioned CPU 14 executing a program and the I / F 15 operating under the control of the CPU 14.

[0057] The input unit 107 accepts various instructions input by an operator to the abnormality determination device 10. The input unit 107 is realized, for example, by the input device 17 that operates under the control of the above-mentioned CPU 14. The output unit 108 outputs the abnormality determination result of the mail sheet SH and the like inside the vehicle VH. The output unit 108 is realized, for example, by the output device 18 that operates under the control of the above-mentioned CPU 14.

[0058] When an abnormality is found in the mail sheet SH, the communication unit 109 transmits the warning display instruction generated by the judgment unit 104 to the display device 60 via the network NT. The display device 60, which has received the warning display instruction from the abnormality judgment device 10, displays that an abnormality has occurred in the mail sheet SH. The communication unit 109 is realized, for example, by the communication unit 109 operating under the control of the CPU 14 described above.

[0059] The storage unit 110 stores image information 111. The image information 111 is, for example, a trained model constructed by machine learning. More specifically, the image information 111 is constructed using various image data of a mail sheet SH for each vehicle model as training data, and is stored in the storage unit 110 in a format that can be referenced and executed by, for example, the determination unit 104.

[0060] As a result, based on the image information 111, the judgment unit 104 can determine whether or not there are any abnormalities in the mail sheets SH temporarily installed inside the vehicle VH, such as errors in the type, number, orientation, and placement of the mail sheets SH for each vehicle type, whether the mail sheets SH have been forgotten to be installed, and whether the mail sheets SH are misaligned or torn.

[0061] The storage unit 110 is realized by, for example, the auxiliary storage device 11 that operates under the control of the CPU 14 described above.

[0062] (Example of abnormality detection system function) Next, the details of the functions of the abnormality determination system 1 will be described with reference to Fig. 4 to Fig. 6. Fig. 4 to Fig. 6 are schematic diagrams exemplarily showing how the abnormality determination system 1 according to the embodiment determines an abnormality.

[0063] As described above, the abnormality determination system 1 performs the abnormality determination, for example, before heat sealing, that is, on the vehicle VH on which the unfixed mail sheet SH is temporarily installed. In this case, it is preferable to perform the abnormality determination immediately before the mail sheet SH is fixed. This is because, as described above, abnormalities such as misalignment can occur in the mail sheet SH even when the vehicle VH is transported.

[0064] 4, information about the vehicle type of the vehicle VH currently being transported on the belt conveyor CNV is displayed on the display device 60. The command unit 106 of the abnormality determination device 10 sends a command to the reading device 30 to read the vehicle type information of the vehicle VH displayed on the display device 60.

[0065] The vehicle model information acquisition unit 101 of the abnormality determination device 10 acquires the vehicle model information read by the reading device 30. The vehicle model information conversion unit 102 converts the vehicle model information read by the reading device 30 into character data using an OCR function or the like.

[0066] Furthermore, when the vehicle VH is transported on the belt conveyor CNV, the position detection device 50a, which is the most upstream of the multiple position detection devices 50a, 50b, and 50c, first detects the vehicle VH. When the position detection device 50a detects that the vehicle VH has reached a predetermined position, the command unit 106 sends a command to the imaging device 40 to capture an image of the interior of the vehicle VH at the predetermined position detected by the position detection device 50a.

[0067] The image captured by the imaging device 40 at the predetermined position detected by the position detection device 50a includes, for example, the sheet metal SHa, SHb, and SHc temporarily installed on the floor panel under the front seats. In other words, the installation position of the position detection device 50a is adjusted so that when the vehicle VH reaches a position where the vicinity of the floor surface under the front seats on the floor panel is included in the angle of view of the imaging device 40, the position detection device 50a detects this.

[0068] The determination unit 104 of the abnormality determination device 10 identifies the vehicle VH currently being transported on the production line based on the obtained vehicle type information.

[0069] In addition, based on the image information 111 stored in the memory unit 110, the judgment unit 104 evaluates the state of the mail sheets SHa, SHb, and SHc contained in the image captured by the imaging device 40 according to the model of the vehicle VH, and makes an abnormality judgment for these mail sheets SHa, SHb, and SHc.

[0070] In the example of Figure 4, it is assumed that the mail sheets SHa, SHb, and SHc inside the vehicle VH have no abnormalities such as incorrect model, number, orientation, or placement, or that the mail sheets SHa, SHb, and SHc have been left out of place, are misaligned, or are torn. The determination unit 104 determines that these mail sheets SHa, SHb, and SHc are normal. In this case, the display device 60 maintains a quality display indicating that the quality of the mail sheets SHa, SHb, and SHc is normal.

[0071] 5, as the vehicle VH moves on the belt conveyor CNV, the central position detection device 50b among the multiple position detection devices 50a, 50b, and 50c detects the vehicle VH. When the position detection device 50b detects that the vehicle VH has reached a predetermined position, the command unit 106 sends a command to the imaging device 40 to capture an image of the interior of the vehicle VH at the predetermined position detected by the position detection device 50b.

[0072] The image captured by the imaging device 40 at the predetermined position detected by the position detection device 50b includes, for example, the sheet metal SHd, SHe, and SHf temporarily installed on the floor panel under the rear seats. In other words, the installation position of the position detection device 50b is adjusted so that when the vehicle VH reaches a position where the vicinity of the floor surface under the rear seats on the floor panel is included in the angle of view of the imaging device 40, the position detection device 50b detects this.

[0073] Based on the image information 111 stored in the memory unit 110, the judgment unit 104 evaluates the state of the mail sheets SHd, SHe, and SHf contained in the image captured by the imaging device 40 according to the vehicle model of the vehicle VH, and makes an abnormality judgment on these mail sheets SHd, SHe, and SHf.

[0074] In the example of Figure 5, among the mail sheets SHd, SHe, and SHf inside the vehicle VH, it is assumed that the mail sheet SHd has no abnormalities such as errors in model, number, orientation, or placement, misalignment, forgetting to install the mail sheet SHd, or tears. On the other hand, it is assumed that the mail sheet SHe has tears TR in multiple places, and the mail sheet SHf has a misalignment.

[0075] The determining unit 104 determines that the mail sheet SHd is normal among these mail sheets SHd, SHe, and SHf, and also determines that an abnormality has occurred in the mail sheets SHe and SHf.

[0076] In this way, if an abnormality is found in at least one of the target mail sheets SHd, SHe, and SHf, the determination unit 104 generates a warning display instruction indicating that an abnormality has occurred in any of SHd, SHe, and SHf. The communication unit 109 transmits the warning display instruction generated by the determination unit 104 to the display device 60.

[0077] In accordance with the warning display instruction received from the abnormality determination device 10, the display device 60 switches the display indicating the quality to a display indicating that the quality of any one of SHd, SHe, and SHf is abnormal.

[0078] 6, as the vehicle VH moves further on the belt conveyor CNV, the position detection device 50c, which is the most downstream of the multiple position detection devices 50a, 50b, and 50c, detects the vehicle VH. When the position detection device 50c detects that the vehicle VH has reached a predetermined position, the command unit 106 sends a command to the imaging device 40 to capture an image of the interior of the vehicle VH at the predetermined position detected by the position detection device 50c.

[0079] The image captured by the imaging device 40 at the predetermined position detected by the position detection device 50c includes, for example, the meridians SHg, SHh, and SHi temporarily installed on the floor of the deck. In other words, the installation position of the position detection device 50c is adjusted so that when the vehicle VH reaches a position where the vicinity of the deck floor is included in the angle of view of the imaging device 40, the position detection device 50c detects this.

[0080] Based on the image information 111 stored in the memory unit 110, the judgment unit 104 evaluates the state of the mail sheets SHg, SHh, SHi contained in the image captured by the imaging device 40 according to the vehicle model of the vehicle VH, and judges whether these mail sheets SHg, SHh, SHi are abnormal.

[0081] In the example of Figure 6, it is assumed that the mail sheets SHg, SHh, and SHi inside the vehicle VH have no abnormalities such as incorrect model, number, orientation, or placement, or that the mail sheets SHg, SHh, and SHi have been left out of place, are misaligned, or are torn. The determination unit 104 determines that these mail sheets SHg, SHh, and SHi are normal. In this case, the quality display on the display device 60 indicates that the quality of the mail sheets SHg, SHh, and SHi is normal.

[0082] In this way, the abnormality determination system 1 appropriately identifies the position of the vehicle VH using the multiple position detection devices 50a, 50b, and 50c. Furthermore, the imaging devices 40 capture images of the interior of the vehicle VH at the multiple identified positions. Furthermore, based on these images, the determination unit 104 performs abnormality determination on the mail sheets SHa to SHi that are temporarily installed throughout the vehicle VH.

[0083] The number and arrangement of the position detection devices 50 are not limited to the above example and are arbitrary. The number and arrangement of the position detection devices 50 can be adjusted as appropriate depending on the overall length and shape of the vehicle VH, as well as the number and arrangement of the mail seats SH within the vehicle VH. By adjusting the number and arrangement of the position detection devices 50, it is possible to capture images including various positions in the vehicle VH in addition to the areas near the front and rear seats on the floor panel and near the deck, as described above.

[0084] Thereafter, the output unit 108 of the abnormality determination device 10 outputs the abnormality determination result for the vehicle VH. The abnormality determination result may be displayed on a monitor or the like constituting the output unit 108, or may be printed out by a printer or the like constituting the output unit 108, for example.

[0085] However, the means and form of output of the abnormality determination result by the output unit 108 are not limited to the above examples. The output unit 108 may output the abnormality determination result as a warning sound or voice from a speaker or the like, or may output the abnormality determination result by turning on or blinking a light.

[0086] (Example of processing by an abnormality detection system) Next, an example of the abnormality determination process by the abnormality determination system 1 according to the embodiment will be described with reference to Fig. 7. Fig. 7 is a flow chart showing an example of the procedure of the abnormality determination process by the abnormality determination system 1 according to the embodiment.

[0087] As shown in FIG. 7, when a vehicle VH is transported on a belt conveyor CNV in a production line, the reading device 30 reads the vehicle type information displayed on the display device 60 (step S110).

[0088] The vehicle model information acquisition unit 101 of the abnormality determination device 10 acquires the vehicle model information read by the reading device 30 (step S120). The vehicle model information conversion unit 102 converts the vehicle model information read by the reading device 30 into character data (step S130).

[0089] The position detection devices 50a, 50b, and 50c respond to the vehicle VH that has reached a predetermined position on the belt conveyor CNV (step S140). The abnormality determination device 10 waits until any of the position detection devices 50a, 50b, and 50c detects the vehicle VH and responds (step S140: No).

[0090] When any of the position detection devices 50a, 50b, 50c detects the vehicle VH (step S140: Yes), the command unit 106 of the abnormality determination device 10 causes the imaging device 40 to capture an image of the interior of the vehicle VH (step S150).

[0091] The determination unit 104 of the abnormality determination device 10 identifies the vehicle type of the vehicle VH currently being transported on the production line based on the vehicle information obtained in the process of step S130 described above (step S160).

[0092] However, the determination unit 104 may identify the vehicle type at a timing other than the above, as long as it is after the vehicle information is obtained in the processing of step S130 and before the abnormality determination by the determination unit 204 in the processing of steps S170 and S180 described below.

[0093] The determination unit 104 evaluates the state of the mail sheet SH included in the image captured by the imaging device 40 according to the vehicle model identified in the process of step S160 based on the image information 111 stored in the storage unit 110 (step S170). Furthermore, the determination unit 104 determines whether or not there is an abnormality in the mail sheet SH according to the evaluation content (step S180).

[0094] If an abnormality has occurred in the mail sheet SH (step S180: Yes), the determination unit 104 generates a warning display instruction, and the communication unit 109 transmits the warning display instruction to the display device 60 (step S190). If an abnormality has not occurred in the mail sheet SH (step S180: No), the processing of step S190 is skipped.

[0095] The command unit 106 of the abnormality determination device 10 determines whether the last position detection device 50, that is, the most downstream position detection device 50, responded in the process of step S140 (step S200).

[0096] If the position detection device 50 that responded in the process of step S140 is not the last one (step S200: No), the process is repeated from step S140. If the position detection device 50 that responded in the process of step S140 is the last one (step S200: Yes), the process is ended.

[0097] The output unit 108 of the abnormality determination device 10 collectively outputs the abnormality determination results of all the mail sheets SH temporarily installed in the vehicle VH (step S210).

[0098] However, the output unit 108 may output the abnormality determination result each time the process of step S180 is performed. That is, the output unit 108 may output the abnormality determination result appropriately each time it obtains an abnormality determination result for each of the melt sheets SH temporarily installed on the floor below the front seats of the vehicle VH, on the floor below the rear seats, and on the deck floor.

[0099] This completes the abnormality determination process by the abnormality determination system 1 of the embodiment.

[0100] (Overview) For example, in a vehicle manufacturing line, a vehicle may be transported with components such as a metal sheet temporarily installed and not secured. In this case, the components may become misaligned during the transport of the vehicle. Furthermore, these components may differ in model, number, installation location, and placement depending on the vehicle model, and may be temporarily installed in an incorrect state due to operator error, etc. Therefore, it is preferable to perform an abnormality determination on components before they are finally secured.

[0101] However, in order to detect abnormalities in components temporarily installed inside a vehicle transporting the components on a production line, measures such as installing multiple sensors and imaging devices are required. This increases manufacturing costs and raises concerns that complex analysis may be required. Furthermore, even when multiple sensors and imaging devices are used, it is difficult to collectively detect abnormalities in components inside a vehicle during transport, such as incorrect model, number, orientation, and placement, misalignment, missing mark sheets, and tears.

[0102] According to the embodiment of the abnormality determination system 1, the system is equipped with a reading device 30 that reads the vehicle model information of the vehicle VH displayed on a display device 60 for a vehicle VH in transit with an unfixed mail sheet SH temporarily installed inside, a position detection device 50 that detects that the vehicle VH has reached a predetermined position, an imaging device 40 that captures images of the interior of the vehicle VH at the predetermined position, and an abnormality determination device 10 that performs abnormality determination of the mail sheet SH based on the image captured by the imaging device 40.

[0103] In this way, by performing an abnormality judgment based on vehicle model information obtained as text data using, for example, an OCR function, and images captured while detecting the position of the vehicle VH, abnormalities in components such as the Mel Sheet SH can be detected inexpensively and easily.

[0104] According to the embodiment of the abnormality determination device 10, for each vehicle type corresponding to the vehicle model information, the state of the mail sheet SH contained in the image captured by the imaging device 40 is evaluated based on image information 111 such as a learned model constructed by machine learning, and an abnormality determination is made for the mail sheet SH temporarily installed inside the vehicle VH.

[0105] In this way, by using a trained model constructed by machine learning, for example, it is possible to perform abnormality detection for various vehicle types in a manner similar to human inspection. Therefore, it is possible to perform abnormality detection for components such as Mel Sheet SH, whose specifications vary depending on the vehicle type. Furthermore, it is possible to collectively determine abnormalities such as incorrect model, number, orientation, and placement of Mel Sheet SH for each vehicle type, forgetting to install Mel Sheet SH, and misalignment and tears of Mel Sheet SH for Mel Sheet SH in the vehicle VH being transported.

[0106] Furthermore, with the above configuration, it is possible to determine an abnormality without using, for example, multiple imaging devices and sensors, etc. This not only reduces manufacturing costs but also makes it possible to determine an abnormality without performing complex analysis.

[0107] According to the abnormality determination device 10 of the embodiment, the position detection device 50 detects the vehicle VH at a plurality of positions, and the imaging device 40 captures images of the interior of the vehicle VH at these plurality of positions.

[0108] In this way, by detecting the vehicle VH at multiple positions using multiple position detection devices 50, it is possible to perform abnormality determination for multiple mail sheets SH throughout the entire vehicle VH using, for example, only one inexpensive imaging device 40.

[0109] (Variation) Next, an abnormality determination system according to a modified example of the embodiment will be described with reference to Figures 8 and 9. The abnormality determination system according to the modified example differs from the embodiment described above in that, when determining an abnormality, the abnormality determination system also refers to the environment around the vehicle SH being transported on the production line.

[0110] In the following description, the same components as those in the abnormality determination system 1 of the above-described embodiment are denoted by the same reference numerals, and the description thereof may be omitted.

[0111] Fig. 8 is a block diagram showing an example of the functional configuration of an abnormality determination device 20 according to a modified example of the embodiment. As shown in Fig. 8, the abnormality determination device 20 of the modified example includes an environmental information acquisition unit 207. Furthermore, the abnormality determination device 20 of the modified example includes a determination unit 204 and a storage unit 210 instead of the determination unit 104 and the storage unit 110 of the above-described embodiment.

[0112] The environmental information acquisition unit 207 acquires environmental information about the surroundings of the vehicle VH while it is being transported on the production line. The environmental information about the surroundings of the vehicle VH includes information about the temperature and humidity within the production line, as well as information about obstacles and slopes present within the production line.

[0113] The environmental information acquisition unit 207 acquires information about temperature and humidity in the production line from sensors (not shown) in the factory via the network NT shown in FIG. 2, such as a factory LAN.

[0114] Obstacles in the production line may be factory equipment and fixtures such as processing machines, which may cause the transport route of the vehicle VH, such as the belt conveyor CNV, to be detoured. Such detour routes may have sharp curves, for example, and information about such obstacles is linked to data such as the curvature of the curved detour route and is stored in advance in the equipment information 212 of the storage unit 210.

[0115] The inclination in the production line is the inclination of the transport path of the vehicle VH, such as a belt conveyor CNV. Information about such inclination is linked to data such as the degree of inclination of the inclination, and is stored in advance in the equipment information 212 of the storage unit 210.

[0116] The environmental information acquisition unit 207 reads out the equipment information 212 from the storage unit 210 to acquire information such as obstacles and inclinations within the production line.

[0117] The storage unit 210 stores facility information 212 and correlation information 213 in addition to the image information 111 of the above-described embodiment.

[0118] As described above, the equipment information 212 stores information about obstacles in the production line linked to curvature data of detour routes, etc. Also, as described above, the equipment information 212 stores information about the slope of the production line linked to data such as the slope degree. Note that the situation of obstacles and slopes in the production line differs from factory to factory. Therefore, the equipment information 212 may store information about obstacles and slopes in the production line for each factory.

[0119] The correlation information 213 stores information on the amount of positional deviation corresponding to various conditions such as temperature and humidity in the production line, as well as obstacles and inclinations.

[0120] The temperature and humidity in the production line can affect the positional deviation of the melt sheet SH temporarily installed inside the vehicle VH. For example, melt sheet SH, which is based on asphalt or the like, tends to harden at lower temperatures and dry out at lower humidity. For this reason, the amount of positional deviation of the melt sheet SH tends to increase when the temperature and humidity are lower. The correlation information 213 stores correlation data, such as, for example, the lower the temperature and humidity, the greater the amount of positional deviation of the melt sheet SH.

[0121] If a conveyor belt CNV or the like has a detour due to an obstacle in the production line, for example, when a vehicle VH passes through a curved detour, an unfixed mail sheet SH is likely to become misaligned. The correlation information 213 stores correlation data, such as, for example, the greater the curvature of the detour, the greater the amount of misalignment of the mail sheet SH.

[0122] Furthermore, if the belt conveyor CNV in the production line has an incline, the fixed mail sheet SH is likely to become misaligned when the vehicle VH passes over the incline. The correlation information 213 stores correlation data, such as the fact that the greater the inclination of the conveyance path, the greater the amount of misalignment of the mail sheet SH.

[0123] Similar to the judgment unit 104 in the above-described embodiment, the judgment unit 204 analyzes the image captured by the imaging device 40 and, by referring to the image information 111 stored in the memory unit 210, judges whether there are any abnormalities in the mail sheet SH of the vehicle VH contained in the captured image.

[0124] At this time, the determination unit 204 of the modified example also refers to environmental information such as the temperature and humidity in the production line, and obstacles and inclinations present in the production line.

[0125] In other words, the judgment unit 204 refers to the correlation data between the temperature and humidity, the curvature and inclination of the detour route stored in the correlation information 213 of the memory unit 210, and the amount of positional deviation of the melt sheet SH, and estimates the amount of positional deviation based on the information such as the temperature and humidity, obstacles and inclination in the production line acquired by the environmental information acquisition unit 207, and makes an abnormality judgment based on these estimated values.

[0126] FIG. 9 is a flowchart showing an example of a procedure of an abnormality determination process performed by an abnormality determination system according to a modified example of the embodiment.

[0127] As shown in FIG. 9, the abnormality determination process by the abnormality determination system of the modified example is performed in substantially the same manner as the abnormality determination process of the above-described embodiment.

[0128] More specifically, the processing from steps S110 to S160 shown in FIG. 9 is the same as the processing from steps S110 to S160 shown in FIG. 7 described above.

[0129] That is, the reading device 30 reads the vehicle model information from the display device 60 (step S110), the vehicle model information acquisition unit 101 acquires it (step S120), and the vehicle model information conversion unit 102 converts it into character data (step S130). Also, when any of the position detection devices 50 detects and reacts to the vehicle VH (step S140), the imaging device 40 captures an image of the interior of the vehicle VH (step S150). The determination unit 204 identifies the vehicle model based on the vehicle model information (step S160).

[0130] Here, the environmental information acquisition unit 207 acquires temperature and humidity information from sensors in the factory, for example, via the network NT, and also acquires information such as obstacles and slopes in the production line from the equipment information 212 in the memory unit 210 (step S161).

[0131] However, the environmental information acquisition unit 207 may acquire environmental information at a timing other than the above, as long as it is before the abnormality determination by the determination unit 204. For example, the environmental information acquisition unit 207 may acquire environmental information at a timing when processing in the abnormality determination system of the modified example is started. Alternatively, the environmental information acquisition unit 207 may acquire environmental information in synchronization with the imaging device 40 capturing an image of the inside of the vehicle VH. Alternatively, the environmental information acquisition unit 207 may periodically acquire temperature and humidity information.

[0132] The determination unit 204 evaluates the state of the mail sheet SH included in the image captured by the imaging device 40 based on the image information 111 stored in the storage unit 210 (step S171). Furthermore, the determination unit 204 determines whether or not there is an abnormality in the mail sheet SH depending on the evaluation content (step S181).

[0133] In the processes of steps S171 and S181, the determination unit 204 refers to the correlation information 213 in the storage unit 210, and also uses the environmental information acquired by the environmental information acquisition unit 207 as a determination criterion.

[0134] The subsequent processing of steps S190 to S210 is also similar to the processing of steps S190 to S210 shown in FIG.

[0135] That is, depending on the result of the abnormality determination for the mail sheet SH (step S181), a warning display instruction is generated and transmitted to the display device 60 (step S190), or the process is skipped, and the above process is repeated until the last position detection device 50 shows a response (step S200). The output unit 108 outputs the abnormality determination result (step S210).

[0136] This completes the abnormality determination process by the abnormality determination system of the modified example.

[0137] The abnormality determination device 20 of the modified example includes an environmental information acquisition unit 207 that acquires environmental information indicating the environment around the vehicle VH during transportation, and the determination unit 204 refers to the environmental information when determining an abnormality, thereby enabling the abnormality determination to be performed with higher accuracy.

[0138] In the above-described embodiment and modified examples, abnormality determination is performed on the sheet SH of the vehicle VH. However, the abnormality determination of the above-described embodiment and modified examples can be applied to other cases. That is, the abnormality determination of the above-described embodiment and modified examples can be applied to various unsecured sheet-like members in the vehicle VH. Alternatively, the abnormality determination of the above-described embodiment and modified examples can be applied to various unsecured sheet-like members in objects other than automobiles, such as motorcycles, trains, small boats, and small airplanes.

[0139] The unfixed sheet-like member may be a member that will ultimately be fixed, such as the above-mentioned Mel Sheet SH, or it may be a member that will not ultimately be fixed, such as an insulating material sheet. [Explanation of symbols]

[0140] 1. Abnormality detection system 10,20 Abnormality determination device 30 Reading device 40 Imaging device 50 Position detection device 60 Display device 101 Vehicle information acquisition unit 102 Vehicle information conversion unit 103 Image acquisition unit 104,204 Judgment section 105 Location information acquisition unit 106 Command Department 107 Input section 108 Output section 109 Communications Department 110,210 Storage section 111 Image Information 207 Environmental Information Acquisition Department 212 Equipment information 213 Correlation Information CNV Belt Conveyor SH Merseat VH vehicle

Claims

1. an object information acquisition unit that acquires object information from a reading device that reads object information indicating the type of object displayed on a display device that is fixed to a predetermined position within the production line and that displays the operating status of the production line, for an object that has an unfixed member temporarily installed inside and is being transported along the production line; an image acquisition unit that acquires an image from an imaging device that images the inside of the object that has reached a predetermined position, based on a detection result of a position detection device that detects that the object has reached the predetermined position; a determination unit that evaluates the state of the component included in the image based on a trained model constructed by machine learning for each of the objects according to the object information, and determines whether the component temporarily installed inside the object is abnormal; The position detection device Detecting that the object has reached a plurality of different transport positions; The imaging device is taking images of the inside of the object at the plurality of different transport positions; Abnormality determination device.

2. The display device a liquid crystal display or an organic EL display, The reading device is A camera, a reader with a scanning function, or a recording device that records a video signal output from the display device by a screenshot function. The abnormality determination device according to claim 1 .

3. an environmental information acquisition unit that acquires environmental information indicating an environment around the object being transported; The determination unit When making the abnormality determination, the environmental information is referenced; the environmental information includes at least one of temperature and humidity information around the object and information about detours and inclinations of a transport path along which the object is transported; The abnormality determination device according to claim 1 or 2.

4. a reading device that reads object information indicating the type of object displayed on a display device that is fixed to a predetermined position within the production line and that displays the operating status of the production line, for an object that has an unfixed member temporarily installed inside and is being transported along the production line; a position detection device that detects when the object reaches a predetermined position; an imaging device that images the inside of the object that has reached the predetermined position; an abnormality determination device that determines an abnormality in the member based on the image captured by the imaging device, The position detection device Detecting that the object has reached a plurality of different transport positions; The imaging device is taking images of the inside of the object at the plurality of different transport positions; The abnormality determination device For each of the objects according to the object information, a state of the component included in the image is evaluated based on a trained model constructed by machine learning, and an abnormality determination is made for the component temporarily installed inside the object. Anomaly detection system.

5. For an object with an unfixed member temporarily installed inside and being transported through a production line, read object information indicating the type of the object displayed on a display device that is fixed to a predetermined position within the production line and that displays the operating status of the production line; Detecting that the object has reached a plurality of different transport positions; taking images of the inside of the object at the plurality of different transport positions; For each of the objects according to the object information, a state of the component included in an image of the interior of the object is evaluated based on a trained model constructed by machine learning, and an abnormality determination is made for the component temporarily installed inside the object. Abnormality determination method.

Images