Operation support device and operation support method

Abstract

To provide an operational support device that enables workers to operate transport carts more efficiently and appropriately in the work environment. [Solution] The operation support device according to the present invention comprises: an acquisition unit that acquires an image of the work environment including a transport cart; a state determination unit that performs a process to extract the transport cart included in the image of the work environment by machine learning or the like and determines the state of the transport cart extracted from the image; a change determination unit that determines a change to the transport cart to be arranged based on the production plan of the products to be produced in the work environment and the state of the transport cart; a specification unit that specifies the content of the display for operation of the transport cart to be displayed on the display means provided on the transport cart, depending on whether or not the transport cart has been changed; and an output unit that outputs the result of specifying the content to be displayed on the display means to the transport cart.

Description

Technical Field

[0001] The present invention relates to an operation support device and an operation support method for a transport vehicle.

Background Art

[0002] There is known a technique for providing information that contributes to the efficiency of transportation operations using a transport vehicle such as a cart to a transporter of members in a factory. In Patent Document 1, there is disclosed a technique for displaying an estimated free time of a cart calculated based on information indicating a manufacturing apparatus to which a member is to be transported by the cart, the record of the transported member, the remaining amount of the member to be transported, and the time from when the member is loaded until it is unloaded.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the technique disclosed in Patent Document 1, for example, due to unforeseen defects or troubles such as shortages of supplied parts, a delay in the process with respect to the plan occurs, so if the transport cart is supplied as originally planned, there is a risk of problems such as redoing the work. Therefore, in the working environment, it is desirable that an operator can operate an appropriate transport cart more efficiently.

[0005] An object of the present invention is to provide an operation support device that enables an operator to more efficiently operate an appropriate transport vehicle in a working environment.

Means for Solving the Problems

[0006] The operation support device according to the present invention comprises: an acquisition unit that acquires an image of the work environment including a transport cart; a state determination unit that processes the image of the work environment to extract the transport cart and determines the state of the transport cart extracted from the image; a change determination unit that determines a change to the transport cart to be arranged based on the production plan of the products to be produced in the work environment and the state of the transport cart; an identification unit that identifies the content of the display for operation of the transport cart to be displayed on the display means provided on the transport cart, depending on whether or not the transport cart has been changed; and an output unit that outputs the result of identifying the content to be displayed on the display means to the transport cart. [Effects of the Invention]

[0007] The present invention provides an operational support device that enables workers to operate transport carts more efficiently and appropriately in a work environment. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing the functional configuration of an operational support device according to one embodiment of the present invention. [Figure 2] This figure shows an example of a display in the display means of a transport trolley according to one embodiment of the present invention. [Figure 3] This figure shows a flow chart of an operation support method performed by an operation support device according to one embodiment of the present invention. [Figure 4] This figure shows the processing flow of Example 1 of the operation support device according to one embodiment of the present invention. [Figure 5] This figure shows the processing flow of Example 2 of the operation support device according to one embodiment of the present invention. [Figure 6] This figure shows the processing flow for Example 3 of the operation support device according to one embodiment of the present invention. [Modes for carrying out the invention]

[0009] The embodiments will be described below with reference to the drawings. Note that the drawings are simplified, and the technical scope of the embodiments should not be narrowly interpreted based on their depiction. Furthermore, the same elements are denoted by the same reference numerals, and redundant explanations are omitted. Also, in the following embodiments, when referring to the number of elements (including quantity, numerical value, amount, range, etc.), unless specifically stated or clearly limited to a particular number in principle, the number is not limited to that particular number and may be greater than or less than that number.

[0010] Furthermore, in the following embodiments, the components are not necessarily essential unless otherwise explicitly stated or considered to be clearly essential in principle. Similarly, in the following embodiments, when referring to the shape, positional relationship, etc., of the components, etc., it shall include those that substantially approximate or resemble their shape, etc., unless otherwise explicitly stated or considered to be clearly not essential in principle. The same applies to the numbers, etc. (including the number of items, numerical values, quantities, ranges, etc.).

[0011] [Embodiment] <Configuration of Operation Support Device 10> Figure 1 is a block diagram showing the functional configuration of an operation support device 10 according to one embodiment of the present invention. As shown in the figure, the operation support device 10 comprises an acquisition unit 11, a status determination unit 12, a change determination unit 13, a specification unit 14, and an output unit 15. The operation support device 10 is configured to communicate with a transport cart 20 and terminal devices held by the worker. The transport cart 20 is also equipped with a display means 21.

[0012] The operation support device 10 comprises a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), HDD (Hard Disk Drive), and an I / F (Interface) as its hardware configuration. These are electrically connected to each other via a bus. The CPU controls the operation of the operation support device 10. The ROM stores programs and other data executed by the CPU. The RAM is used as the CPU's workspace. The HDD stores various data such as programs. The I / F is an interface for inputting and outputting various signals and data to and from external devices.

[0013] The communication means provided by the operation support device 10 and the transport cart 20 can be wired or wireless. The communication means may be short-range wireless communication between the operation support device 10 and each part. Various short-range wireless communication methods with short communication distances can be used as short-range wireless communication. For example, communication methods conforming to any communication standard established by IEEE, ISO, IEC, etc. can be used as short-range wireless communication. Examples include Wi-Fi (registered trademark), Bluetooth (registered trademark), and ZigBee (registered trademark) which are used for short-range wireless communication. In addition to these, technologies such as RFID (Radio Frequency Identification) and DSRC (Dedicated Short Range Communication) may be used as technologies for short-range wireless communication.

[0014] The acquisition unit 11 acquires images of the work environment, including the transport cart 20. The work environment may be, for example, an automobile manufacturing plant, or a factory where production, processing, and repair of products are carried out, but is not limited to this. The work environment may also include multiple buildings and multiple rooms. The work environment may be equipped with communication means such as a wireless LAN (Local Area Network) access point so that the operation support device 10 and the transport cart 20 can communicate with each other.

[0015] The image of the working environment is captured by an imaging device (not shown) provided in the working environment. The imaging device is arranged so as to be able to capture the carrier truck 20 in the working environment. The captured image is output to the operation support device 10. A plurality of imaging devices may be provided in the working environment. The imaging device includes image detection elements such as a CCD (Charge Coupled Device) and a CMOS (Complementary Metal Oxide Semiconductor).

[0016] The carrier truck 20 is a general-purpose truck used, for example, in the working environment to move various boxes (boxes of various sizes) for transporting various parts. Also, the carrier truck 20 may be configured to be able to move up and down by connection and engagement with a lifting device provided separately from the carrier truck 20.

[0017] The carrier truck 20 includes display means 21. The display means 21 performs display for the operation of the carrier truck 20. A device composed of an LED (Light-Emitting Diode) or the like is used as the display means 21.

[0018] The state determination unit 12 extracts the carrier truck 20 included in the image of the working environment and determines the state of the extracted carrier truck 20. The state determination unit 12 extracts the carrier truck 20 from the image of the working environment using an existing object detection algorithm. As the existing object detection algorithm, for example, YOLO (YOU Only Look Once) is used, but it is not limited to this.

[0019] The state determination unit 12 may determine the state of the carrier truck 20 extracted from the image of the working environment using a learned model that performs the process of extracting the carrier truck 20 included in the image of the working environment. The learned model is generated, for example, by machine learning with the image of the working environment as input data and information regarding the appearance of the carrier truck 20 as the correct label.

[0020] The state of the transport cart 20 determined by the state determination unit 12 is a state related to the loading of parts necessary for the production of products on the transport cart 20. Note that the parts necessary for the production of products are the parts that make up the products, but are not limited to this, and may include parts that are only used in the production process of the products and do not make up the products. Also, for example, when the product is an automobile, the parts are a transaxle, a transmission, an electronic control unit, gears, bolts, nuts, etc. Further, the state related to the loading of parts is a state indicating whether or not parts are loaded on the transport cart 20, but is not limited to this, and may be information indicating the degree to which parts are loaded on the transport cart 20.

[0021] Also, the state of the transport cart 20 determined by the state determination unit 12 is a state related to the presence or absence of the transport cart 20 in a predetermined area. The predetermined area is, for example, a place where parts or the like transported by the transport cart 20 are loaded and unloaded, and parts are loaded. In the working environment, a plurality of predetermined areas may be provided.

[0022] Also, the acquisition unit 11 acquires the start time and the completion time of the work related to a predetermined process using the parts loaded on the transport cart 20 based on the determination result related to the presence or absence of the transport cart 20 in the predetermined area determined by the state determination unit 12. The work related to the predetermined process is work performed by an operator and work performed by a robot or the like collaborating with the operator in the process of producing a product in the working environment. The predetermined process is, for example, various processes such as the replacement work of the transport cart 20.

[0023] The information on the start time and the completion time of the work acquired by the acquisition unit 11 can be stored in a storage means (not shown) of the operation support device 10, and can be stored in another device configured to be communicable with the operation support device 10, and a server on the cloud or the like. The storage means is constituted by a ROM, a RAM, etc. By storing the start time and the completion time of the work, the work results of each process will be accumulated, which can contribute to reducing the energy unit in the working environment.

[0024] The change determination unit 13 determines whether to change the transport cart 20 to be arranged based on the production plan for the products scheduled to be produced in the work environment and the status of the transport cart 20. When the change determination unit 13 determines that the transport cart 20 initially scheduled to be arranged based on the production plan should be changed, it displays this fact on the display means 21, etc., so that workers can easily recognize the change in the transport cart 20.

[0025] The identification unit 14 determines whether or not the transport cart 20 has been changed, as determined by the change determination unit 13, and specifies the content to be displayed on the display means 21 provided on the transport cart 20. The content to be displayed on the display means 21 may include, for example, identification information to identify the transport cart 20, the name of the process in which the loaded parts are used, and information regarding the time when the transport cart 20 is positioned. Depending on whether or not the transport cart 20 has been changed, the identification unit 14 determines the content to be displayed on the display means 21, such as changing the color of at least a part of the display on the display means 21 of the transport cart 20.

[0026] The output unit 15 outputs the result of specifying the content to be displayed on the display means 21 to the transport cart 20. The output unit 15 may also output the result of specifying the content to be displayed on the display means 21 to the worker's terminal device. In this case, the display means 21 of the transport cart 20 and the worker's terminal device will display based on the display content specified by the specification unit 14. The display content for the display means 21, etc., may be stored in advance in the operation support device 10 as multiple patterns.

[0027] Figure 2 shows an example of a display in the display means 21 of a transport trolley 20 according to one embodiment of the present invention. In Figure 2, (a) shows an example of a display when the transport trolley 20 is ordered according to the production plan, and (b) shows an example of a display when the transport trolley 20 initially scheduled to be ordered based on the production plan is changed.

[0028] In the illustrated table, "Vehicle No." indicates identification information for the transport trolley 20, "Process" indicates the name of the process in which the loaded parts are used, and "Time" indicates information about the time when the transport trolley 20 is positioned. A two-dimensional code 22 is displayed in the time column. The two-dimensional code 22 may also be printed as information indicating the destination. The two-dimensional code 22 is a matrix-type code such as a QR code (registered trademark), but is not limited to this.

[0029] As shown in Figure 2(b), the specific unit 14 can specify the display content such that, when there is a change in the transport trolley 20, it changes the color of the time column, which is at least part of the display of the display means 21. Note that the display change is not limited to the illustrated example, and various forms can be used, such as changing the color of the text.

[0030] <Operation support method> Figure 3 is a diagram showing the flow of the operation support method performed by the operation support device 10 according to one embodiment of the present invention.

[0031] The acquisition unit 11 acquires an image of the work environment including the transport cart 20 (S101). The state determination unit 12 extracts the transport cart 20 from the image of the work environment (S102). The state determination unit 12 determines the state of the extracted transport cart 20 (S103).

[0032] The change determination unit 13 determines whether to change the transport cart 20 to be arranged based on the production plan of the products scheduled to be produced in the work environment and the status of the transport cart 20 (S104). The identification unit 14 determines whether or not to change the transport cart 20 and specifies the content to be displayed on the display means 21 that is provided on the transport cart 20 and is used to display information for the operation of the transport cart 20 (S105).

[0033] The output unit 15 outputs the result of specifying the content to be displayed on the display means 21 for the transport cart 20 (S106). Through processing by the operation support device 10, the worker can recognize the change in the transport cart 20 via the display means 21 and continue the work smoothly.

[0034] These steps enable the implementation of an operational support method according to one embodiment of the present invention. However, the operational support method according to one embodiment of the present invention may include other steps as appropriate, depending on the implementation conditions and environment.

[0035] In the following, an example of the processing performed by the operation support device 10 according to this embodiment will be described using Figures 4 to 6. In the following embodiment, an automobile body welding process is carried out in an automobile manufacturing plant, which is the work environment, and a place for transport carts 20 is set up on the line side of the process.

[0036] Furthermore, the process of sorting parts onto the transport cart 20 and the body welding process are carried out in a separate building. The transport cart 20 is automatically towed, loaded with parts, and transported. When the parts loaded onto the transport cart 20 are removed and the transport cart 20 becomes empty, it is replaced with the transport cart 20 loaded with parts. The empty cart then moves to the location where the sorting process is carried out.

[0037] [Example 1] Figure 4 is a diagram showing the processing flow of an embodiment 1 of the operation support device 10 according to one embodiment of the present invention. In this embodiment, the display means 21 of the transport cart 20 flashes blue when the transport cart 20 is operating according to the production plan, and flashes red when the operation sequence of the transport cart 20 is different from the production plan.

[0038] The acquisition unit 11 acquires an image of the work environment including the transport cart 20 (S201). The state determination unit 12 extracts the transport cart 20 included in the image of the work environment (S202). The state determination unit 12 determines that the transport cart 20 in a predetermined area is an empty cart that does not have any parts loaded on it (S203).

[0039] The change determination unit 13 checks the production plan (S204), and if it determines that there are no changes (YES in S205), the identification unit 14 specifies that the display on the display means 21 of the transport cart 20 should flash blue (S206). The output unit 15 outputs the result of specifying the content to be displayed on the display means 21 of the transport cart 20 (S207).

[0040] If the change determination unit 13 determines that there is a change in the production plan (NO in S205), the identification unit 14 specifies that the display on the display means 21 of the transport cart 20 should flash red (S208). The output unit 15 outputs the result of specifying the content to be displayed on the display means 21 of the transport cart 20 (S207).

[0041] By checking the display means 21, the worker can confirm whether the transport carts 20 are operating according to the production plan or if the order of operation of the transport carts 20 differs from the production plan. The worker can then take the transport carts 20 out of storage and operate them to the destination, and replace the empty transport carts 20 with the actual carts loaded with parts.

[0042] [Example 2] Figure 5 is a diagram showing the processing flow of Example 2 of the operation support device 10 according to one embodiment of the present invention.

[0043] The acquisition unit 11 acquires an image of the work environment including the transport cart 20 (S301). The state determination unit 12 extracts the transport cart 20 included in the image of the work environment (S302). The state determination unit 12 determines that the transport cart 20 in a predetermined area is an empty cart that does not have any parts loaded on it (S303).

[0044] If the transport cart 20 is not empty (NO in S303), the acquisition unit 11 does not acquire the start time and completion time of the work because the parts are being supplied in the work environment (S304). If the status determination unit 12 determines that the transport cart 20 is empty (YES in S303), when the transport cart 20 replacement work is performed, it extracts the transport cart 20 from the image of the work environment (S305).

[0045] If the status determination unit 12 determines that the empty transport cart 20 has not started moving (NO in S306), it returns to the process in step S305. If the status determination unit 12 determines that the empty transport cart 20 has started moving (YES in S306), the acquisition unit 11 acquires the start time of the transport cart 20 replacement operation (S307).

[0046] The status determination unit 12 extracts the transport cart 20 from the image of the work environment (S308). If the status determination unit 12 determines that the supply of the transport cart 20, which is the actual cart, has not been completed (NO in S309), it returns to the process of step S308. If the status determination unit 12 determines that the supply of the transport cart 20, which is the actual cart, has been completed (YES in S309), the acquisition unit 11 acquires the completion time of the transport cart 20 replacement work (S310).

[0047] As in this embodiment, work performance data is accumulated by storing information regarding the start and completion times of the work acquired by the acquisition unit 11.

[0048] [Example 3] Figure 6 is a diagram showing the processing flow of Example 3 of the operation support device 10 according to one embodiment of the present invention.

[0049] The acquisition unit 11 acquires an image of the work environment including the transport cart 20 (S401). The state determination unit 12 extracts the transport cart 20 included in the image of the work environment (S402). If the state determination unit 12 determines that the operation of swapping empty carts and actual carts has not been completed for the extracted transport cart 20 (NO in S403), it continues the process in step S403.

[0050] If the status determination unit 12 determines that the exchange of the empty trolley and the actual trolley has been completed (YES in S403), the identification unit 14 specifies the display content to reset the display on the display means 21 of the transport trolley 20 whose exchange has been completed (S404). The output unit 15 outputs an instruction to the transport trolley 20 to reset the display on the display means 21 (S405).

[0051] If the display on the display means 21 of the transport cart 20 at the sorting process location has not been reset (NO in S406), the process returns to step S405, and the output unit 15 outputs a reset instruction again. If the display on the display means 21 of the transport cart 20 at the sorting process location has been reset (YES in S406), the identification unit 14 identifies the transport cart 20 related to the next sorting process and identifies the content to be displayed on the display means 21 (S407). Note that confirmation that the display on the display means 21 of the transport cart 20 at the sorting process location has been reset may be performed when the transport cart 20 enters the access point installed at the sorting process location.

[0052] As in this embodiment, when the display means 21 of the transport cart 20 after the replacement work is completed is reset, the worker can easily distinguish the transport cart 20 when it is returned to the sorting process after being collected as an empty cart. In addition, the effort required for the worker to reset the display means 21 of the transport cart 20 and the display of their own terminal device is eliminated.

[0053] The examples of processes executed by the operation support device 10 according to this embodiment are not limited to the embodiment described with reference to Figures 4-6, but various other processes can be employed.

[0054] <Effects of the operation support device 10 according to this embodiment> In this embodiment, the operation support device 10 extracts the transport cart 20 included in the image of the work environment and determines the status of the transport cart 20. Then, based on the production plan and the status of the transport cart 20, the operation support device 10 determines whether or not there is a change in the transport cart 20 to be ordered and changes the display on the display means 21 of the transport cart 20.

[0055] Therefore, it is possible to suppress discrepancies in the transport cart 20 caused by unexpected problems such as shortages of supplied parts, and workers can continue their work smoothly. Accordingly, the operation support device 10 according to this embodiment allows workers to operate the transport cart 20 more efficiently and appropriately in the work environment.

[0056] Although the present invention has been described above in accordance with the embodiments described above, the present invention is not limited to the configuration of the embodiments described above, and of course includes various modifications, alterations, and combinations that can be made by a person skilled in the art within the scope of the claims of the present patent application. [Explanation of Symbols]

[0057] 10 Operation support equipment 11 Acquisition Department 12 State determination unit 13 Change Determination Unit 14 Specific section 15 Output section 20 Transport carts 21 Display means

Claims

1. An acquisition unit that acquires images of the work environment, including the transport cart, A state determination unit performs a process to extract the transport cart included in the image of the work environment and determines the state of the transport cart extracted from the image, A change determination unit determines, based on the production plan of the products to be produced in the aforementioned work environment and the status of the transport trolley, to determine a change in the transport trolley to be arranged based on the production plan, A specific unit that identifies the content of the display for the operation of the transport trolley to be displayed on the display means provided on the transport trolley, depending on whether or not the transport trolley has been changed, The transport trolley includes an output unit that outputs the result of specifying the content to be displayed on the display means, An operational support device equipped with the following features.

2. The state of the transport trolley determined by the state determination unit is the state of the transport trolley relating to the loading of parts necessary for the production of the product. The operational support device according to claim 1.

3. The state of the transport trolley determined by the state determination unit is a state relating to the presence or absence of the transport trolley in a predetermined area. The acquisition unit acquires the start time and completion time of work related to a predetermined process using the parts loaded on the transport trolley, based on the determination result regarding the presence or absence of the transport trolley in the predetermined area. The operational support device according to claim 1.

4. The specified unit changes the color of at least a portion of the display on the display means of the transport trolley depending on whether or not the transport trolley has been changed. The operational support device according to claim 1.

5. An operational support method performed by an operational support device, The steps include: acquiring images of the work environment, including the transport cart; The process involves extracting the transport cart from the image of the work environment and determining the state of the transport cart extracted from the image. A step of determining a change to the transport cart to be arranged based on the production plan, based on the production plan of the products to be produced in the aforementioned work environment and the status of the transport cart, Depending on whether or not the transport trolley has been changed, the step is to specify the content of the display for the operation of the transport trolley to be displayed on the display means provided on the transport trolley, The steps include outputting the result of specifying the content to be displayed on the display means for the transport trolley, Operational support methods including those mentioned above.

Images