Production support apparatus

Abstract

To provide a production support device capable of efficiently preparing a production process of a production object.SOLUTION: A production support device according to the present invention is a production support device that supports work including a process of attaching a component to a production object in a work environment, the production support device including a virtual viewpoint image generation unit that generates a two-dimensional virtual viewpoint image based on a virtual viewpoint in a virtual three-dimensional space of the work environment including the production object based on work environment information that is information related to the work environment stored in advance and production object design information including a shape and dimensions of the production object, and an image change unit that changes the virtual viewpoint image based on a parameter related to an image.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a production support device. [Background technology]

[0002] There is a known technology that uses AI technology with cameras to automatically inspect for defects such as incorrect or forgotten parts during the assembly process in a factory. Patent Document 1 discloses a technology in a remote monitoring device that links a camera in a virtual environment with an actual camera, in which when the position and direction of the virtual camera's viewpoint are changed within the movable range of the actual camera, an image from the changed viewpoint is displayed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 6-245124 Summary of the Invention [Problem to be solved by the invention]

[0004] When producing a new product in a factory or the like, it is necessary to set the camera's position and angle of view according to the shape of the product, the mounting position of parts on the product, and the work process. The technology disclosed in Patent Document 1 controls a real camera using positioning data from a virtual camera. Therefore, when producing a new product, for example, a vehicle, it is difficult to appropriately set the real camera's position and angle of view until the production process for at least the first vehicle is completed, which means that preparation before the production process begins takes time, resulting in inefficiency.

[0005] An object of the present invention is to provide a production support device that can efficiently prepare for a production process of an object to be produced. [Means for solving the problem]

[0006] The production support device of the present invention is a production support device that supports work including a process of attaching parts to a production object in a work environment, and is equipped with a virtual viewpoint image generation unit that generates a two-dimensional virtual viewpoint image based on a virtual viewpoint in a virtual three-dimensional space of the work environment including the production object, based on work environment information, which is information about the work environment that is stored in advance, and production object design information, which includes the shape and dimensions of the production object, and an image modification unit that modifies the virtual viewpoint image based on parameters related to the image. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide a production support device that can efficiently prepare for the production process of an object to be produced. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating a hardware configuration of a production support device according to an embodiment of the present invention. [Figure 2] 1 is a block diagram showing a functional configuration of a production support device according to an embodiment of the present invention; [Figure 3] FIG. 2 is a diagram showing an example of a parts list in the production support device according to an embodiment of the present invention. [Figure 4] FIG. 2 is a diagram showing an example of a virtual viewpoint image in the production support device according to one embodiment of the present invention. [Figure 5] FIG. 2 is a diagram illustrating an example of a camera control unit in a virtual viewpoint image of a production support device according to an embodiment of the present invention. [Figure 6] 1 is a diagram showing a flow of a production support method executed by a production support device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments will be described with reference to the drawings. Note that the drawings are simplified, and the technical scope of the embodiments should not be narrowly interpreted based on the description in the drawings. Furthermore, identical elements are given the same reference numerals, and redundant explanations will be omitted. Furthermore, in the following embodiments, when referring to the number of elements (including the number, numerical value, amount, range, etc.), unless otherwise specified or when it is clearly limited to a specific number in principle, it is not limited to that specific number, and may be more or less than the specific number.

[0010] Furthermore, in the following embodiments, the components are not necessarily essential unless otherwise specified or considered to be clearly essential in principle. Similarly, in the following embodiments, when referring to the shape, positional relationship, etc. of the components, etc., it is intended to include those that are substantially similar or similar to the shape, etc., unless otherwise specified or considered to be clearly not essential in principle. The same applies to the above numbers, etc. (including numbers, numerical values, amounts, ranges, etc.).

[0011] <Embodiment> <Hardware configuration example> The hardware configuration of a production support device 1 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing the hardware configuration of a production support device 1 according to one embodiment of the present invention.

[0012] As shown in FIG. 1, the production support device 1 includes a CPU 101, a ROM 102, a RAM 103, an HDD (Hard Disk Drive) 104, an HDD (Hard Disk Drive) controller 105, a display 106, an external device connection I / F (Interface) 107, a network I / F 108, a bus line 109, a keyboard 110, a pointing device 111, an optical drive 113, and a media I / F 115.

[0013] Of these, the CPU 101 controls the overall operation of the production support device 1. The ROM 102 stores programs used to drive the CPU 101, such as an IPL (Initial Program Loader). The RAM 103 is used as a work area for the CPU 101. The HDD 104 stores various data such as programs.

[0014] The HDD controller 105 controls the reading and writing of various data from and to the HDD 104 under the control of the CPU 101. The display 106 displays various information such as a cursor, menu, window, text, or image. The external device connection I / F 107 is an interface for connecting various external devices. In this case, the external device is, for example, a USB (Universal Serial Bus) memory, a printer, etc.

[0015] The network I / F 108 is an interface for data communication using a network. The bus line 109 is an address bus, a data bus, etc. for electrically connecting the components such as the CPU 101.

[0016] The keyboard 110 is a type of input means having multiple keys used to input characters, numbers, various instructions, etc. The pointing device 111 is a type of input means used to select and execute various instructions, select a processing target, move a cursor, etc.

[0017] The optical drive 113 controls reading and writing of various data from and to a DVD-RW 112, which is an example of a removable recording medium. The optical drive 113 is not limited to a DVD-RW, and may be a DVD-R or the like. The media I / F 115 controls reading and writing (storing) of data from and to a recording medium 114, such as a flash memory.

[0018] <Example of functional configuration> Next, the functional configuration of the production support device 1 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the production support device 1 according to one embodiment of the present invention. The production support device 1 includes a virtual viewpoint image generation unit 10, an image change unit 20, and a storage unit 30.

[0019] The production support device 1 supports work including a process of attaching parts to a production object in a work environment. The production object is, for example, a vehicle. The work environment is, for example, a factory that produces vehicles. The parts may be, for example, parts that are attached to the vehicle, or parts that are used in the part attachment work. In the following description, the production object is a vehicle, and the work environment is an automobile production factory.

[0020] The virtual viewpoint image generating unit 10 generates a two-dimensional virtual viewpoint image based on work environment information 31, which is information related to the work environment stored in advance, and production object design information 32, which includes the shape and dimensions of a vehicle, which is the production object. The virtual viewpoint image is an image based on a virtual viewpoint in a virtual three-dimensional space of a production factory, which is the work environment including, for example, the vehicle, which is the production object. Note that the virtual viewpoint image generating unit 10 may generate the virtual viewpoint image based on initial values ​​of parameters related to the image.

[0021] The image change unit 20 changes the virtual viewpoint image based on image-related parameters, such as parameters for controlling pan-tilt-zoom operations and parameters indicating the position of the virtual viewpoint.

[0022] The storage unit 30 stores work environment information 31, production object design information 32, and a parts list 33. The storage unit 30 may be configured with the ROM 102, RAM 103, and optical drive 113 such as a DVD-RW (Digital Versatile Disk Rewritable) shown in FIG.

[0023] In addition, the work environment information 31, production object design information 32, and parts list 33 may be stored in other devices, etc. of the production support device 1, and the production support device 1 may be configured to acquire these by downloading them from the other devices, etc.

[0024] The work environment information 31 may be, for example, scan data obtained by 3D scanning a vehicle production factory using a 3D scanner, etc. Based on the obtained scan data, the production factory is displayed in the virtual viewpoint image.

[0025] The production object design information 32 may be, for example, three-dimensional design data of a vehicle, which is the production object. Based on the obtained three-dimensional design data, the vehicle is displayed in the virtual viewpoint image with the production factory as the background.

[0026] 3 is a diagram showing an example of a parts list 33 in the production support device 1 according to one embodiment of the present invention. The image change unit 20 adds parts selected from the parts list 33, which includes pre-stored parts of a vehicle that is the production target, to the virtual viewpoint image. The parts list 33 is, for example, a list as shown in FIG. 3 and may be configured in the format of, for example, CSV data, but is not limited to this.

[0027] As shown in the figure, the parts list 33 may display parts in association with the process name and inspection item in the vehicle production process. The "process name" may be, for example, the name of a process indicating the specific work content in the vehicle production process, or may be displayed as "Process 1" or "Process 2." The "inspection item" may be, for example, information on the inspection item performed on the inspection line. The information on the inspection item may be, for example, whether or not any parts have been omitted from the vehicle being produced. The "part" is, for example, the name of a part to be installed on the vehicle.

[0028] FIG. 4 is a diagram showing an example of a virtual viewpoint image in a production support device 1 according to an embodiment of the present invention. As shown in the figure, the virtual viewpoint image includes, for example, a production factory 2 as a work environment, a vehicle 3 as a production object, multiple virtual cameras 4a, 4b, and 4c, a camera selection unit 5, a work object display unit 6, a screen operation unit 7, and a camera control unit 8. The virtual viewpoint image may further include a part selection unit (not shown) for displaying parts. Note that, hereinafter, the multiple virtual cameras 4a, 4b, and 4c will be simply referred to as "virtual cameras 4" unless there is a need to distinguish them.

[0029] The virtual viewpoint image generator 10 displays the production factory 2 as a foreground model based on the acquired work environment information 31. Then, a virtual viewpoint image representing the view from the virtual viewpoint is generated based on the acquired production object design information 32. The virtual viewpoint image may be generated by, for example, a so-called game engine, a platform for creating video games that renders three-dimensional computer graphics.

[0030] The image of the production factory 2 is generated based on work environment information 31, which is 3D scan data of the work environment. The image of the vehicle 3 is generated based on production object design information 32, which includes the shape and dimensions of the vehicle 3. The image of the vehicle 3 is displayed based on the shape and dimensions based on the position of the virtual viewpoint, etc.

[0031] As shown in the figure, the number of virtual cameras 4a, 4b, and 4c may be three, but is not limited to this and may be any number. Furthermore, the number of virtual cameras 4 can be added as desired. The virtual camera 4 indicates the position of a virtual viewpoint. The position of the virtual camera 4 may be a position where a camera can be installed in the actual production factory 2, but is not limited to this and may be any position.

[0032] The camera selection unit 5 selects a virtual camera 4, which is the position of the virtual viewpoint displayed in the work object display unit 6. In the example shown in the figure, "Camera 1" is selected in the camera selection unit 5, and an image of the virtual viewpoint of virtual camera 4a is displayed in the work object display unit 6. The virtual cameras 4a, 4b, and 4c selected in the camera selection unit 5 may be highlighted.

[0033] The screen operation unit 7 operates the position and angle of view of the vehicle 3 in the virtual viewpoint image. As shown in the figure, the screen operation unit 7 displays the model of the vehicle 3, and the position and angle of view of the production object 3 can be specified using XYZ coordinates.

[0034] The part selection unit (not shown) may be displayed in a table format corresponding to the parts list 33. The part selection unit may be configured to enable selection of parts to be attached to the vehicle 3, for example, in the production process of the vehicle 3. The selected parts are displayed in addition to the virtual viewpoint image. By displaying the selected parts in the virtual viewpoint image in this way, it is possible to check, for example, whether any parts that should be attached in a given process have been omitted.

[0035] 5 is a diagram showing an example of the camera control unit 8 in a virtual viewpoint image of the production support device 1 according to one embodiment of the present invention. As shown in the figure, in the camera control unit 8, the virtual camera 4, which is the position of the virtual viewpoint displayed in the work object display unit 6, is selected as the "camera name."

[0036] The camera control unit 8 can specify the position of the selected virtual camera 4 using XYZ coordinates as parameters indicating the position of the virtual viewpoint, and can also specify the direction to view from the specified position using XYZ coordinates. The image modification unit 20 displays the virtual viewpoint image modified based on the specified position and rotation direction on the work object display unit 6.

[0037] Furthermore, the camera control unit 8 can specify parameters for controlling the pan-tilt-zoom operation of the selected virtual camera 4. Based on the specified parameters, the image change unit 20 displays a virtual viewpoint image corresponding to the pan, tilt, and zoom magnification, which indicates the line of sight from the virtual viewpoint, on the work object display unit 6. Furthermore, the camera control unit 8 can specify the aspect ratio and the angle of view of the virtual viewpoint image.

[0038] The data of the virtual viewpoint image shown as an example in FIG. 4 may be output to a learning device (not shown) together with parameters for controlling the pan-tilt-zoom operation used for generation and parameters indicating the position of the virtual viewpoint. The learning device can perform annotation processing using the acquired virtual viewpoint image data as training data and use it as a learning model. Therefore, the virtual viewpoint image is used as support data in the production process of the same vehicle model, i.e., the same type of production object 3.

[0039] FIG. 6 is a diagram showing the flow of a production support method executed by the production support device 1 according to one embodiment of the present invention.

[0040] The virtual viewpoint image generation unit 10 acquires the work environment information 31, the production object design information 32, and the parts list 33 from the storage unit 30 (S101). The virtual viewpoint image generation unit 10 acquires the parameters related to the image, that is, the parameters for controlling the pan-tilt-zoom operation and the parameters indicating the position of the virtual viewpoint (S102). The virtual viewpoint image generation unit 10 generates a virtual viewpoint image based on the information acquired in the processes of S101 and S102 (S103).

[0041] If the parameters controlling the pan-tilt-zoom operation and the parameters indicating the position of the virtual viewpoint are changed (YES in S104), the image change unit 20 changes the virtual viewpoint image (S105). If the parameters controlling the pan-tilt-zoom operation and the parameters indicating the position of the virtual viewpoint are not changed (NO in S104), the production support device 1 performs the process of S106.

[0042] If a part is selected in the virtual viewpoint image (YES in S106), the image change unit 20 adds the selected part to the virtual viewpoint image (S107). If no part is selected in the virtual viewpoint image (NO in S106), the production support device 1 completes the process.

[0043] These steps implement the production support method according to one embodiment of the present invention. However, the measurement method according to one embodiment of the present invention may include other steps as appropriate depending on the implementation conditions, implementation environment, etc. Also, in the illustrated flow, the order of the processes of S104 and S105 and the processes of S106 and S107 may be reversed.

[0044] <Effects of the production support device 1 according to this embodiment> In the production support device 1 according to this embodiment, the virtual viewpoint image generator 10 generates a virtual viewpoint image including the production factory 2, which is the work environment, and the vehicle 3, which is the object to be produced, based on the work environment information 31 and the object to be produced design information 32. Therefore, the placement and angle of view of the surveillance cameras in the production factory 2 can be set before the production process, such as the prototype stage of the new product, the vehicle 3. Therefore, the production support device 1 according to this embodiment can efficiently prepare for the production process of the object to be produced.

[0045] Furthermore, in the virtual viewpoint image, the display of the vehicle 3 can be switched, its position changed, and the angle of view changed, and the parts to be inspected can also be displayed, which allows for more appropriate setting of the placement and angle of view of the surveillance cameras in the production factory 2.

[0046] Furthermore, the data of the virtual viewpoint image is input to the learning device as training data together with parameters for controlling the pan-tilt-zoom operation used for generation and parameters indicating the position of the virtual viewpoint. Therefore, the virtual viewpoint image can be used to construct an AI learning model, that is, as support data in the production process of a vehicle 3, which is the same type of production object.

[0047] The present invention has been described above in accordance with the above-mentioned embodiments, but the present invention is not limited to the configurations of the above-mentioned embodiments, and naturally includes various modifications, alterations, and combinations that can be made by a person skilled in the art within the scope of the invention claimed in the claims of this application. [Explanation of symbols]

[0048] 1 Production support equipment 10 Virtual viewpoint image generation unit 20 Image change section 30 Storage section 31 Working Environment Information 32 Product design information 33 Parts List

Claims

1. A production support device that supports a task including a step of attaching parts to a production object in a work environment, a virtual viewpoint image generation unit that generates a two-dimensional virtual viewpoint image based on a virtual viewpoint in a virtual three-dimensional space of the work environment including the object to be produced, based on work environment information that is information about the work environment stored in advance and product object design information that includes the shape and dimensions of the object to be produced; an image modification unit that modifies the virtual viewpoint image based on parameters related to the image; A production support device equipped with:

2. The image-related parameters are parameters for controlling pan-tilt-zoom operations and parameters indicating the position of the virtual viewpoint. The production support device according to claim 1.

3. the image change unit adds the part selected from a pre-stored parts list including the part to the virtual viewpoint image. The production support device according to claim 1.

4. The production object is a vehicle. The production support device according to claim 1.

5. A production support method executed by a production support device that supports a task including a step of attaching parts to a production object in a work environment, comprising: generating a two-dimensional virtual viewpoint image based on a virtual viewpoint in a virtual three-dimensional space of the work environment including the object to be produced, based on work environment information, which is information about the work environment stored in advance, and product object design information, which includes the shape and dimensions of the object to be produced; modifying the virtual viewpoint image based on image parameters; A production support method including:

Images