Information processing device and information processing method

Abstract

To provide a technology capable of identifying a moving object using data output from a camera.SOLUTION: An information processing device is provided with: an acquisition unit for acquiring fixed-point data output from a fixed-point camera that captures a bird's-eye view of the location where a moving object is manufactured, and operator gaze data output from a wearable camera worn by an operator engaged in the manufacture of the moving object at a manufacturing site, a wearable camera capturing areas according to the operator's gaze; a recognition unit for recognizing moving object identification information attached to the exterior of the moving object from the operator gaze data; a detection unit for detecting the moving object from fixed-point data; and an identification unit for identifying the moving object in association with the moving object identification information recognized from the operator gaze data to the moving object detected in the fixed-point data by the detection unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device and an information processing method. [Background technology]

[0002] A technology has been known in the past that uses image data output from a camera installed on the production line to identify workpieces flowing on the production line (Patent Document 1). In this technology, an image processing device that acquires the image data assigns a unique identification number to each workpiece in the image data, thereby identifying the workpiece. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-55334 Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional technology, an image processing device assigns a unique identification number to a workpiece in image data. Therefore, when multiple image processing devices are used in the workpiece production process, each image processing device may assign a different identification number to the same workpiece. If different identification numbers are assigned to the same workpiece, the workpiece may not be uniquely identified during the workpiece production process. Therefore, the inventors of the present application came up with the idea of ​​identifying the workpiece using identification information previously assigned to the outer surface of the workpiece. However, when producing large workpieces such as moving objects, a camera may be installed at a position distant from the workpiece to capture the entire outer shape of the workpiece. In this case, there is a risk that the identification information may not be recognized from the image data. [Means for solving the problem]

[0005] The present disclosure can be realized in the following forms.

[0006] (1) According to one aspect of the present disclosure, an information processing device is provided. The information processing device includes: an acquisition unit that acquires fixed-point data output from a fixed-point camera capturing an overhead image of a manufacturing site of a mobile object; and worker gaze data output from a wearable camera worn by a worker engaged in manufacturing the mobile object at the manufacturing site to capture an area corresponding to the worker's gaze. The acquisition unit recognizes, from the worker gaze data, mobile object identification information attached to the outer surface of the mobile object for identifying the mobile object. The recognition unit detects the mobile object from the fixed-point data. The identification unit identifies the mobile object by linking the mobile object identification information recognized from the worker gaze data to the mobile object detected by the detection unit. According to this aspect, the information processing device can identify the mobile object by linking the mobile object identification information acquired by capturing images using the wearable camera to the mobile object in the fixed-point data. This allows the information processing device to centrally manage mobile objects across multiple manufacturing processes in the production process of the mobile object. In this case, because the wearable camera is attached to the worker, it can capture the moving object identification information attached to the outer surface of the moving object from a closer position than a fixed camera. Also, by the worker moving around while wearing the wearable camera or changing the orientation of the wearable camera, the wearable camera can more reliably capture the moving object identification information attached to the outer surface of the moving object. This makes it easier to recognize the moving object identification information from the worker's line of sight data output from the wearable camera. (2) In the above aspect, the fixed-point data and the worker gaze data each include data captured at a plurality of different times, and the detection unit may perform time synchronization to associate the capture times of the fixed-point data with the capture times of the worker gaze data, thereby extracting the fixed-point data corresponding to the worker gaze data for which the moving object identification information has been recognized, and detecting the moving object from the extracted fixed-point data. According to this aspect, the information processing device can perform time synchronization and extract fixed-point data corresponding to the worker gaze data for which the moving object identification information has been recognized. Then, the information processing device can detect the moving object from the extracted fixed-point data. (3) In the above aspect, the acquisition unit acquires the fixed point data output from the fixed point cameras installed at different locations, and when the moving object associated with the moving object identification information moves from the shooting range of one of the fixed cameras into the shooting range of another fixed camera different from the one fixed camera, the detection unit detects the moving object from the fixed point data output from the other fixed camera, and the identification unit associates the moving object identification information with the moving object in the other fixed camera detected by the detection unit, thereby identifying and tracking the moving object between the shooting ranges of the multiple fixed cameras. According to this aspect, when the moving object associated with the moving object identification information moves from the shooting range of one fixed camera into the shooting range of another fixed camera, the information processing device can identify and track the moving object between the shooting ranges of the multiple fixed cameras. (4) The above aspect may further include a creation unit that creates a database linking the moving object identification information, the fixed-point data obtained by photographing the moving object identified by the moving object identification information, and information correlated with a defect that occurred when manufacturing the moving object identified by the moving object identification information, a learning unit that uses the database to train a machine learning model, and an output unit that uses the trained machine learning model to estimate factors related to the occurrence of the defect and output the estimation result. According to this aspect, the information processing device can create a database and train a machine learning model using the database, thereby estimating and outputting factors related to the occurrence of the defect. (5) Another aspect of the present disclosure provides an information processing method. The information processing method includes an acquisition step of acquiring fixed-point data output from a fixed-point camera capturing an overhead view of a manufacturing site of a mobile object and worker gaze data output from a wearable camera worn by a worker engaged in manufacturing the mobile object at the manufacturing site to capture an area corresponding to the worker's gaze; a recognition step of recognizing, from the worker gaze data, mobile object identification information attached to the outer surface of the mobile object for identifying the mobile object; a detection step of detecting the mobile object from the fixed-point data; and an identification step of identifying the mobile object by linking the mobile object identification information recognized from the worker gaze data to the mobile object detected in the detection step. According to this aspect, the mobile object can be identified by linking the mobile object identification information acquired by capturing images using the wearable camera to the mobile object in the fixed-point data. This allows for centralized management of mobile objects across multiple manufacturing processes in the production process of the mobile object. In this case, because the wearable camera is attached to the worker, it can capture the moving object identification information attached to the outer surface of the moving object from a closer position than a fixed camera. Also, by the worker moving around while wearing the wearable camera or changing the orientation of the wearable camera, the wearable camera can more reliably capture the moving object identification information attached to the outer surface of the moving object. This makes it easier to recognize the moving object identification information from the worker's line of sight data output from the wearable camera. The present disclosure can be realized in various forms other than the above-described information processing device and information processing method, such as an information processing system including an information processing device, a fixed camera, and a wearable camera, a method for manufacturing the information processing device and the information processing system, a method for controlling the information processing device and the information processing system, a computer program for realizing the control method, a non-transitory recording medium on which the computer program is recorded, etc. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing the configuration of an information processing system. [Figure 2]FIG. 1 is a block diagram showing the configuration of an information processing device. [Figure 3] 1 is a flowchart illustrating an example of a vehicle identification method. [Figure 4] 1 is a flowchart illustrating an example of a vehicle tracking method. DETAILED DESCRIPTION OF THE INVENTION

[0008] A. First embodiment: 1 is a diagram showing the configuration of information processing system 1. Information processing system 1 is used in a factory FC that manufactures mobile objects. Information processing system 1 includes one or more mobile objects, information processing device 20, one or more fixed cameras 31-35, and wearable camera 40.

[0009] In the present disclosure, the term "mobile body" refers to an object that can move, such as vehicles 11 to 15 and electric vertical take-off and landing aircraft (flying cars). In this embodiment, the mobile bodies are vehicles 11 to 15. Vehicles 11 to 15 may be vehicles that run on wheels or vehicles that run on caterpillars, such as passenger cars, trucks, buses, motorcycles, four-wheeled vehicles, tanks, and construction vehicles. Vehicles 11 to 15 include electric vehicles, gasoline-powered vehicles, hybrid vehicles, and fuel cell vehicles. When the mobile body is other than vehicles 11 to 15, the terms "vehicle" and "car" in the present disclosure can be appropriately replaced with "mobile body," and the term "traveling" can be appropriately replaced with "moving."

[0010] The fixed cameras 31-35 capture images of the manufacturing locations L1-L5 of the vehicles 11-15 from above. The fixed cameras 31-35 output fixed point data as a detection result. In this embodiment, the fixed point data is video data. That is, the fixed point data includes data captured at multiple different times. The fixed cameras 31-35 are equipped with a communication device (not shown) and can communicate with other devices such as the information processing device 20 via wired or wireless communication. The fixed cameras 31-35 are fixed to support members such as ceilings, walls, and pillars at the manufacturing locations L1-L5 of the vehicles 11-15 in the factory FC, for example.

[0011] In this embodiment, multiple fixed cameras 31-35 are installed in the factory FC. The first fixed camera 31 includes, in its imaging range G1, a first manufacturing location L1 where a first manufacturing process P1 is performed. The first manufacturing process P1 is, for example, a press process in which vehicle bodies and parts are manufactured by press working. The second fixed camera 32 includes, in its imaging range G2, a second manufacturing location L2 where a second manufacturing process P2 is performed. The second manufacturing process P2 is, for example, a welding process in which vehicle bodies and parts are welded together. The third fixed camera 33 includes, in its imaging range G3, a third manufacturing location L3 where a third manufacturing process P3 is performed. The third manufacturing process P3 is, for example, a painting process in which vehicle bodies and the like are painted. The fourth fixed camera 34 includes, in its imaging range G4, a fourth manufacturing location L4 where a fourth manufacturing process P4 is performed. The fourth manufacturing process P4 is, for example, an assembly process in which vehicles 11-15 are assembled by attaching parts to vehicle bodies. The fifth fixed camera 35 has a photographing range G5 that includes a fifth manufacturing location L5 where a fifth manufacturing process P5 is performed. The fifth manufacturing process P5 is, for example, an inspection process in which the functions of the vehicles 11-15 are inspected.

[0012] Wearable camera 40 is worn by worker E engaged in the manufacture of vehicles 11-15 at manufacturing locations L1-L5 of vehicles 11-15, and captures an area corresponding to worker E's line of sight. Wearable camera 40 outputs worker line of sight data as a detection result. Worker E moves within manufacturing locations L1-L5 or changes the orientation of wearable camera 40 so that vehicle identification information I1-I5 attached to the exterior surfaces of vehicles 11-15 is included within the imaging range G9 of wearable camera 40. In this way, wearable camera 40 captures an area of ​​the exterior surfaces of vehicles 11-15 that includes at least vehicle identification information I1-I5 from a position closer than fixed cameras 31-35. Vehicle identification information I1-I5 is unique information assigned to each vehicle 11-15 so that it does not overlap among multiple vehicles 11-15. Vehicle identification information I1-I5 may be, for example, text information or graphic information such as a code. Vehicle identification information I1-I5 may be printed on the exterior of vehicles 11-15. Alternatively, media bearing vehicle identification information I1-I5 may be attached to the exterior of vehicles 11-15. The worker gaze data is associated with worker position information indicating whether the data was output from wearable camera 40 worn by worker E located within any one of fixed cameras 31-35's shooting ranges G1-G5. In this embodiment, the worker gaze data is video data. In other words, the worker gaze data includes data captured at multiple different times. Wearable camera 40 is equipped with a communication device (not shown) and can communicate with other devices, such as information processing device 20, via wired or wireless communication.

[0013] FIG. 2 is a block diagram showing the configuration of information processing device 20. Information processing device 20 is configured as a computer including processor 201, memory 202, input / output interface 203, and bus 204. Processor 201, memory 202, and input / output interface 203 are connected via bus 204 to enable bidirectional communication. Input / output interface 203 is connected to communication device 205 for communicating with various devices external to information processing device 20. Communication device 205 can communicate with fixed cameras 31-35 and wearable camera 40 via wired or wireless communication. Processor 201 executes program PG stored in memory 202 to function as acquisition unit 211, recognition unit 212, detection unit 213, identification unit 214, determination unit 215, creation unit 216, learning unit 217, and output unit 218.

[0014] The acquisition unit 211 acquires fixed-point data and worker gaze data. In this embodiment, the acquisition unit 211 acquires a plurality of fixed-point data output from a plurality of fixed-point cameras 31 to 35 installed at different locations as shown in FIG.

[0015] The recognition unit 212 recognizes the vehicle identification information I1 to I5 from the worker gaze data. The recognition unit 212 recognizes the vehicle identification information I1 to I5 from the worker gaze data by extracting the vehicle identification information I1 to I5 from the worker gaze data using, for example, pattern recognition technology.

[0016] The detection unit 213 detects the vehicles 11 to 15 from the fixed point data. The detection unit 213 detects the outer shapes of the vehicles 11 to 15 from the fixed point data using, for example, segmentation technology. In this embodiment, the fixed point data and the worker gaze data each include data captured at multiple different times. Therefore, the detection unit 213 performs time synchronization. Time synchronization is a process of associating the capture time of the fixed point data output from the fixed point cameras 31 to 35 corresponding to the position of the worker E identified by the worker position information associated with the worker gaze data with the capture time of the worker gaze data corresponding to the time when the vehicle identification information I1 to I5 was captured. As a result, the detection unit 213 extracts fixed point data corresponding to the worker gaze data at the time when the vehicle identification information I1 to I5 was captured, and detects the vehicles 11 to 15 from the extracted fixed point data. Furthermore, when a vehicle 11 to 15 linked to vehicle identification information I1 to I5 moves from within the shooting range G1 to G5 of one fixed camera 31 to 35 to within the shooting range G1 to G5 of another fixed camera 31 to 35, the detection unit 213 detects the vehicle 11 to 15 from the fixed data output from the other fixed camera 31 to 35 whose shooting range G1 to G5 includes the destination of the vehicle 11 to 15.

[0017] The identification unit 214 identifies the vehicles 11-15 by linking the vehicle identification information I1-I5 recognized from the worker line of sight data to the vehicles 11-15 in the fixed point data detected by the detection unit 213. Furthermore, when the vehicles 11-15 linked with the vehicle identification information I1-I5 move from within the shooting range G1-G5 of one fixed camera 31-35 into the shooting range G1-G5 of another fixed camera 31-35, the identification unit 214 links the same vehicle identification information I1-I5 as the vehicle identification information I1-I5 linked to the fixed point data output from the one fixed camera 31-35 to the vehicles 11-15 in the fixed point data output from the other fixed camera 31-35 detected by the detection unit 213, using the shooting time and management information of the fixed point data. The management information is, for example, information indicating the order in which the multiple vehicles 11-15 move within the shooting ranges G1-G5 of the fixed cameras 31-35, and the expected positions of the vehicles 11-15 at a predetermined time. This allows the identification unit 214 to identify and track the vehicles 11-15 within the shooting ranges G1-G5 of the multiple fixed cameras 31-35.

[0018] The determination unit 215 determines whether the vehicles 11-15 associated with the vehicle identification information I1-I5 have moved from within the image capture range G1-G5 of one fixed camera 31-35 into the image capture range G1-G5 of another fixed camera 31-35. The determination unit 215 determines whether the vehicles 11-15 have moved between the image capture ranges G1-G5 of two adjacent fixed cameras 31-35, for example, by comparing coordinates representing the positions of the vehicles 11-15 with a predetermined threshold representing the positions of the manufacturing locations L1-L5. The threshold is, for example, a coordinate value obtained by expressing the coordinate values ​​on the boundary lines Q1-Q6 between the two adjacent manufacturing locations L1-L5 in an arbitrary coordinate system such as an image coordinate system. In this embodiment, one fixed camera 31-35 is set for each manufacturing location L1-L5. 1, where one manufacturing process P1 to P4 is performed, to a next manufacturing process L2 to L5, where a next manufacturing process P2 to P5 is performed after the first manufacturing process P1 to P4. The determination unit 215 may determine whether the vehicles 11 to 15 have moved between the imaging ranges G1 to G5 of the two adjacent fixed cameras 31 to 35 by using a relational expression that represents the boundary lines Q1 to Q6 between the two adjacent manufacturing processes L1 to L5, instead of a threshold value.

[0019] The creation unit 216 shown in FIG. 2 creates a database D that links vehicle identification information I1 to I5, fixed-point data obtained by photographing the vehicles 11 to 15 identified by the vehicle identification information I1 to I5, and correlation information that correlates with defects that occurred when manufacturing the vehicles 11 to 15 identified by the vehicle identification information I1 to I5. The creation unit 216 stores the created database D in the memory 202. Every time new correlation information is acquired, the creation unit 216 links the newly acquired correlation information to the vehicle identification information I1 to I5. Every time a vehicle 11 to 15 linked with vehicle identification information I1 to I5 moves from within the shooting range G1 to G5 of one fixed camera 31 to 35 into the shooting range G1 to G5 of another fixed camera 31 to 35, the creation unit 216 links the fixed-point data output from the one camera 31 to 35 to the vehicle identification information I1 to I5. As a result, the creation unit 216 updates the database D in the memory 202.

[0020] The correlation information includes, for example, fixed point data including vehicles 11 to 15 identified by vehicle identification information I1 to I5, vehicle specification information regarding the specifications of vehicles 11 to 15, defect information regarding the occurrence status of the defect, and process information regarding manufacturing processes P1 to P5.

[0021] The vehicle specification information includes, for example, at least one of vehicle model information indicating the type of vehicle 11-15, drive information indicating the drive system of the vehicle 11-15, destination information indicating the destination of the vehicle 11-15, and installation information indicating the installation position of the steering wheel, which differs depending on the destination. The drive information is information indicating which drive system the vehicle 11-15 will use to run, such as an electric vehicle, a gasoline vehicle, a hybrid vehicle, or a fuel cell vehicle. The installation information is information indicating whether the vehicle 11-15 is a left-hand drive vehicle or a right-hand drive vehicle.

[0022] The defect information includes, for example, at least one of occurrence presence / absence information indicating whether a defect has occurred, defect content information indicating the content of the defect, occurrence number information indicating the number of occurrences of the defect, and cause information indicating the cause of the defect. The defect information is output to the information processing device 20 from, for example, inspection equipment used in the inspection process. The defect information may be input to the information processing device 20 by a user, such as worker E, via an input device (not shown).

[0023] The process information includes, for example, at least one of required time information indicating the required time for each manufacturing process P1 to P5, progress information indicating the progress status of each manufacturing process P1 to P5, work content information indicating the work content of each manufacturing process P1 to P5, and process identification information identifying each manufacturing process P1 to P5. The required time for each manufacturing process P1 to P5 is calculated, for example, according to the location of the vehicles 11 to 15. In this embodiment, one fixed camera 31 to 35 is set for each manufacturing location L1 to L5. Therefore, the required time for the second manufacturing process P2 is calculated, for example, by subtracting the first time from the second time. The first time is the time when the position of the vehicles 11 to 15 changes from the shooting range G1 of the first fixed camera 31 to within the shooting range G1 to G5 of the second fixed camera 32. The second time is the time when the positions of the vehicles 11-15 change from the shooting range G2 of the second fixed camera 32 to within the shooting range G3 of the third fixed camera 33. The progress information is information that indicates whether the progress of each of the manufacturing processes P1-P5 is ahead or behind schedule. The progress information may be information that indicates the difference between the actual required time of each of the manufacturing processes P1-P5 and a predetermined standard required time of each of the manufacturing processes P1-P5.

[0024] The learning unit 217 trains the machine learning model M using the database D. As a result, the learning unit 217 trains the machine learning model M to output a predetermined element related to a malfunction when at least one of the vehicle identification information I1 to I5, the fixed point data, and any one of the multiple types of information included in the correlation information is input.

[0025] The learning unit 217 trains the machine learning model M by supervised learning using, for example, one or more training datasets prepared for each of the vehicle identification information I1 to I5, each training dataset including at least a portion of the information in the database D. The training dataset includes, for example, information corresponding to information input when the machine learning model M is used, and a correct answer label indicating an element corresponding to the output result of the machine learning model M among elements related to the defect. The training dataset may further include additional information for obtaining the output result.

[0026] The learning unit 217 trains, for example, a first machine learning model. When fixed-point data and required time information for the manufacturing processes P1 to P5 performed at the manufacturing locations L1 to L5 represented in the fixed-point data are input, the first machine learning model outputs, as an estimated result, the probability of a defect occurring in the manufacturing processes P1 to P5 performed at the manufacturing locations L1 to L5 represented in the input fixed-point data. When training the first machine learning model, the learning unit 217 uses, for example, multiple first training datasets prepared for each of the vehicle identification information I1 to I5. The first training dataset includes, for example, fixed-point data as training images, required time information for the manufacturing processes P1 to P5 performed at the manufacturing locations L1 to L5 represented in the fixed-point data, occurrence presence / absence information as a correct label, and additional information. The additional information includes, for example, the image capture time of the fixed-point data included in the first training dataset and process identification information for the manufacturing processes P1 to P5 performed at the manufacturing locations L1 to L5 represented in the fixed-point data. This allows the learning unit 217 to make the first machine learning model learn, for example, whether or not a defect occurs in each of the manufacturing processes P1 to P5, and the association between the occurrence of a defect and the time required for each of the manufacturing processes P1 to P5.

[0027] The learning unit 217 may train a second machine learning model. When drive information is input, the second machine learning model outputs, as an estimation result, the number of occurrences of defects that are expected to occur when manufacturing vehicles 11-15 having the drive system identified by the input drive information. When training the second machine learning model, the learning unit 217 uses, for example, multiple second training datasets prepared for each of the vehicle identification information I1-I5. The second training dataset includes, for example, drive information and occurrence number information as a correct answer label. This allows the learning unit 217 to train the second machine learning model about the association between the drive system of the vehicles 11-15 and the number of occurrences of defects.

[0028] The learning unit 217 may train a third machine learning model. When process identification information is input, the third machine learning model outputs, as an estimation result, the time expected to be required for each of the manufacturing processes P1 to P5 when a defect occurs. When training the third machine learning model, the learning unit 217 uses, for example, a plurality of third training datasets prepared for each of the vehicle identification information I1 to I5. The third training dataset includes, for example, process identification information, and occurrence / non-occurrence information and required time information as correct labels. This allows the learning unit 217 to train the third machine learning model to learn the time required for each of the manufacturing processes P1 to P5 when a defect occurs.

[0029] The output unit 218 uses the trained machine learning model M to estimate factors related to the occurrence of a malfunction and outputs the estimation result. The output unit 218 visualizes the estimation result using the database D, for example, by displaying the estimation result on a display device (not shown). At this time, the output unit 218 may display the estimation result using a diagram, table, graph, or the like. The output unit 218 may output information included in the database D without using the machine learning model M.

[0030] Fig. 3 is a flowchart showing an example of a vehicle identification method. The flow in Fig. 3 is executed, for example, when production of target vehicles 11-15 begins. Fig. 3 illustrates a case in which first vehicle 11 in Fig. 1 is identified from other vehicles 12-14, and worker E wearing wearable camera 40 is engaged in the production of first vehicle 11 at first production location L1.

[0031] In step S11, acquisition unit 211 of information processing device 20 transmits a request signal requesting worker gaze data to wearable camera 40. In step S12, wearable camera 40, which has received the request signal, associates the worker gaze data with worker position information and transmits the data to information processing device 20. In step S13, recognition unit 212 of information processing device 20 recognizes vehicle identification information I1 of first vehicle 11 from the worker gaze data. In step S14, detection unit 213 of information processing device 20 transmits a request signal requesting fixed point data to first fixed point camera 31 corresponding to the position of worker E identified by the worker position information associated with the worker gaze data. In step S15, first fixed point camera 31, which has received the request signal, transmits the fixed point data to information processing device 20. In step S16, the detection unit 213 of the information processing device 20 performs time synchronization to associate the shooting time of the fixed point data output from the first fixed point camera 31 with the shooting time of the worker gaze data corresponding to the timing when the vehicle identification information I1 was captured. As a result, the detection unit 213 extracts fixed point data corresponding to the worker gaze data at the timing when the vehicle identification information I1 was captured. In step S17, the detection unit 213 detects the first vehicle 11 from the extracted fixed point data. In step S18, the identification unit 214 of the information processing device 20 associates the vehicle identification information I1 recognized from the worker gaze data with the first vehicle 11 in the fixed point data detected by the detection unit 213. As a result, the identification unit 214 distinguishes the first vehicle 11 from the other vehicles 12 to 15.

[0032] Fig. 4 is a flowchart showing an example of a vehicle tracking method. The flow in Fig. 4 is executed, for example, after the identification of vehicles 11 to 15 is completed. Fig. 4 illustrates an example of tracking the first vehicle 11 in the section from the shooting range G1 of the first fixed camera 31 to the shooting range G2 of the second fixed camera 32 shown in Fig. 1. Note that the information processing device 20 tracks the first vehicle 11 by repeatedly performing the same process for the shooting ranges G3 to G5 of the other fixed cameras 33 to 35 until it reaches a point where tracking ends.

[0033] In step S21, the determination unit 215 of the information processing device 20 calculates the position of the first vehicle 11. In step S22, the determination unit 215 determines whether the first vehicle 11 has moved from the shooting range G1 of the first fixed camera 31 into the shooting range G2 of the second fixed camera 32. If it is determined that the first vehicle 11 has moved from the shooting range G1 of the first fixed camera 31 into the shooting range G2 of the second fixed camera 32 (step S22: Yes), in step S23, the detection unit 213 of the information processing device 20 transmits a request signal requesting fixed point data to the second fixed camera 32. Having received the request signal, the second fixed camera 32 transmits the fixed point data to the information processing device 20 in step S24. In step S25, the detection unit 213 of the information processing device 20 detects the first vehicle 11 from the acquired fixed point data. In step S26, the identification unit 214 of the information processing device 20 links the vehicle identification information I1 of the first vehicle 11 that was linked to the fixed point data output from the first fixed point camera 31 to the first vehicle 11 in the fixed point data output from the second fixed point camera 32. As a result, the identification unit 214 identifies and tracks the first vehicle 11 between the shooting range G1 of the first fixed point camera 31 and the shooting range G2 of the second fixed point camera 32.

[0034] According to the above embodiment, information processing device 20 can identify vehicles 11-15 by linking vehicle identification information I1-I5 acquired by photographing vehicles 11-15 with vehicles 11-15 in the fixed point data. This allows information processing device 20 to centrally manage vehicles 11-15 across multiple manufacturing processes P1-P5 in the production process of vehicles 11-15. In this case, wearable camera 40 is attached to worker E, so it can photograph vehicle identification information I1-I5 attached to the exterior surfaces of vehicles 11-15 from a closer position than fixed cameras 31-35. Furthermore, by worker E moving around while wearing wearable camera 40 or changing the orientation of wearable camera 40, wearable camera 40 can more reliably photograph vehicle identification information I1-I5 attached to the exterior surfaces of vehicles 11-15. This makes it easier to recognize vehicle identification information I1-I5 from worker gaze data output from wearable camera 40.

[0035] Furthermore, according to the above embodiment, worker E can change the shooting range G9 of wearable camera 40 by moving around while wearing wearable camera 40 or by changing the orientation of wearable camera 40. Therefore, when the layout of production equipment or production lines is variable, it is possible to avoid a situation in which information processing device 20 is unable to recognize vehicle identification information I1 to I5.

[0036] Furthermore, according to the above embodiment, the information processing device 20 can extract fixed-point data corresponding to the worker gaze data from which the vehicle identification information I1 to I5 has been recognized by performing time synchronization, and can then detect the vehicles 11 to 15 from the extracted fixed-point data.

[0037] Furthermore, according to the above embodiment, when a vehicle 11 to 15 linked to vehicle identification information I1 to I5 moves from within the shooting range G1 to G5 of one fixed camera 31 to 35 into the shooting range G1 to G5 of another fixed camera 31 to 35, the information processing device 20 can identify and track the vehicle 11 to 15 between the shooting ranges G1 to G5 of the multiple fixed cameras 31 to 35.

[0038] Furthermore, according to the above embodiment, the information processing device 20 can create a database D that merges various pieces of correlation information using the vehicle identification information I1 to I5 as a key. The information processing device 20 can train a machine learning model M using the database D. The information processing device 20 can estimate factors related to the occurrence of a defect using the trained machine learning model M and output the estimation results. In this way, the information processing device 20 can acquire and analyze correlation information for each of the vehicles 11 to 15 across multiple manufacturing processes P1 to P5 in the production process of the vehicles 11 to 15. Therefore, by utilizing the database D, it is possible to estimate factors related to the occurrence of a defect, such as identifying the occurrence of a defect for each manufacturing process P1 to P5.

[0039] Furthermore, according to the above embodiment, the information processing device 20 can automatically calculate the required time for each of the manufacturing processes P1 to P5 according to the positions of the vehicles 11 to 15. This reduces the burden on the worker E of measuring the required time, etc.

[0040] B. Other Embodiments: (B1) When a plurality of vehicles 11-15 are present within the shooting ranges G1-G5 of the same fixed cameras 31-35 at the same time, the detection unit 213 may use, for example, management information to identify the target vehicles 11-15. In this manner, even when a plurality of vehicles 11-15 are present within the shooting ranges G1-G5 of the same fixed cameras 31-35 at the same time, the information processing device 20 can identify the vehicles 11-15.

[0041] (B2) When multiple workers E wearing wearable cameras 40 are present within the shooting ranges G1 to G5 of the same fixed cameras 31 to 35, the worker gaze data may be associated with worker identification information indicating which worker E is wearing the wearable camera 40 from which the worker gaze data is output. In this manner, information processing device 20 can identify vehicles 11 to 15 even when multiple workers E wearing wearable cameras 40 are present within the shooting ranges G1 to G5 of the same fixed cameras 31 to 35.

[0042] (B3) The fixed point data and the worker gaze data may each be still image data. In other words, the fixed point data and the worker gaze data may each be data captured at the same time. In this manner, the information processing device 20 can identify the vehicles 11 to 15 without time synchronization.

[0043] (B4) A plurality of fixed cameras 31-35 may be installed at one of the manufacturing locations L1-L5, or a single fixed camera 31-35 may be installed at a plurality of manufacturing locations L1-L5. Even in this configuration, the information processing device 20 can identify the vehicles 11-15.

[0044] (B5) The information processing device 20 does not need to include at least one of the determination unit 215, the creation unit 216, the learning unit 217, and the output unit 218, but only needs to include at least the acquisition unit 211, the recognition unit 212, the detection unit 213, and the identification unit 214. Furthermore, the identification unit 214 only needs to be able to identify the vehicles 11-15, and does not necessarily need to have the function of tracking the vehicles 11-15 within the shooting ranges G1-G5 of the multiple fixed cameras 31-35.

[0045] The present disclosure is not limited to the above-described embodiments and can be realized in various configurations without departing from the spirit thereof. For example, the technical features of the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention section can be appropriately replaced or combined to solve some or all of the above-described problems or achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted. [Explanation of symbols]

[0046] 1...information processing system, 11-15...vehicle, 20...information processing device, 31-35...fixed camera, 40...wearable camera, 201...processor, 202...memory, 203...input / output interface, 204...bus, 205...communication device, 211...acquisition unit, 212...recognition unit, 213...detection unit, 214...identification unit, 215...determination unit, 216...creation unit, 217...learning unit, 218...output unit, D...database, E...worker, FC...factory, G1-G5, G9...shooting range, I1-I5...vehicle identification information, L1-L5...manufacturing location, M...machine learning model, P1-P5...manufacturing process, PG...program, Q1-Q6...boundary

Claims

1. An information processing device, an acquisition unit that acquires fixed-point data output from a fixed-point camera that takes a bird's-eye view of a manufacturing site of a moving body, and worker line-of-sight data output from a wearable camera that is worn by a worker engaged in manufacturing the moving body at the manufacturing site and captures an area corresponding to the worker's line of sight; a recognition unit that recognizes moving body identification information attached to an outer surface of the moving body for identifying the moving body from the worker line of sight data; a detection unit that detects the moving object from the fixed point data; an identification unit that identifies the moving object by linking the moving object in the fixed point data detected by the detection unit with the moving object identification information recognized from the worker line of sight data.

2. 2. The information processing device according to claim 1, the fixed point data and the worker line of sight data each include data captured at a plurality of different times, The detection unit performs time synchronization to associate the shooting time of the fixed point data with the shooting time of the worker gaze data, thereby extracting the fixed point data corresponding to the worker gaze data for which the moving object identification information has been recognized, and detecting the moving object from the extracted fixed point data.

3. 2. The information processing device according to claim 1, the acquisition unit acquires a plurality of fixed point data output from a plurality of fixed point cameras installed at different locations, When the moving object associated with the moving object identification information moves from within the shooting range of one of the fixed cameras into the shooting range of another fixed camera different from the one fixed camera, the detection unit detects the moving object from the fixed point data output from the other fixed camera, The identification unit associates the moving object identification information with the moving object in the other fixed camera detected by the detection unit, thereby identifying and tracking the moving object among the shooting ranges of the multiple fixed cameras.

4. The information processing device according to claim 1, further comprising: a creating unit that creates a database that links the moving body identification information, the fixed point data obtained by photographing the moving body identified by the moving body identification information, and information that is correlated with a defect that occurred when manufacturing the moving body identified by the moving body identification information; a learning unit that uses the database to learn a machine learning model; an output unit that uses the trained machine learning model to estimate factors related to the occurrence of the malfunction and outputs the estimation result.

5. An information processing method, comprising: an acquisition process for acquiring fixed-point data output from a fixed-point camera that takes a bird's-eye view of a manufacturing site of the mobile body, and worker gaze data output from a wearable camera that is worn by a worker engaged in manufacturing the mobile body at the manufacturing site and captures an area corresponding to the worker's gaze; a recognition step of recognizing, from the worker's line of sight data, moving body identification information attached to an outer surface of the moving body for identifying the moving body; a detection step of detecting the moving object from the fixed point data; an identification process for identifying the moving object by linking the moving object identification information recognized from the worker line of sight data to the moving object in the fixed point data detected in the detection process.

Images