Method, system and device for evaluating work task, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a method, system and device for evaluating a work task, electronic equipment and a storage medium. The method comprises the following steps: collecting point cloud data of a workpiece in a work task and a three-dimensional image frame of a worker in the work task; identifying attribute information of the workpiece in a machine vision mode based on the point cloud data; identifying action information of the worker in a deep learning mode based on the three-dimensional image frame; comparing the combined information of the attribute information and the action information with a standard operation program of the work task, wherein the standard operation program comprises the standard attribute of the workpiece and the standard action of the worker in the work task; and evaluating the work task based on the comparison result. And the work task is subjected to joint evaluation in two dimensions including the action of the worker and the attribute of the workpiece, so that the evaluation accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of manufacturing task processing, and in particular to a method, system, device, electronic device and storage medium for evaluating a work task. Background Art

[0002] At present, production workers basically operate according to paper specifications, but paper specifications cannot intuitively guide workers on how to work. Moreover, it is difficult to detect whether each work step of the workers is carried out according to the specifications.

[0003] In addition, due to the complexity of the operation, higher requirements are placed on the operator's operating skills and attention, and workers may encounter problems such as unskilled operation during the operation process, which will affect work efficiency and reduce product quality. Therefore, it is necessary to evaluate the worker's work tasks. Summary of the invention

[0004] The embodiments of the present invention provide a method, system, device, electronic device and storage medium for evaluating a work task.

[0005] In a first aspect, a method for evaluating a work task is provided. The method comprises:

[0006] Collecting point cloud data of a workpiece in a work task and a three-dimensional image frame of a worker in the work task;

[0007] Identifying attribute information of the workpiece in a machine vision manner based on the point cloud data;

[0008] recognizing the action information of the worker in a deep learning manner based on the three-dimensional image frame;

[0009] comparing the combination of the attribute information and the action information with a standard operating procedure for the work task, wherein the standard operating procedure includes standard attributes of the workpiece in the work task and standard actions of the worker; and

[0010] The work task is evaluated based on the comparison result.

[0011] Therefore, the work task is jointly evaluated on two dimensions including the worker's action information and the workpiece's attribute information, thereby improving the evaluation accuracy.

[0012] Preferably, the attribute information includes the type of the workpiece; the method further comprises:

[0013] When the type of the workpiece does not conform to the standard type of the workpiece in the work task, an alarm message is issued.

[0014] Therefore, the safety of work tasks can be improved.

[0015] Preferably, it further comprises: determining the real-time point of the action information; determining the standard action at the next time point of the real-time point based on the standard operating procedure; and issuing a reminder message to remind the worker to perform the standard action at the next time point.

[0016] Therefore, the efficiency of work tasks can be improved.

[0017] Preferably, the work task includes multiple work stages, and the standard operating procedure includes multiple sub-standard operating procedures corresponding to the multiple work stages respectively; wherein the comparison of the combined information of the attribute information and the action information with the standard operating procedure of the work task includes: based on the corresponding comparison with the corresponding start actions of the multiple work stages, splitting the combined information into multiple sub-combination information, wherein the multiple sub-combination information corresponds to the multiple work stages respectively; comparing each sub-combination information with the corresponding sub-standard operating procedure.

[0018] Therefore, splitting complex work tasks into multiple subtasks reduces the difficulty of comparison.

[0019] Preferably, the multiple sub-standard operating procedures have corresponding standard working hours; wherein the comparing each sub-combination information with the corresponding sub-standard operating procedure includes: comparing the actual working hours corresponding to each sub-combination information with the standard working hours of the corresponding sub-standard operating procedure; comparing the attribute information in each sub-combination information with the standard attributes of the corresponding sub-standard operating procedure; and comparing the action information in each sub-combination information in the sub-combination information with the standard action of the corresponding sub-standard operating procedure.

[0020] Therefore, a multi-dimensional comprehensive evaluation is achieved and the comprehensiveness of the evaluation is improved.

[0021] Preferably, it further comprises:

[0022] Determining an overtime work stage, wherein the actual working hours in the overtime work stage are greater than the corresponding standard working hours;

[0023] Determining the ratio of the number of overtime work stages to the number of all work stages;

[0024] When the ratio is less than a predetermined threshold, determining a working tool used by the worker in the overtime working stage;

[0025] A reminder message is issued to check the working tool.

[0026] Therefore, a timeout due to the working tool can be detected in time.

[0027] Preferably, the evaluating the work task based on the comparison result comprises:

[0028] When the action information meets the standard action and the attribute information meets the standard attribute, generating a first evaluation value;

[0029] When the action information conforms to the standard action, the attribute information does not conform to the standard attribute, and the attribute information is within a predetermined attribute interval, generating a second evaluation value;

[0030] When the action information conforms to the standard action, the attribute information does not conform to the standard attribute, and the attribute information is not within the predetermined attribute interval, generating a third evaluation value;

[0031] When the action information does not meet the standard action, generating a fourth evaluation value;

[0032] The first evaluation value is greater than the second evaluation value, the second evaluation value is greater than the third evaluation value, and the third evaluation value is greater than the fourth evaluation value.

[0033] Therefore, compared with the influence of the comparison between the attribute information and the standard attribute on the evaluation value, the action comparison between the worker's action and the standard action has a greater influence factor, which is beneficial for the worker to improve the accuracy of the action.

[0034] In a second aspect, a system for evaluating a work task is provided, the system comprising:

[0035] A collection module, the collection module is configured to collect point cloud data of a workpiece in a work task and a three-dimensional image frame of a worker in the work task;

[0036] A server, wherein the server is configured to: identify attribute information of the workpiece in a machine vision manner based on the point cloud data; identify action information of the worker in a deep learning manner based on the three-dimensional image frame; compare the combined information of the attribute information and the action information with a standard operating procedure of the work task, wherein the standard operating procedure contains standard attributes of the workpiece in the work task and standard actions of the worker; and evaluate the work task based on the comparison result.

[0037] Preferably, the attribute information includes the type of the artifact; the server is configured to issue an alarm message when the type of the artifact does not conform to the standard type of the artifact in the work task; the system further includes: a workstation, which is configured to display the alarm message.

[0038] Preferably, the server is configured to: determine the real-time point of the action information; determine the standard action at the next time point after the real-time point based on the standard operating procedure; issue a reminder message to remind the worker to perform the standard action at the next time point; the system further includes: a workstation configured to display the reminder message.

[0039] Preferably, the work task includes multiple work stages, and the standard operating procedure includes multiple sub-standard operating procedures corresponding to the multiple work stages respectively; the server is configured to split the combination information into multiple sub-combination information based on corresponding comparisons with corresponding start actions of the multiple work stages, wherein the multiple sub-combination information respectively corresponds to the multiple work stages; and compare each sub-combination information with the corresponding sub-standard operating procedure.

[0040] Preferably, the multiple sub-standard operating procedures have corresponding standard working hours; the server is configured to compare the actual working hours corresponding to each sub-combination information with the standard working hours of the corresponding sub-standard operating procedure; compare the attribute information in each sub-combination information with the standard attributes of the corresponding sub-standard operating procedure; and compare the action information in each sub-combination information in the sub-combination information with the standard action of the corresponding sub-standard operating procedure.

[0041] Preferably, the server is configured to: determine an overtime work stage, wherein the actual working hours in the overtime work stage are greater than the corresponding standard working hours; determine the ratio of the number of overtime work stages to the number of all work stages; when the ratio is less than a predetermined threshold, determine the work tools used by the worker in the overtime work stage; and issue a prompt message to check the work tools; the system further includes: a workstation, which is configured to display the prompt message.

[0042] In a third aspect, a device for evaluating a work task is provided. The device comprises: an acquisition module, the acquisition module being configured to acquire point cloud data of a workpiece in the work task and a three-dimensional image frame of a worker in the work task; a first recognition module, the first recognition module being configured to recognize attribute information of the workpiece in a machine vision manner based on the point cloud data; a second recognition module, the second recognition module being configured to recognize action information of the worker in a deep learning manner based on the three-dimensional image frame; a comparison module, the comparison module being configured to compare the combination of the attribute information and the action information with a standard operating procedure of the work task, wherein the standard operating procedure comprises standard attributes of the workpiece in the work task and standard actions of the worker; and an evaluation module, the evaluation module being configured to evaluate the work task based on the comparison result.

[0043] In a fourth aspect, an electronic device is provided, wherein the device comprises a processor and a memory, wherein an application program executable by the processor is stored in the memory, so that the processor executes the method for evaluating a work task according to any one of the above items.

[0044] In a fifth aspect, a computer-readable medium is provided, comprising computer-readable instructions stored thereon, wherein the computer-readable instructions are used to execute the method for evaluating a work task according to any one of the above items.

[0045] In a sixth aspect, a computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program is used to perform the method for evaluating a work task according to any one of the above items. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to make the technical solutions of the examples of the present disclosure clearer, only the drawings used to describe the examples will be introduced below. Obviously, the drawings to be described below are only some examples of the present disclosure. Those skilled in the art can obtain other drawings based on these drawings without creative work.

[0047] Figure 1 is a flowchart of a method for evaluating a work task according to an embodiment of the present invention.

[0048] Figure 2 is a block diagram of a system for evaluating a work task according to an embodiment of the present invention.

[0049] Figure 3 is an exemplary schematic diagram of a process for evaluating a work task according to an embodiment of the present invention.

[0050] Figure 4 is an exemplary structural diagram of an apparatus for evaluating a work task according to an embodiment of the present invention.

[0051] Figure 5 is an exemplary structural diagram of an electronic device according to an embodiment of the present invention.

[0052] List of reference numerals:

[0053]

[0054] DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail with reference to the following embodiments.

[0056] For the sake of brevity and intuitiveness in description, the scheme of the present invention is explained below by describing several representative implementations. A large number of details in the implementations are only used to help understand the scheme of the present invention. However, it is obvious that the technical scheme of the present invention may not be limited to these details when implemented. In order to avoid unnecessarily obscuring the scheme of the present invention, some implementations are not described in detail, but only a framework is given. Hereinafter, "including" means "including but not limited to", and "according to..." means "at least according to..., but not limited to only according to...". Due to the language habits of Chinese, when the number of a component is not specifically specified below, it means that the component can be one or more, or can be understood as at least one.

[0057] Figure 1 is a flow chart of a method for evaluating a work task according to an embodiment of the present invention. Figure 1 As shown, the method comprises:

[0058] Step 101: Collect point cloud data of a workpiece in a work task and a three-dimensional (3D) image frame of a worker in the work task.

[0059] According to the production volume and terminology (number of different types of products), production regularity (frequency of reproduction of the same type of products) and stability (time of production of the same type of products), work tasks usually consist of the following three types of production: single production (single piece), series production and mass production. Work tasks can be workers manually operating mechanical equipment such as milling machines, lathes, drilling machines, sawing machines, etc. to process various tasks. The mechanical equipment is suitable for manual processing of small batches and simple parts. Work tasks can also be workers using CNC equipment to process workpieces, where CNC equipment can include machining centers, turning and milling centers, EDM equipment, wire cutting machine tools, etc.

[0060] Point cloud scanners can be used to collect point cloud data of workpieces in work tasks. The workpiece is scanned using a point cloud scanner, and the scanned data is recorded in the form of points. Each point contains three-dimensional coordinates and may also contain color information (RGB) or reflection intensity information, etc.

[0061] A 3D camera device may be used to capture 3D image frames of a worker in a work task, such as a real-time 3D image sequence. For example, the 3D imaging device is a fisheye camera or a camera with depth information.

[0062] Step 102: Based on the point cloud data, identify the workpiece and recognize the attribute information of the workpiece through machine vision.

[0063] Specifically: First, perform denoising and filtering of the point cloud data, calculate the normal vector for the denoised point cloud data, and then perform voxel grid downsampling to obtain key points, thereby reducing the amount of calculation. Moreover, for the key points, calculate the histogram feature descriptor of the normal direction to obtain the SHOT descriptor associated with the target point cloud and the scene point cloud, and use the K tree structure search algorithm to find similarities based on the Euclidean distance. In addition, call the clustering algorithm to obtain a local point cloud set in the scene that is compared with the target point cloud based on the Hough voting algorithm, and then use the point cloud comparison algorithm to compare the target point cloud with the locked local point cloud set. Finally, in order to reduce the error rate, a global hypothesis verification algorithm can be used to filter out erroneous points, and the transformation matrix, rotation matrix and translation vector between the compared local point cloud set and the model point cloud can be printed, and the identified target can be calibrated for visual output. Optionally, the workpiece CAD model can be used as the target for point cloud comparison.

[0064] For example, the attribute information of an artifact identified based on point cloud data may include:

[0065] (1) Physical properties of the workpiece, including surface features such as shape, color, characters, barcodes, etc.

[0066] (2) Physical dimension measurement parameters of the workpiece.

[0067] (3) Spatial position information of the workpiece.

[0068] (4) Surface condition information of the workpiece, which is used to determine whether the workpiece has quality defects, such as appearance defects of parts, adhesion of pollutants, and functional defects.

[0069] Step 103: Based on the 3D image frame, identify the worker's motion information through deep learning.

[0070] Here, the deep learning algorithm used may include: behavior recognition algorithm based on unsupervised learning, behavior recognition algorithm based on convolutional neural network, behavior recognition algorithm based on recurrent neural network, etc. The worker's action may be a control action issued by physical contact with the workpiece (for example, moving a box), or may be a remote control action issued without physical contact with the workpiece (for example, control operation on a touch screen).

[0071] In one embodiment, the identified action information may include: (1) motion classification, such as twisting, moving, or hammering; (2) motion features, such as optical flow, speed, direction, trajectory, spatiotemporal context, or spatiotemporal context interest points.

[0072] Step 104: Compare the combination of the attribute information and the action information with the standard operating procedure (SOP) of the work task, wherein the SOP contains standard attributes of the workpiece in the work task and standard actions of the worker.

[0073] Here, the attribute information and the action information can be combined in the time dimension to form combined information. For example, the attribute information, the action information, the first collection time of the attribute information and the second collection time of the action information are combined to obtain combined information with a time dimension. Preferably, the first collection time and the second collection time are preferably the same time, so as to achieve synchronous combination of the attribute information and the action information. The first collection time and the second collection time can also be different, and the action information and the attribute information have a causal relationship. For example, the action is to lift a box, the attribute is the height of the box after it is lifted, and so on.

[0074] Here, the standard operating procedures for work tasks are established in advance. Standard operating procedures refer to the description of the standard operating procedures and requirements of work tasks in a unified format to guide and standardize the specific process of performing work tasks.

[0075] In one embodiment, the SOP may be a description file in a text format, which describes multiple standard attributes of a workpiece and multiple standard actions of a worker in a text format. Preferably, the description file further describes the temporal association between the standard actions and the standard attributes. By natural language processing, the standard attributes (such as physical dimensions and spatial positions) described in text and the standard actions described in text, as well as the temporal association between the two, are extracted from the standard operating procedure in a text format.

[0076] For example, the text description file includes a description sentence: "After the workpiece cleaning action is performed, the workpiece has no contaminants." Based on the description sentence, the standard attribute includes: no contaminants are attached to the workpiece; the standard action includes: cleaning, and the relationship between the standard attribute and the standard action is: the cleaning action occurs before no contaminants are attached, that is, no contaminants are attached is the result of the cleaning action.

[0077] In one embodiment, the standard operating procedure can be a video file associated with a subtitle file, which describes the standard actions of the worker in a video format, and describes the standard properties of the workpiece in a text format in the subtitle file. The SOP can be presented to the worker in a video format to provide SOP training. In addition, the standard operating procedure can be reused to achieve the evaluation of the worker's work tasks. The features of the standard actions are extracted from the video file through video extraction processing based on deep learning. The text description of the standard properties of the workpiece is extracted from the subtitle file through natural language processing. Preferably, a temporal correlation relationship between the features of the standard actions and the text description of the standard attributes is established.

[0078] Step 105: Evaluate the work task based on the comparison result.

[0079] Here, the extracted action information can be compared with the corresponding standard action, and the attribute information associated with the action information can be compared with the standard attribute associated with the standard action. In one embodiment, the evaluation task based on the comparison result includes: when the action information meets the standard action and the attribute information meets the standard attribute, a first evaluation value is generated; when the action information meets the standard action, the attribute information does not meet the standard attribute, and the attribute information is located in a predetermined attribute interval, a second evaluation value is generated; when the action information meets the standard action, the attribute information does not meet the standard attribute, and the attribute information is not located in a predetermined attribute interval, a third evaluation value is generated; when the action information does not meet the standard action, a fourth evaluation value is generated; wherein the first evaluation value is greater than the second evaluation value, the second evaluation value is greater than the third evaluation value, and the third evaluation value is greater than the fourth evaluation value.

[0080] Compared with the influence of the comparison between the attribute information and the standard attributes on the evaluation value, the motion comparison between the worker's motion and the standard motion has a greater influence factor.

[0081] In one embodiment, the attribute information includes the type of the workpiece; the method further includes: when the type of the workpiece does not conform to the standard workpiece type of the work task, sending an alarm message. Therefore, the safety of the work task can be improved.

[0082] In one embodiment, the method further comprises: determining a time point at which the action information is detected (i.e., the time at which the action is performed); determining a standard action at the next time point of the time point based on the SOP; and sending a reminder message to remind the worker to perform the standard action at the next time point. Therefore, work efficiency can be improved by reminding the worker of the next standard action.

[0083] In one embodiment, the work task includes multiple work stages, and the SOP includes multiple sub-standard operating procedures corresponding to the multiple work stages respectively; wherein comparing the combination information of the attribute information and the action information with the standard operating procedure of the work task includes: splitting the combination information into multiple sub-combination information based on the corresponding comparison with the corresponding start actions of the multiple work stages, wherein the multiple sub-combination information corresponds to the multiple work stages respectively; and comparing each sub-combination information with the corresponding sub-standard operating procedure. Therefore, splitting a complex work task into multiple sub-tasks reduces the difficulty of comparison.

[0084] In one embodiment, the plurality of sub-standard operating procedures have corresponding standard working hours; wherein comparing each sub-combination information with the corresponding sub-standard operating procedure comprises: comparing the actual working hours corresponding to each sub-combination information with the standard working hours of the corresponding sub-standard operating procedure; comparing the attribute information in each sub-combination information with the standard attribute of the corresponding sub-standard operating procedure; and comparing the action information in each sub-combination information in the sub-combination information with the standard action of the corresponding sub-standard operating procedure. Therefore, a multi-dimensional comprehensive evaluation is achieved, and the comprehensiveness of the evaluation is improved.

[0085] In one embodiment, it further comprises: determining an overtime work stage, wherein the actual working hours of the overtime work stage are greater than the corresponding standard working hours; determining the ratio of the number of overtime work stages to the number of all work stages; when the ratio is less than a predetermined threshold, determining the work tools used by the worker in the overtime work stage; and issuing a prompt message to check the work tools. Therefore, the overtime caused by the work tools can be detected in time.

[0086] Considering that skilled workers are usually proficient in various work stages, if a skilled worker takes too long to work in a specific work stage, it may be that the work tool used in the work stage is faulty or the work stage related to the work tool is defective, so the problem is raised. Check the prompt information of the work tool, thereby improving work efficiency.

[0087] Figure 2 is a block diagram of a system for evaluating work tasks according to an embodiment of the present invention. Figure 2 In the system used to evaluate work tasks, there are:

[0088] A collection module 10, which is configured to collect point cloud data of a workpiece in a work task and a three-dimensional image frame of a worker in the work task;

[0089] Server 20, the server is configured to: identify attribute information of the workpiece in a machine vision manner based on point cloud data; identify action information of the worker in a deep learning manner based on the three-dimensional image frame; compare the combined information of the attribute information and the action information with the standard operating procedure of the work task, wherein the standard operating procedure contains standard attributes of the workpiece in the work task and standard actions of the worker; and evaluate the work task based on the comparison result.

[0090] The acquisition module 10 may include a point cloud scanner 11 and a 3D camera device 12 arranged in a work site environment. The point cloud scanner 11 scans a workpiece to obtain point cloud data. The 3D camera device 12 captures 3D image frames of a worker working on the workpiece.

[0091] The acquisition module 10 sends the point cloud data and the 3D image frame to the server 20. The server 20 stores an algorithm library 23. The algorithm library 23 includes a point cloud comparison algorithm 21 (eg, an ICP algorithm) and a neural network-based target detection algorithm 22 (eg, CNN, RCNN, Fast RCNN, or Faster RCNN).

[0092] The server 20 performs real-time processing 24 using the point cloud data and the 3D image frame. In the real-time processing 24, the point cloud comparison algorithm 21 is called to perform machine vision recognition to identify the attribute information of the workpiece, which specifically includes physical features, measurement parameters, spatial position and surface state. In the real-time processing 24, the target detection algorithm 22 is also called to identify the worker's action from the 3D image frame. The server 20 also stores an SOP 28 containing standard attributes of the workpiece and standard actions of the worker. The combined information of the attribute information of the workpiece and the action information of the worker is compared with the SOP of the work task. In the real-time processing 24, the work task is evaluated based on the comparison result.

[0093] In addition, the server 20 transmits the evaluation results to a workstation 30, which is usually located at the work site. The evaluation results are displayed on a display interface 31 of the workstation 30. Based on the comparison results, the server 20 can also generate prompt messages and alarm messages that can timely remind the workers. Correspondingly, a prompt icon 32 and an alarm icon 33 can be displayed in the display interface 31 of the workstation 30.

[0094] In one embodiment, the attribute information includes the type of the workpiece; the server 20 is configured to issue an alarm message when the type of the workpiece does not conform to the standard workpiece type of the work task; the system further includes: a workstation 30 for displaying the alarm message.

[0095] In one embodiment, the server 20 is configured to determine a time point of action information; determine a standard action for the next time point of the time point based on the SOP; and send a reminder message; the system further includes a workstation 30 configured to display the reminder message.

[0096] In one embodiment, the work task includes multiple work stages, and the SOP includes multiple sub-standard operating procedures corresponding to the multiple work stages respectively; the server 20 is configured to split the combination information into multiple sub-combination information based on corresponding comparisons with corresponding start actions of the multiple work stages, wherein the multiple sub-combination information respectively corresponds to the multiple work stages; and compare each sub-combination information with the corresponding sub-standard operating procedure.

[0097] In one embodiment, the multiple sub-standard operating procedures have corresponding standard working hours; the server 20 is configured to compare the actual working hours corresponding to each sub-combination information with the standard working hours of the corresponding sub-standard operating procedure; compare the attribute information in each sub-combination information with the standard attributes of the corresponding sub-standard operating procedure; and compare the action information in each sub-combination information in the sub-combination information with the standard action of the corresponding sub-standard operating procedure.

[0098] In one embodiment, the server 20 is configured to: determine an overtime work stage, wherein the actual working hours in the overtime work stage are greater than the corresponding standard working hours; determine the ratio of the number of overtime work stages to the number of all work stages; when the ratio is less than a predetermined threshold, determine the work tools used by the worker in the overtime work stage; and issue a prompt message to check the work tools; the system further includes: a workstation configured to display the prompt message.

[0099] Figure 3 4 is an exemplary schematic diagram of a process for evaluating a work task according to an embodiment of the present invention. In the acquisition process 40, a point cloud acquisition process 41 is performed on a workpiece to acquire point cloud data; and a 3D imaging process 42 is performed on a worker who uses a tool to operate the workpiece to acquire a 3D image frame. In the recognition process 50, data preprocessing 51 is first performed on the point cloud data and the 3D image frame, and then data association 52 is performed on the 3D image frame and the point cloud data in the time dimension, that is, a time stamp is set for the point cloud data and the 3D image frame respectively. Using the workpiece CAD model 61 stored in the database 60, machine vision recognition 53 is performed on the point cloud data to identify the attribute information of the workpiece, and the attribute information includes physical characteristics, measurement parameters, spatial position and surface state. Action recognition 54 is performed on the 3D image frame to identify the action of the worker. Through the comparison process with the SOP 62 stored in the database 60, a combined evaluation is performed on the point cloud data and the worker's movement. In the evaluation process 70, an evaluation value generation process 71 is performed, and an alarm and reminder 72 are displayed.

[0100] Figure 4 is an exemplary structural diagram of an apparatus for evaluating a work task according to an embodiment of the present invention. The evaluation apparatus comprises:

[0101] An acquisition module 401, which is configured to acquire point cloud data of a workpiece in a work task and a three-dimensional image frame of a worker in the work task; a first recognition module 402, which is configured to recognize attribute information of the workpiece in a machine vision manner based on the point cloud data; a second recognition module 403, which is configured to recognize motion information of the worker in a deep learning manner based on the three-dimensional image frame; a comparison module 404, which is configured to compare the combination of attribute information and motion information with a standard operating procedure of the work task, wherein the standard operating procedure includes standard attributes of the workpiece in the work task and standard motions of the worker; and an evaluation module 405, which is configured to evaluate the work task based on the comparison result.

[0102] An embodiment of the present invention further provides an electronic device having a processor-memory architecture. Figure 5 is an exemplary structural diagram of an electronic device according to an embodiment of the present invention. Figure 5 As shown, the electronic device 500 includes a processor 501, a memory 502, and a computer program stored in the memory 502 and running on the processor 501. When the computer program is executed by the processor 501, any of the above methods for evaluating work tasks can be implemented. The memory 502 can be specifically implemented as a variety of storage media such as an electrically erasable programmable read-only memory (EEPROM), a flash memory (Flash memory), a programmable program read-only memory (PROM), etc. The processor 501 can be implemented to include one or more central processing units or one or more field programmable gate arrays, wherein the field programmable gate array integrates one or more central processing unit cores. Specifically, the central processing unit or the central processing unit core can be implemented as a CPU or an MCU or a DSP, etc.

[0103] It should be noted that not all steps and modules in the above process and structure diagram are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed, and can be adjusted as needed. The division of each module is only for the convenience of describing the functional division adopted. In the actual implementation scheme, a module can be implemented by multiple modules, and the functions of multiple modules can also be implemented by the same module. These modules can be located in the same device, or can be located in different devices.

[0104] The hardware modules in each embodiment can be implemented mechanically or electronically. For example, a hardware module may include a specially designed permanent circuit or logic device (e.g., a dedicated processor, such as an FPGA or ASIC) for performing a specific operation. The hardware module may also include a programmable logic device or circuit (e.g., including a general-purpose processor or other programmable processor) temporarily configured by software for performing operations. As for the specific mechanical method, or the use of a dedicated permanent circuit, or the use of a temporarily configured circuit (e.g., configured by software) to implement the hardware module, it can be determined based on cost and time considerations.

[0105] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for evaluating a work task, characterized in that Include: Collecting (101) point cloud data of a workpiece in a work task and a three-dimensional image frame of a worker in the work task; recognizing (102) attribute information of the workpiece in a machine vision manner based on the point cloud data; recognizing (103) the worker's motion information in a deep learning manner based on the three-dimensional image frame; comparing the combination of the attribute information and the action information with a standard operating procedure for the work task (104), wherein the standard operating procedure includes standard attributes of the workpiece in the work task and standard actions of the worker; as well as The work task is evaluated (105) based on the comparison result.

2. The method according to claim 1, characterized in that The attribute information includes the type of the workpiece; the method further includes: When the type of the workpiece does not conform to the standard type of the workpiece in the work task, an alarm message is issued.

3. The method according to claim 1, characterized in that Further including: Determining the real-time point of the action information; Determine a standard action at a next time point of the real-time point based on the standard operating procedure; A reminder message is issued to remind the worker to perform the standard action at the next time point.

4. The method according to claim 1, characterized in that: The work task includes a plurality of work stages, and the standard operating procedure includes a plurality of sub-standard operating procedures corresponding to the plurality of work stages respectively; Wherein, comparing the combination of the attribute information and the action information with the standard operating procedure of the work task comprises: Based on a corresponding comparison with corresponding start actions of the multiple working stages, splitting the combined information into a plurality of sub-combined information, wherein the plurality of sub-combined information respectively corresponds to the multiple working stages; Each sub-assembly information is compared to the corresponding sub-standard operating procedure.

5. The method according to claim 4, characterized in that The plurality of sub-standard operating procedures have corresponding standard working hours; Wherein comparing each sub-combination information with the corresponding sub-standard operating procedure comprises: Compare the actual working hours corresponding to each sub-combination information with the standard working hours of the corresponding sub-standard operating procedures; comparing the attribute information in each sub-assembly information with the standard attributes of the corresponding sub-standard operating procedure; The action information in each of the sub-combination information is compared with the standard action of the corresponding sub-standard operating procedure.

6. The method according to claim 5, characterized in that Further including: Determining an overtime work stage, wherein the actual working hours in the overtime work stage are greater than the corresponding standard working hours; Determining the ratio of the number of overtime work stages to the number of all work stages; When the ratio is less than a predetermined threshold, determining a working tool used by the worker in the overtime working stage; A reminder message is issued to check the working tool.

7. The method according to claim 1, characterized in that The work task of evaluating (105) based on the comparison result includes: When the action information meets the standard action and the attribute information meets the standard attribute, generating a first evaluation value; When the action information conforms to the standard action, the attribute information does not conform to the standard attribute, and the attribute information is within a predetermined attribute interval, generating a second evaluation value; When the action information conforms to the standard action, the attribute information does not conform to the standard attribute, and the attribute information is not within the predetermined attribute interval, generating a third evaluation value; When the action information does not meet the standard action, generating a fourth evaluation value; The first evaluation value is greater than the second evaluation value, the second evaluation value is greater than the third evaluation value, and the third evaluation value is greater than the fourth evaluation value.

8. A system for evaluating a work task, characterized in that: Include: A collection module (10), the collection module being configured to collect point cloud data of a workpiece in a work task and a three-dimensional image frame of a worker in the work task; A server (20), the server being configured to: identify attribute information of the workpiece in a machine vision manner based on the point cloud data; recognizing the action information of the worker in a deep learning manner based on the three-dimensional image frame; comparing the combined information of the attribute information and the action information with a standard operating procedure of the work task, wherein the standard operating procedure contains standard attributes of the workpiece in the work task and standard actions of the worker; and evaluating the work task based on the comparison result.

9. The system according to claim 8, characterized in that The attribute information includes the type of the artifact; The server (20) is configured to issue an alarm message when the type of the workpiece does not conform to the standard type of the workpiece in the work task; the system further comprises: A workstation (30) is configured to display the alarm message.

10. The system according to claim 8, characterized in that The server (20) is configured to: determine the real-time point of the action information; determine the standard action at the next time point of the real-time point based on the standard operating procedure; and send a reminder message to remind the worker to perform the standard action at the next time point. The system further comprises: A workstation (30) is configured to display the reminder message.

11. The system according to claim 8, characterized in that The work task includes a plurality of work stages, and the standard operating procedure includes a plurality of sub-standard operating procedures corresponding to the plurality of work stages respectively; The server (20) is configured to split the combined information into a plurality of sub-combination information based on a corresponding comparison with the corresponding start actions of the plurality of work stages, wherein the plurality of sub-combination information respectively corresponds to the plurality of work stages; and compare each sub-combination information with a corresponding sub-standard operating procedure.

12. The system according to claim 11, characterized in that The plurality of sub-standard operating procedures have corresponding standard working hours; The server (20) is configured to compare the actual working hours corresponding to each sub-combination information with the standard working hours of the corresponding sub-standard operating procedure; compare the attribute information in each sub-combination information with the standard attribute of the corresponding sub-standard operating procedure; and compare the action information in each sub-combination information in the sub-combination information with the standard action of the corresponding sub-standard operating procedure.

13. The system according to claim 12, characterized in that The server (20) is configured to: determine an overtime work stage, wherein the actual working time of the overtime work stage is greater than the corresponding standard working time; determine the ratio of the number of overtime work stages to the number of all work stages; When the ratio is less than a predetermined threshold, determining the working tool used by the worker in the overtime working stage; and issuing a prompt message to check the working tool; the system further comprises: A workstation (30) is configured to display the prompt message.

14. A device for evaluating a work task, characterized in that Include: A collection module (401), the collection module is configured to collect point cloud data of a workpiece in a work task and a three-dimensional image frame of a worker in the work task; A first recognition module (402), the first recognition module being configured to recognize attribute information of the workpiece in a machine vision manner based on the point cloud data; A second recognition module (403), the second recognition module being configured to recognize the action information of the worker in a deep learning manner based on the three-dimensional image frame; A comparison module (404), the comparison module being configured to compare the combination of the attribute information and the action information with a standard operating procedure of the work task, wherein the standard operating procedure contains standard attributes of the workpiece in the work task and standard actions of the worker; as well as An evaluation module (405), configured to evaluate the work task based on the comparison result.

15. An electronic device, characterized in that: It comprises a processor (501) and a memory (502), wherein an application program executable by the processor (501) is stored in the memory (502), so that the processor (501) executes the method for evaluating a work task according to any one of claims 1 to 7.

16. A computer readable medium, characterized in that Contains computer readable instructions stored thereon, wherein the computer readable instructions are used to execute the method for evaluating a work task according to any one of claims 1 to 7.

17. A computer program product, characterized in that The invention comprises a computer program, which is used to perform the method for evaluating a work task according to any one of claims 1 to 7 when the computer program is executed by a processor.

Images