Process management system and device

Abstract

The embodiment of the invention provides a process management system and device, the system comprises an acquisition module and a guide module, the acquisition module is configured to acquire process information of an operator executing a target operation, and the process information is configured to reflect the completion condition of the operator on the target operation; the guiding module comprises a prompting unit and a processing unit, the processing unit is configured to generate guiding information for guiding the operator to implement the target operation, and the prompting unit is configured to control the display equipment to display the guiding information.

Description

Technical Field

[0001] This specification relates to the field of operation process quality management, and particularly to a process management system and device. Background Art

[0002] In engineering projects, in order to ensure the quality of engineering projects, it is often necessary to control the actual process of engineering projects. Usually, engineering personnel need to carry out actual work according to a preset engineering execution process. For example, after engineering personnel view the tasks to be executed on their personal devices, they then go to the corresponding devices to execute the tasks they are responsible for. However, the above process requires operators to spend a lot of time on querying and confirming multiple tasks. Due to the limited energy of operators, the above method lacks efficiency and intelligence.

[0003] Therefore, a process management system and device are needed to facilitate automatically prompting the operator of the operation steps to be executed and the execution methods of the operation steps to be executed, thereby improving the execution efficiency of engineering projects. Summary of the Invention

[0004] One or more embodiments of this specification provide a process management system, which includes: a collection module configured to obtain process information of an operator performing a target operation, and the process information is configured to reflect the completion situation of the operator for the target operation; a guidance module including a prompt unit and a processing unit; the processing unit is configured to generate guidance information for guiding the operator to implement the target operation; the prompt unit is configured to control a display device to display the guidance information.

[0005] One or more embodiments of this specification provide a process management system device, which includes: an input device, a recording device, and a processor; the input device is configured to collect input information of the operator, and the input information includes at least one of time information of the operator implementing the target operation and personnel information of the operator; the recording device is configured to collect process information of the operator implementing the target operation; the processor is configured to: obtain the process information; generate guidance information for guiding the operator to implement the target operation based on the process information; control a display device to display the guidance information. Brief Description of the Drawings

[0006] This specification will be further described in the form of exemplary embodiments, and these exemplary embodiments will be described in detail through the drawings. These embodiments are not restrictive. In these embodiments, the same numbers represent the same structures, where:

[0007] Figure 1 is a schematic diagram of an application scenario of a process management device shown according to some embodiments of this specification;

[0008] Figure 2 is an exemplary module diagram of a process management system shown in some embodiments of this specification;

[0009] Figure 3 is an exemplary flowchart showing the generation of reminder information according to some embodiments of this specification;

[0010] Figure 4 is an exemplary flowchart of another method for generating reminder information according to some embodiments of this specification;

[0011] Figure 5 is an exemplary schematic diagram of a guidance model shown in some embodiments of this specification;

[0012] Figure 6 is an exemplary schematic diagram of guidance information according to some embodiments of this specification. Detailed implementation manners

[0013] To more clearly illustrate the technical solutions of the embodiments of this specification, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some examples or embodiments of this specification. For those of ordinary skill in the art, without creative efforts, this specification can also be applied to other similar scenarios based on these drawings. Unless obvious from the language context or otherwise stated, the same reference numerals in the figures represent the same structure or operation.

[0014] It should be understood that the "system", "device", "unit" and / or "module" used herein is a way to distinguish different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the said words can be replaced by other expressions.

[0015] As shown in this specification and the claims, unless the context clearly indicates an exception, words such as "a", "an", "one" and / or "the" are not specifically singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the clearly identified steps and elements, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0016] Flowcharts are used in this specification to illustrate the operations performed by the systems according to the embodiments of this specification. It should be understood that the previous or subsequent operations do not necessarily need to be executed precisely in sequence. On the contrary, the steps can be processed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or more steps can be removed from these processes.

[0017] Figure 1 It is a schematic diagram of the application scenario 100 of the process management device shown in some embodiments of this specification. As Figure 1 shown, the process management device may include an input device 110, a recording device 120, and a processor 130.

[0018] The input device 110 is configured to collect the input information of the operator.

[0019] The input information refers to the content input by the operator on the input device 110. In some embodiments, the input information may include at least one of the time information of the operator's implementation of the target operation and the personal information of the operator.

[0020] For more content about the input information, reference can be made to Figure 6 and its description.

[0021] In some embodiments, the operator can perform an interactive operation with the input device 110. For example, the input device 110 may include a tablet computer, a VR glasses, a smart watch, etc. The operator can input information on the tablet computer, and the tablet computer can also display information on its display screen.

[0022] In some embodiments, the input device 110 can obtain the input information based on the content input by the operator on the input device 110. For example, the input device 110 can display a plurality of target operation option boxes on its display device. If the operator clicks on the target operation option box corresponding to the target operation that needs to be executed, a start operation option box and an end operation option box will pop up on the display device of the input device 110. If the operator clicks on the start operation option box, the input device 110 will automatically record the current time as the start time of the target operation. If the operator clicks on the end operation option box, the input device 110 will automatically record the current time as the end time of the target operation.

[0023] The target operation refers to the operation corresponding to the input information. In some embodiments, the target operation may include each process in the aircraft final assembly process. For example, aircraft painting, installing screws, and installing wings, etc. For further description of the target operation, see Figure 2 the corresponding content.

[0024] For example, the operation type may be aircraft painting, and the start time and end time respectively correspond to the start time and end time of aircraft painting.

[0025] The personal information refers to the relevant information of the operator. In some embodiments, the personal information may include information such as the operator's name, work number, position, and length of service.

[0026] The recording device 120 is configured to collect process information of an operator performing a target operation. In some embodiments, the recording device 120 may include an image acquisition device, an audio acquisition device, etc.

[0027] An image acquisition device refers to a device used to collect picture information and video information. In some embodiments, the image acquisition device may include one or more image sensors, cameras, etc.

[0028] An audio acquisition device refers to a device used to collect sound information. In some embodiments, the audio acquisition device may include one or more audio sensors, microphones, etc.

[0029] Process information refers to information characterizing the process of an operator actually performing a target operation. In some embodiments, the process information may include sound information, image information, and video information, etc. during the operator's execution of the target operation. For example, the video information may include the work video of the operator performing the target operation, the image information may include images before and / or after the operation at multiple points, and the sound information may include the work voices among multiple operators, the reading voices of the operator, the sounds emitted when the operation tool is working, etc.

[0030] In some embodiments, the process information may be collected by the recording device 120.

[0031] For example, the image acquisition device may use multiple image sensors to collect images at multiple points corresponding to the target operation, and the image acquisition device may also capture the entire process of the operator actually performing the target operation through a camera.

[0032] Also, for example, the audio acquisition device may use an audio sensor to record the reading voice of the operator for the measuring tool, and may also record the prompt voice after the process steps corresponding to the target operation are met or the sounds emitted when the operation tool is working, etc.

[0033] In some embodiments, the process management device may further include a storage device 140 and a display device 150. In some embodiments, the storage device 140 and the display device 150 may be separate device units, or may be integrated into the input device 110, the recording device 120, and other possible device components. For example, the input device 110, the recording device 120, the storage device 140, and the display device 150 may be integrated into a wearable device, such as a VR glasses with an input interface, etc.

[0034] In some embodiments, the recording device 120 may start or end recording under the manual setting of the operator. For example, the operator triggers the corresponding control button to trigger or end the recording.

[0035] In some embodiments, the recording device can trigger or end recording automatically. For example, in response to meeting a preset condition, the recording device can start or end recording automatically. Merely by way of example, the preset condition can be to start recording automatically when a preset component is detected, and to stop recording automatically when the preset component is not detected.

[0036] In some embodiments, when collecting process information, the recording device 120 can send the collected process information to the storage device 140 for storage.

[0037] In some embodiments, the processor 130 is configured to: obtain process information; generate guidance information for guiding an operator to perform a target operation based on the process information; control the display device 150 to display the guidance information.

[0038] In some embodiments, the processor 130 can obtain process information from the recording device 120 or from the storage device 140.

[0039] The guidance information refers to information for prompting an operator's next operation. In some embodiments, the guidance information can prompt the operation to be performed by the operator next and the specific implementation manner of the operation. For details about the guidance information, reference can be made to Figure 2 and its description.

[0040] In some embodiments, the processor 130 can generate guidance information for guiding an operator to perform a target operation based on the process information. For the specific content of generating the guidance information, reference can be made to Figure 2 and its description.

[0041] In some embodiments, the processor 130 can control the display device 150 to display the guidance information.

[0042] In some embodiments, the processor 130 can send the guidance information and the display information instruction corresponding to the guidance information to the display device 150. The display device 150 can receive the guidance information and the display information instruction corresponding to the guidance information, read and parse the display information instruction, and then display the guidance information according to the parsed result.

[0043] Merely for example, when the operator completes the current target operation, the operator can click on the option box corresponding to the current operation on the input device 110 and select the completed option. Then, the processor of the input device 110 automatically sends the next operation to be performed and the display information instruction corresponding to this operation to the VR glasses worn by the operator, and the VR glasses will automatically display the next operation to be performed and the specific implementation manner of the operation.

[0044] In some embodiments, the processor 130 is a cloud processor and can also be integrated into the input device 110, the recording device 120, and other possible device components.

[0045] In some embodiments, the input device 110, the recording device 120, the processor 130, the storage device 140, and the display device 150 can communicate based on the network 160.

[0046] In some embodiments of this specification, the process management device collects the input information of the operator, collects the process information during the actual work process of the operator, and generates guiding information to guide the operator to perform actual operations, improving the work efficiency of the operator and reducing the error rate in actual projects.

[0047] It should be understood that Figure 1 The devices and their modules shown can be implemented in various ways. It should be noted that the above description of the process management device and its modules is only for convenience of description and does not limit this specification to the scope of the examples given. It can be understood that for those skilled in the art, after understanding the principle of the system, they may, without departing from this principle, make any combination of the various modules, or form a subsystem and connect it with other modules. In some embodiments, Figure 1 The input device, the recording device, and the processor disclosed in can be different modules in a system, or a module can implement the functions of two or more of the above modules. For example, each module can share a storage module, or each module can have its own storage module respectively. Such variations are all within the protection scope of this specification.

[0048] Figure 2 is a system module diagram of the process management system according to some embodiments of this specification. In some embodiments, as Figure 2 shown, the process management system 200 includes a collection module 210 and a guidance module 220.

[0049] In some embodiments, the collection module 210 is configured to obtain the process information of the operator performing the target operation.

[0050] The operator refers to the person using the process management system 200. For example, an operator, a manager, etc.

[0051] The target operation refers to the specific operation to be completed by the operator. The target operation can be determined manually according to the actual scenario. For example, in the scenario of aircraft maintenance, the target operation can include individual steps in the aircraft general assembly process (such as aircraft painting, installing screws, and installing wings, etc.).

[0052] The process information is configured to reflect the completion of the target operation by the operator. In some embodiments, the process information includes at least one of sound information, picture information, and video information. For further description of the process information, see Figure 1 the corresponding content.

[0053] In some embodiments, the acquisition module 210 may obtain process information based on acquisition devices (such as cameras, recording devices, etc.).

[0054] In some embodiments, the acquisition module includes a sound acquisition unit 211, an image acquisition unit 212, and an analysis unit 213.

[0055] The sound acquisition unit 211 is configured to acquire sound information.

[0056] The sound information refers to the sound-related information generated by the operator or the operating tool when the operator performs the target operation.

[0057] The image acquisition unit 212 is configured to acquire image information.

[0058] The image information refers to the image-related information generated when the operator performs the target operation. For example, the image information may include two-dimensional image information, three-dimensional image information, etc. Another example is that the image information may include images of a single point, images of multiple points, etc. A point refers to the acquisition point where the acquisition device performs image acquisition. One or more acquisition devices may be arranged near the location where the target operation is performed, and different points can be obtained according to different arrangement methods of the acquisition devices. The arrangement method of the acquisition devices can be set manually according to requirements.

[0059] In some embodiments, the sound information further includes the working voice of the operator and / or the sound emitted by the construction tool when the construction is completed.

[0060] For example, the working voice of the operator may be the voice emitted by the operator when using the construction tool (such as the reading information of the measuring tool), etc.

[0061] Another example is that when the construction tool is a torque wrench, the sound emitted by the construction tool when the construction is completed can be the corresponding sound (such as the sound of "click click click") emitted by the torque wrench when it reaches the set torque. The sound emitted by the construction tool when the construction is completed can be pre-recorded.

[0062] In some embodiments, the image information includes images and / or video information before, during, after, and during inspection of the execution of the target sub-operation.

[0063] The target sub-operation refers to the working steps of the target operation. The target operation includes one or more target sub-operations. For more content about the target sub-operation, see Figure 3 and related descriptions.

[0064] In some embodiments, the image information includes the pre-execution, in-execution, post-execution, and inspection-time images and / or video information of one or more points of the target sub-operation. In some embodiments, the acquisition module can determine whether the image and / or video is the pre-execution, in-execution, post-execution, or inspection-time image and / or video information by comparing the acquisition time of the image and / or video information with the start execution time, end execution time, and start inspection time recorded by the user at the start, end, and inspection of the target sub-operation. The above images can be images taken before execution, during execution, after execution, or at the inspection time, or can be images automatically extracted by the acquisition module after collecting the video.

[0065] In some embodiments, the above images include standard detection images. A standard detection image refers to a detection image based on operation process standards. For example, when detecting whether it meets the process standards, the image information includes an image of the reading of the detection tool.

[0066] In some embodiments, the image information may further include depth information. Depth information refers to information related to the distance from the acquisition device to each point. By acquiring the sound information and image information of the entire process of the operator and construction tools of the target operation, the operation process can be recorded more comprehensively and accurately, reducing errors caused by manual recording, etc., so that the subsequent provided guidance information is more accurate, thereby guiding the operator to perform correct operations.

[0067] In some embodiments, collecting the sound emitted by the construction tool when the construction is completed includes: collecting the sound data of the construction process and detecting the sound data through a sound recognition algorithm to achieve collecting the sound emitted by the construction tool when the construction is completed.

[0068] Sound data refers to sound-related data generated by the construction tool during the construction process.

[0069] A sound recognition algorithm is an algorithm used to recognize sound data. For example, the sound recognition algorithm can be a DTW (Dynamic Time Wrap) algorithm, etc.

[0070] In some embodiments, the sound acquisition unit can identify the sound emitted by the construction tool when the construction is completed based on the DTW algorithm through the following steps: collecting the sound data of the actual construction process as the sound data to be recognized; dividing the sound data to be recognized into multiple frames according to a preset frame length (for example, taking the frame length as 20 ms); performing optimization processing on the multiple frames corresponding to the sound data to be recognized to obtain multiple optimized frames; calculating multiple frame matching distances between the multiple optimized frames and the frames corresponding to the standard sound data based on the DTW algorithm, finding the shortest path among the frame matching distances for each frame, and calculating the cumulative distance corresponding to the preset frame (i.e., the sum of the distances of each shortest path). When the cumulative distance of the shortest path is less than the preset distance threshold, it is considered that the standard sound data is recognized. The standard sound data refers to the sound emitted when the construction tool meets the standard completion condition and is pre-recorded.

[0071] The standard completion condition can be set according to experience. For example, the standard sound data can be the "click, click, click" sound emitted when the torque wrench reaches the set value. The sound acquisition unit can identify the sound data based on the DTW algorithm. When the distance between the recognized sound data and the "click, click, click" sound is less than the preset distance threshold, the corresponding sound data is recorded and saved. In some embodiments, the sound acquisition unit can obtain a sound recognition algorithm capable of detecting the "click, click, click" sound after comparing and training based on the "click, click, click" sound emitted when a specific model of torque wrench reaches the set value in the historical data.

[0072] In some embodiments, during the aforementioned optimization processing, the sound acquisition unit can perform windowing processing on the multiple frames corresponding to the sound data to be recognized, and multiply by a window function (such as a Hamming window) to reduce the Gibbs effect caused by truncation processing, thereby maintaining the short-term stationarity of the speech signal. In some embodiments, in order to only store and process valid information, the acquisition module can find the start point and end point of the sound data to be recognized through an endpoint detection method. Among them, the endpoint detection method can be a double-threshold endpoint detection of short-time energy and short-time zero-crossing rate.

[0073] Through the sound recognition algorithm, the monitoring and automatic recording of the usage process of the construction tool are realized, preventing accidents caused by the operator's illegal use of the construction tool.

[0074] The analysis unit 213 is configured to generate process information based on the sound information collected by the sound acquisition unit 211 and the image information collected by the image acquisition unit 212. The process information includes the analysis result.

[0075] In some embodiments, the analysis unit 213 can analyze the sound information and the image information to obtain the analysis result. The analysis result at least reflects the completion status of the corresponding operation (such as whether it is completed, whether the completion meets the standard, etc.). The analysis result of the analysis unit will be sent to the receiving unit of the guidance module. For the guidance module, reference can be made toFigure 2 The following text and related descriptions.

[0076] In some embodiments, the analysis unit 213 may determine whether a target sub - operation corresponding to a target operation is completed based on sound information. For example, the analysis unit may directly determine that the target sub - operation is completed based on the sound information containing preset keywords. The preset keywords can be preset in advance, such as keywords related to the completion of an operation like "completed", "ended", etc. Another example is that the analysis unit may determine the completion of the target sub - operation based on the sound emitted by the construction tool when the construction is completed. The sound emitted by the construction tool when the construction is completed can be recognized through a sound recognition algorithm. For details, please refer to Figure 2 The above text and related descriptions.

[0077] In some embodiments, the analysis unit 213 may determine whether one or more target sub - operations are completed based on the image information after the execution of the target sub - operations at one or more points, so as to obtain an analysis result on whether the current operation is completed. For example, the analysis unit 213 may calculate the similarity between the image after the execution of the target sub - operations at one or more points and the standard image of the completed operation, and determine that the current operation is completed based on the similarity being less than the similarity threshold. The standard image can be obtained by manually collecting the image after the operation is completed under standard processes.

[0078] Another example is that the analysis unit 213 may determine whether the operation sequence of each target sub - operation is correct based on the shooting time of the images in the operations at each point. For example, the analysis unit 213 may sort the shooting times of the images in the operations at each point to obtain the actual operation sequence of the target sub - operations corresponding to each point, and compare whether the actual operation sequence is consistent with the preset operation sequence, so as to respectively determine the target sub - operations with correct and incorrect operation sequences in the actual operation sequence.

[0079] Still another example is that the analysis unit 213 may determine the dimensional information of the parts corresponding to different points based on the image information of different points. Exemplarily, when the collected image information includes the placement and measurement screen of a vernier caliper, the analysis unit 213 may determine the reading of the vernier caliper based on this placement and measurement screen. Based on the reading of the vernier caliper, it is determined whether the actual target sub - operation meets the process standard, and an analysis result on whether the current operation meets the standard is obtained.

[0080] Through the sound acquisition unit, image acquisition unit and analysis unit of the acquisition module, intelligent acquisition and analysis of images and sounds during the operation process can be realized, which can comprehensively record the actual completion situation of the target operation and facilitate accurate guidance for the target operation in the follow - up.

[0081] In some embodiments, the guidance module 220 is configured to guide an operator to perform a target operation. The guidance module can be used to give prompts to workers to guide them to implement specific operations. That is, the guidance module can prompt and manage the entire process, such as guiding / requiring the next step and the specific operations of the next step based on the completed steps, and the required completion criteria (such as recording, completing the specified operation, and supervising and confirming the accurate completion of the operation), etc.

[0082] In some embodiments, the guidance module 220 includes a prompt unit 221 and a processing unit 222.

[0083] In some embodiments, the processing unit 222 is configured to generate guidance information for guiding an operator to perform a target operation.

[0084] The guidance information is prompt information for prompting the operator. For example, the guidance information can include operation prompt information, solution prompts, and prompt the operator to record the operation progress of the current operation, etc. The solution prompts can include prompting to play a teaching video, prompting to call a corresponding contact person, etc.

[0085] In some embodiments, the processing unit 222 can generate guidance information based on various methods. For example, the processing unit 222 can generate operation prompt information for prompting the operator to perform the next step in response to the completion time of the previous operation being less than or equal to a preset time; in response to the completion time of the previous operation being greater than the preset time, the processing unit 222 can generate a solution prompt. The completion time of the previous operation can be the difference between the end execution time and the start execution time of the operation. The end execution time and the start execution time can be the start execution time and the end execution time of the target operation in the user input information. Regarding the input information, refer to Figure 2 the description below.

[0086] For another example, the processing unit 222 can generate operation prompt information and / or solution prompts for prompting the operator to complete the current operation based on the start execution time or the end execution time that the acquisition module has not received from the user within a preset time period (i.e., considered to be a timeout). For another example, the processing unit 222 can generate guidance information based on the analysis result of the analysis unit, such as generating prompt information indicating that the current operation does not meet the standard based on the analysis result of the analysis unit that the current operation does not meet the standard, and the operator needs to re-complete the current operation based on this prompt information.

[0087] In some embodiments, the prompt unit 221 is configured to control a display device to display the guidance information.

[0088] In some embodiments, the display device can be configured as a device with a display function such as a tablet computer, a VR glasses, a smart watch, etc. For more descriptions, refer to Figure 1 .

[0089] In some embodiments, the prompting unit 221 may control a display device to display guiding information based on the guiding information generated by the processing unit.

[0090] In some embodiments, the guiding module 220 further includes a receiving unit 223.

[0091] In some embodiments, the receiving unit 223 is configured to receive the process information acquired by the acquisition module and / or the input information 620 input by an operator.

[0092] The input information refers to the information related to the target operation input by the operator. The input information may be voice, text, image, etc. In some embodiments, the input information includes feedback information on the completion of the target operation, such as whether the target operation is completed, the completion progress of the target operation, the start execution time and the end execution time of the target operation, etc.

[0093] For the content of the guiding information and the process information, reference may be made to the above and related descriptions.

[0094] In some embodiments, the processing unit 222 is further configured to generate guiding information based on the process information and / or the input information received by the receiving unit 223, and send it to the prompting unit 221; the prompting unit 221 controls the display device to display the guiding information.

[0095] For example, the processing unit 222 may directly generate guiding information based on the input information input by the operator, or may directly generate guiding information based on the process information. For example, based on the "current operation has been completed" input by the operator or the analysis result of the analysis unit, guiding information for prompting the operator to perform the next operation and / or how to perform the next operation is generated.

[0096] In some embodiments, the processing unit 222 may compare and verify the input information input by the operator with the process information, and determine the guiding information based on the verification result.

[0097] For example, verify whether the target operation input by the operator is completed and the completion progress of the target operation, and whether it is consistent with whether the target operation is completed and the completion progress of the target operation obtained based on the analysis unit. In response to the input information being inconsistent with the process information, the processing unit 222 may generate guiding information prompting re-operation or re-inspection; in response to the input information being consistent with the process information, the processing unit 222 may generate operation prompt information for the next operation.

[0098] Among them, the analysis unit may determine the completion progress of the target operation based on the number of completed target sub-operations in the process information. Exemplarily, the target operation includes 10 target sub-operations, and 8 target sub-operations have been completed currently, then the completion progress of the target operation is 80%.

[0099] By receiving the process information and input information through the receiving unit, the information transmission between the acquisition module and the guidance module can be realized. The receiving unit can cooperate with the prompting unit and the processing unit to realize the guidance and prompting function of the guidance module.

[0100] Compared with manual recording, through the cooperation of the acquisition module and the guidance module, the sound information and image information during the operation process can be collected more accurately and completely, and the operation situation can be evaluated and analyzed to give targeted guidance information, guiding the operation to be completed correctly and in a timely manner, thereby improving the execution efficiency of the engineering project.

[0101] Figure 3 It is an exemplary flowchart for generating guidance information shown in some embodiments of this specification.

[0102] The target sub-operation refers to the working steps of the target operation. The target operation includes one or more target sub-operations. For example, if the target operation is aircraft installation, the target sub-operations corresponding to the target operation may include the structural assembly of the aircraft, the installation of in-cabin systems and equipment, etc.

[0103] In some embodiments, the processing unit 222 is further configured to execute the method described in process 300. Process 300 is a process for generating guidance information for target sub-operations based on process information, including the following steps 310 to 330.

[0104] Step 310, based on the process information, determine whether the current target sub-operation is completed.

[0105] The current target sub-operation refers to the target sub-operation to be completed currently. In some embodiments, the processing unit 222 may determine the corresponding target sub-operation as the current target sub-operation based on the current image information satisfying the preset trigger condition of the target sub-operation. The preset trigger condition may be detecting a preset component, etc. For example, the processing unit 222 may detect the preset component based on a target detection algorithm, a target recognition algorithm, etc.

[0106] In some embodiments, the processing unit 222 may directly determine whether the current target sub-operation is completed based on the process information. For example, the processing unit may determine that the current target sub-operation is completed based on the analysis result in the process information generated by the analysis unit of the acquisition module being completed.

[0107] Step 320, in response to the completion of the current target sub-operation, generate guidance information for the next target sub-operation.

[0108] The next target sub-operation refers to the next target sub-operation that needs to be completed after the current target sub-operation is completed, and the order of the target sub-operations corresponding to the target operation can be preset manually. The guiding information of the next target sub-operation is the prompt information for prompting the operator to perform the next target sub-operation. For example, the reminder information for the next target operation to be completed, the completion method and process standard of the next target operation, etc.

[0109] In some embodiments, the processing unit 222 may generate guiding information for the next target sub-operation in response to the completion of the current target sub-operation. The guiding information may be voice, text, or image. For more information about the guiding information, see Figure 2 and related descriptions.

[0110] Step 330, in response to the current target sub-operation not being completed, generate reminder information to control the display device to display the reminder information based on the prompting unit.

[0111] The reminder information refers to the prompt information for reminding the operator to perform the current target sub-operation. In some embodiments, the reminder information includes the guiding information of the current target sub-operation that is not completed. For example, at least one of voice, text, or image that reminds the operator that the current target sub-operation is not completed. The reminder information may include the operation information corresponding to the target sub-operation.

[0112] The operation information may include the position information of the operation points where the target sub-operation needs to be performed, the operation sequence information, the operation method information of the specific operations to be performed at each point, the standard information of the process standard to be achieved, etc. The position information reflects the operation points where the worker needs to operate; the operation sequence information reflects the order in which the operations at these operation points should be performed; the operation method information reflects how the specific operations at each operation point should be carried out; the standard information reflects the information of the process standard that should be achieved after the operations at each operation point and / or the operations at a part of the operation points are completed. The operation information can be displayed in the form of text, image (static or dynamic), etc. For more descriptions about the operation points and the position information, see Figure 6 the corresponding content.

[0113] In some embodiments, the processing unit 222 may generate reminder information in response to the current target sub-operation not being completed, so as to control the display device to display the reminder information based on the prompting unit. For example, when the actual operation sequence does not meet the operation sequence information, the processing unit generates a reminder of an incorrect operation sequence and generates the correct operation sequence and displays it on the display interface. Another example is that when there are operation points where the current target sub-operation has uncompleted operations, the processing unit generates information indicating which operation points are not completed and the corresponding operation method information. The operator can perform corresponding processing based on the reminder information displayed on the display interface of the display device.

[0114] In some embodiments, the processing unit 222 may control the display device to display a reminder message based on the prompting unit. For example, the processing unit 222 may control the virtual reality screen based on the prompting unit to display corresponding operation information. In order to make the standard information collected by the following acquisition module more accurate, if the display device is a VR glasses, a virtual position indication mark may be displayed to guide the worker to adjust the shooting angle. For example, when a virtual mark of a vernier caliper is displayed, the operator needs to align the actual vernier caliper with the virtual mark.

[0115] In some embodiments, the processing unit is further configured to: in response to the prompting time interval satisfying a preset time condition, perform a preset assistance operation.

[0116] The prompting time interval is the time interval between the current time and the time when the prompting unit last sent a reminder message.

[0117] The preset time condition is a condition satisfied by the preset prompting time interval. In some embodiments, the preset time condition may be that the prompting time interval is greater than a preset threshold.

[0118] The preset assistance operation is an operation that can assist the operator to complete the target operation and is preset. For example, the preset assistance operation may include a provided solution.

[0119] In some embodiments, the preset assistance operation includes playing teaching information corresponding to the current target sub-operation and / or contacting a preset contact person.

[0120] The teaching information may include a teaching video or diagrammatic information for completing the target sub-operation, etc.

[0121] The preset contact person is a contact person corresponding to the current target sub-operation and is preset. The preset contact person may be a technician or a manager corresponding to the current target sub-operation.

[0122] Playing the teaching video can help the operator more clearly understand the correct operation method of the target sub-operation, and calling the corresponding contact person can help the operator solve the problems of doubts or difficulties in the operation.

[0123] When the target sub-operation has not been completed for a long time, it may be that the worker is not proficient in the operation or does not know how to operate. Therefore, the preset assistance operation can assist the operator to complete the target sub-operation in time, improve the completion rate of the target operation, and thus improve the work efficiency.

[0124] By collecting the process information of the acquisition module, it is determined whether the target sub-operation is completed, and corresponding guiding information is generated for the completed and uncompleted situations respectively, guiding the operator to complete the target sub-operation or perform the next target sub-operation. The above method can directly determine whether the target sub-operation is completed based on the information of the acquisition module, improving the speed of determination, so that guiding information can be provided in a timely manner, improving the efficiency of the operator to complete the target sub-operation, and thus improving the completion efficiency of the target operation. At the same time, different guiding schemes can be provided for different scenarios, thereby improving the guiding effect.

[0125] It should be noted that the above description of process 300 is only for illustration and explanation, and does not limit the scope of application of this specification. For those skilled in the art, various modifications and changes can be made to process 300 under the guidance of this specification. However, these modifications and changes are still within the scope of this specification.

[0126] Figure 4 It is an exemplary flowchart of generating guiding information shown according to some embodiments of this specification.

[0127] In some embodiments, the processing unit 222 is further configured to execute the method described in process 400. Process 400 is a process of generating guiding information for target sub-operations based on input information and process information, including the following steps 410 to step 430.

[0128] Step 410, determine whether the completion status of the target operation reflected by the input information and the process information is consistent.

[0129] Regarding the input information and the process information, reference can be made to Figure 2 and the relevant descriptions.

[0130] The completion status of the target operation may include the current completion progress of the target operation, etc.

[0131] In some embodiments, the processing unit 222 may determine the first completion status of the target operation based on input information such as voice, text, and images input by the operator, and the first completion status may include the current completion progress.

[0132] For example, the processing unit 222 may determine the ratio of the number of completed target sub-operations currently input by the user to the total number of target sub-operations included in the target operation as the current completion progress of the first completion status. Exemplarily, the target operation includes m target sub-operations, and the number of completed target sub-operations currently input by the user is n1, then the current completion progress in the first completion status is k1, and k1 is the ratio of n1 to m.

[0133] In some embodiments, the processing unit 222 may determine the second completion status of the target operation based on the process information, and the second completion status may also include the current completion progress.

[0134] For example, the processing unit 222 may determine the proportion of the number of completed target sub-operations in the analysis result among all the target sub-operations included in the target operation as the current completion progress of the second completion status. Exemplarily, the target operation includes m target sub-operations, and n1 completed target sub-operations are determined based on the process information. Then the current completion progress in the second completion status is k2, and k2 is the ratio of n2 to m.

[0135] In some embodiments, when the first completion status is the same as the second completion status, such as when the current completion progress of the first completion status is the same as the current completion progress in the second completion status, that is, when k1 is equal to k2, it is determined that the completion status of the target operation of the input information and the process information is consistent.

[0136] Step 420, in response to the consistent completion status being reflected, generate guiding information for the next target sub-operation.

[0137] In some embodiments, the processing unit 222 may, in response to the consistent completion status being reflected, generate prompt information indicating that the next target sub-operation can start operating, as well as the operation method and operation sequence of the next target sub-operation, etc.

[0138] Step 430, in response to the inconsistent completion status being reflected, generate reminder information to control the display device to display the reminder information based on the prompt unit, and the reminder information includes guiding information for the current target sub-operation.

[0139] In some embodiments, the processing unit 222 may generate guiding information through various methods in response to the inconsistent completion status being reflected. For example, the processing unit may determine the corresponding guiding information based on the operator's interaction screen operations, voice recognition, etc. Exemplarily, when the operator selects the current fuselage spraying operation on the interaction screen, the processing unit generates guiding information on the fuselage spraying method; when the operator says "Start guiding: fuselage spraying", the processing unit generates guiding information on the fuselage spraying method after recognizing the voice.

[0140] For another example, the processing unit may, in response to the inconsistent completion status being reflected, automatically generate prompt information reminding that the current target sub-operation needs to be confirmed and completed and / or solution prompts for the current target sub-operation, etc. For the solution prompts, reference can be made to Figure 2 and related descriptions.

[0141] In some embodiments, the processing unit is further configured to: determine the operation scores of the operator for each target sub - operation based on the input information and the process information; and determine the construction score of the operator for the target operation based on the operation scores of the operator for each target sub - operation.

[0142] The operation score is a score reflecting the completion of the target sub - operation by the operator. The operation score can be represented by a grade or a number.

[0143] In some embodiments, the processing unit can determine the operation scores of the operator for each target sub - operation based on the input information and the process information through various methods. For example, the processing unit can determine the operation duration of the operator for the target sub - operation based on the input information and the process information, and determine the operation score of the operator for each target sub - operation based on the operation duration through a preset relationship. The preset relationship refers to the relationship between the operation duration and the operation score of each target sub - operation set in advance. For example, the operation duration and the operation score can be in an inverse relationship, that is, the longer the operation duration, the lower the operation score.

[0144] For another example, the processing unit can determine the operation scores of the operator for each target sub - operation based on the number of repeated executions or the execution time when the operator reaches the preset conditions (such as passing the detection) for the target sub - operation. Exemplarily, if the detection is passed in one operation, the score is between 90 points and 100 points; if the detection is passed after 3 repeated operations or the execution time of one operation exceeds the preset time threshold, the score is between 60 points and 70 points.

[0145] For another example, the processing unit can determine the operation scores of the operator for each target sub - operation based on the proportion of the number of qualified finished products completed by the operator in a target sub - operation. Exemplarily, if the operator completes 100 finished products in an operation step and 80 of them are detected as qualified, the score for this step is 80 points.

[0146] The construction score is a score reflecting the overall situation of the operator's completion of the target operation. The construction score can be represented by a grade or a number.

[0147] In some embodiments, the processing unit can determine the construction score of the operator for the target operation based on the operation scores of the operator for each target sub - operation through various methods. For example, the processing unit can use the average value of the operation scores of the operator for each target sub - operation as the construction score of the operator for the target operation. For another example, the processing unit can use the weighted average value of the operation scores of each target sub - operation as the construction score of the operator for the target operation, and the weights of each target sub - operation are determined based on the importance of the target sub - operation.

[0148] By scoring the completion of the target operations of the operators and one or more corresponding target sub-operations, the operation levels of different operators in different operation steps can be evaluated, facilitating subsequent targeted guidance for different operators and different operation steps.

[0149] In some embodiments, the processing unit may further be configured to: determine the guidance levels of the operators for each target sub-operation based on the operation scores of the operators for each target sub-operation.

[0150] The guidance level refers to the degree of guidance information that needs to be provided to the operator. The guidance level can be represented by numbers, letters, or words indicating degrees, etc.

[0151] In some embodiments, the processing unit can determine the guidance levels of the operators for each target sub-operation through various methods based on the operation scores of the operators for each target sub-operation. For example, the processing unit can divide the operation score ranges of the operators for each target sub-operation into multiple gradient ranges according to a preset gradient, and different gradient ranges correspond to different guidance levels.

[0152] Exemplarily, the operation score range of the target sub-operation is [0 - 100], and the processing unit can divide the score range into [0 - 60), [60 - 80), [80 - 100], corresponding to guidance levels 1, 2, and 3 respectively. The processing unit can also determine the guidance level based on a guidance model, and for details, please refer to Figure 5 and related descriptions.

[0153] The processing unit can provide different guidance information to operators corresponding to different guidance levels. For example, for operators with a guidance level of 1, the processing unit can provide various guidance information such as image prompts, text prompts, sound prompts, risk prompts, and operation key point prompts; for operators with a guidance level of 2, the processing unit can provide one or two of the guidance information such as image prompts, text prompts, sound prompts, risk prompts, and operation key point prompts; for operators with a guidance level of 3, no guidance information is provided.

[0154] By determining different guidance levels according to the different operation scores of the operators, different degrees of guidance can be provided according to the levels of the operators. Provide various different guidance information to unskilled operators and omit the guidance information for skilled operators, thereby improving the accuracy and effectiveness of the guidance.

[0155] In some embodiments of this specification, based on whether the completion status of the target operation reflected by the input information and the process information is consistent, the guiding information and reminder information of the target sub-operation. Since the operator may have problems such as filling in non-standardly or reversing the filling order when recording, the above method can reduce the judgment error rate caused by input errors and non-standard problems in the manual input process, thereby improving the accuracy of the guiding information provided subsequently, and thus improving the guiding effect.

[0156] Figure 5 It is an exemplary schematic diagram for determining the guiding level through a guiding model as shown in some embodiments of this specification.

[0157] In some embodiments, as Figure 5 shown, the processing unit 222 can determine the operation characteristics 531 of the operator based on the historical operation sequence data 510 of the operator in the target operation through the feature extraction layer 520 of the guiding model 501; based on the operation characteristics 531, the operation information 532 of the target operation, and the completed operation information 533, determine the guiding level 550 of the remaining target sub-operations through the level determination layer 540 of the guiding model 501.

[0158] The historical operation sequence data 510 refers to the sequence data related to the target operation of the same operator in the historical data. The historical operation sequence data 510 includes the types of target sub-operations corresponding to the target operation and the operation scores of the target sub-operations, such as the types and operation scores of the target sub-operations completed by the operator in all historical or preset historical time periods. Exemplarily, the historical operation sequence data can be expressed as [(a1, b1), (a2, b2), (a3, b3) ……], where a1, a2, a3 …… respectively represent the types of target sub-operations completed in the preset historical time period, and a1, a2, a3 …… respectively represent the operation scores corresponding to the target sub-operations completed in the preset historical time period. For more content about the operation score, reference can be made to Figure 4 and related descriptions.

[0159] The operation characteristics 531 are characteristic data reflecting the comprehensive operation proficiency of the operator for the target operation. For example, the operation characteristics may include that the operator is unfamiliar, relatively proficient, proficient, etc. with the target operation.

[0160] The operation information is information related to the operation process, operation method, etc. The operation information 532 of the target operation includes the number of target sub-operations of the target operation, the operation difficulty of different target sub-operations, etc.

[0161] The completed operation information 533 includes a sequence composed of the operation scores of at least one target sub-operation completed by the operator. The completed operation information can reflect the current working state of the operator.

[0162] The remaining target sub-operations refer to the remaining target sub-operations to be completed in the target operation.

[0163] The guidance model 501 is a model used to determine the guidance level of the target sub-operations. In some embodiments, the guidance model can be a machine learning model. For example, the guidance model can be a combination of one or more of a Deep Neural Networks (DNN) model, a Neural Networks (NN), and a Recurrent Neural Network (RNN) model. The input of the guidance model includes historical operation sequence data 510, completed operation information 533, and operation information 532 of the target operation; the output includes the guidance level 550 of the remaining target sub-operations.

[0164] In some embodiments, the guidance model 501 includes a feature extraction layer 520 and a level determination layer 540.

[0165] The feature extraction layer is a model used to determine operation features. The feature extraction model can be a machine learning model, such as a Recurrent Neural Network (RNN), etc. The input of the feature extraction model includes historical operation sequence data 510; the output includes operation features 531.

[0166] The level determination layer is a model used to determine the guidance level. The level determination model can be a machine learning model, such as a Neural Networks (NN), etc. The input of the level determination layer includes the operation features 531 output by the feature extraction layer, and the input of the level determination layer can also include completed operation information 533 and operation information 532 of the target operation; the output includes the guidance level 550 of the remaining target sub-operations.

[0167] In some embodiments, the feature extraction layer and the level determination layer can be jointly trained through multiple sets of training samples. Each set of training samples for joint training includes sample historical operation sequence data, sample completed operation information, and operation information of the sample target operation. The training samples can be obtained based on historical data. The label corresponding to the training sample can be the guidance level of the remaining sample target sub-operations. The first label can be manually marked. For example, for the remaining sample target sub-operations determined to require key guidance, the marked label is 1 (corresponding to a guidance level of level 1); otherwise, the marked label is 0 (corresponding to a guidance level of level 3).

[0168] The processing unit can input the sample historical operation sequence data into the feature extraction layer to obtain the sample features output by the feature extraction layer; use the sample features as training data, and input them together with the sample completed operation information and the operation information of the sample target operation into the level determination layer to obtain the guidance level of the remaining target sub-operations output by the level determination layer. Construct a loss function based on the guidance level of the remaining sample target sub-operations determined manually and the guidance level output by the level determination layer, and synchronously update the parameters of the feature extraction layer and the level determination layer. Through parameter update, the trained feature extraction layer and level determination layer are obtained.

[0169] In some embodiments of the present specification, based on the historical operation sequence data, the completed operation information, and the operation information of the target operation, the guidance level is judged through a guidance model, thereby greatly improving the accuracy of the guidance level judgment, so that the different guidance information provided according to the guidance level subsequently is more in line with the actual needs.

[0170] Figure 6 It is an exemplary schematic diagram of the guidance information shown in some embodiments of the present specification. As Figure 6 shown, the guidance information 630 includes the first guidance information 6310 and / or the second guidance information 6320; the first guidance information 6310 prompts the next target sub-operation 610, and the second guidance information 6320 prompts the operation information 6110 of the next target sub-operation 610.

[0171] The first guidance information 6310 refers to the information used to prompt the operator what sub-operation the next target sub-operation 610 is. For example, the type, name, etc. of the next target sub-operation 610.

[0172] In some embodiments, the next target sub-operation 610 refers to the target sub-operation that needs to be executed next corresponding to the current target sub-operation. For example, if the current target sub-operation has been completed, the next target sub-operation 610 is the name of the next sub-operation that needs to be executed.

[0173] In some embodiments, the next target sub-operation 610 can be prompted by a prompting unit. Details on how to prompt the next target sub-operation 610 can be referred to Figure 3 、 Figure 4 and its related descriptions.

[0174] The second guidance information 6320 refers to the information used to guide the operator on how to execute the next target sub-operation 610. In some embodiments, the second guidance information 6320 may include the operation information 6110 of the next target sub-operation 610. For example, the second guidance information 6320 may include the position, execution method, and execution specifications of the next target sub-operation 610.

[0175] In some embodiments, the guiding information may be determined based on the input information 620 of the operator.

[0176] The input information 620 refers to the content input by the operator on the input device 110. In some embodiments, the input information 620 may include at least one of voice information 6210 and interactive operation information 6220. For more information about the input information, reference may be made to Figure 1 and its related descriptions.

[0177] In some embodiments, both the voice information 6210 and the interactive operation information 6220 may include the operator's demand for help information, execution progress information, etc. For example, the operator may input the steps that need guidance on the input device 110 by voice or text, or input the progress of the current operation on the input device 110 by voice or text.

[0178] For specific details on how to obtain the input information, reference may be made to Figure 1 and its related descriptions.

[0179] In some embodiments, the guiding information 630 may be determined based on the input information 620. For example, if the operator, based on the input device 110 (such as a tablet computer, etc.), inputs "airplane painting, need help" by voice or text, then on the display device 150 (such as VR glasses worn by the operator, a tablet computer, etc.), multiple target sub-operations for completing the airplane painting, as well as the operation methods, operation sequences, and process standards corresponding to the multiple target sub-operations, etc., may be displayed.

[0180] In some embodiments of this specification, generating guiding information based on the content input by the operator can provide a scientific reference standard for the operator during actual work, so as to ensure the standardization of the operation process.

[0181] In some embodiments, the target sub-operations include one or more operation points. The operation information 6110 includes at least one of the point information 6110-1 of the operation points included in the next target sub-operation 610, the operation sequence information 6110-2 of the operation points, the operation method information 6110-3 of the operation points, and the process standard information 6110-4 corresponding to the operation points.

[0182] The operation point 6110 refers to the point corresponding to the step to be executed involved in the target sub-operation. In some embodiments, one or more operation points may correspond to one or more steps to be executed. For example, if the target sub-operation is airplane painting, the operation points corresponding to the airplane painting may include the determination of the paint and its dosage, the acquisition of the paint, the determination and acquisition of the painting tool, and the painting of the paint, etc.

[0183] The point location information 6110-1 refers to the specific information of the operation point location. In some embodiments, the point location information may include the specific information of the steps to be executed corresponding to the operation point location. For example, if the target sub-operation is aircraft painting, the operation point locations corresponding to aircraft painting may include the determination of the paint and its dosage, the acquisition of the paint, the determination and acquisition of the painting tool, and the painting of the paint, etc. The point location information corresponding to the determination of the paint and its dosage may include the type of the paint, the prediction of the paint dosage, etc. The point location information corresponding to the acquisition of the paint may include the determination of the transportation method of the paint, etc. The point location information corresponding to the determination of the tool may include the selection of various tools for painting the paint, etc. The point location information corresponding to the painting of the paint may include the determination of the area to be painted, etc.

[0184] The operation sequence information 6110-2 refers to the information of the execution sequence of the steps to be executed corresponding to the operation point location. In some embodiments, the operation sequence information may include the execution sequence of multiple operation point locations. For example, if the target sub-operation is aircraft painting, the operation point locations corresponding to aircraft painting may include the determination of the paint and its dosage, the acquisition of the paint, the determination and acquisition of the painting tool, and the painting of the paint, etc. That is, the operation sequence information corresponding to aircraft painting may be: first execute "determine the type of paint and estimate the paint dosage", then execute "acquire the paint and acquire the painting tool", and finally execute "paint the paint".

[0185] The operation method information 6110-3 refers to the information of the specific implementation method of the steps to be executed corresponding to the operation point location. In some embodiments, the operation method information may include the specific implementation method of the operation point location.

[0186] For example, if the target sub-operation is aircraft painting, the operation point locations corresponding to aircraft painting may include the determination of the paint and its dosage, the acquisition of the paint, the determination and acquisition of the painting tool, and the painting of the paint, etc. The multiple operation method information corresponding to aircraft painting is the implementation method of each operation point location corresponding to aircraft painting.

[0187] Merely for example, the operation method information corresponding to "the determination of the paint and its dosage" may include: determine the specified type of paint and its dosage, and check the inventory of this type of paint, etc.; the operation method information corresponding to "the acquisition of the paint" may include: determine the storage location of the paint, and the pick-up and drop-off conditions of this type of paint (for example, pick-up and drop-off containers, transportation methods, etc.); the operation method information corresponding to "the determination and acquisition of the painting tool" may include: determine the required painting tool, the storage location of this painting tool, and the preconditions for taking out this painting tool (for example, certificates, passwords, uses, etc.); the operation method information corresponding to "the painting of the paint" may include: determine the spatial location sequence of painting the paint on the aircraft, the precautions during actual painting in the corresponding area, etc.

[0188] The process standard information 6110-4 refers to the information of the process standard for the operation points. In some embodiments, the process standard information may include the execution specifications and standards for the steps to be executed corresponding to the operation points. For example, if the target sub-operation is aircraft painting, the operation points corresponding to aircraft painting may include the determination standards for the paint and its dosage, the acquisition standards for the paint, the determination standards and acquisition of the painting tools, and the standards for painting the paint, etc. The multiple process standard information corresponding to aircraft painting are respectively: the types of available paints and their inventory standards, the maximum and minimum values of the preset paint dosage, the preset maximum time for transporting the paint, the preset measurement value of the uniformity of aircraft painting, etc.

[0189] In some embodiments of this specification, by providing the operator with guidance and reference for specific process steps through the automatically generated guiding information, it can ensure the accuracy, consistency, and repeatability of the manufacturing process to ensure the product quality and production efficiency.

[0190] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only an example and does not constitute a limitation to this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are proposed in this specification, so such modifications, improvements, and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0191] Meanwhile, this specification uses specific terms to describe the embodiments of this specification. Such as "one embodiment", "an embodiment", and / or "some embodiments" mean a certain feature, structure, or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "an embodiment" or "one embodiment" or "an alternative embodiment" mentioned twice or more at different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0192] In addition, unless clearly stated in the claims, the order of the processing elements and sequences, the use of numbers and letters, or the use of other names in this specification are not used to limit the order of the processes and methods of this specification. Although various examples are discussed in the above disclosure for some currently considered useful invention embodiments, it should be understood that such details only serve the purpose of illustration, and the appended claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that conform to the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only through software solutions, such as installing the described system on existing servers or mobile devices.

[0193] Similarly, it should be noted that, in order to simplify the presentation disclosed in this specification and thus assist in the understanding of one or more embodiments of the invention, in the foregoing description of the embodiments of this specification, multiple features are sometimes grouped into one embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the features required by the subject matter of this specification are more than those recited in the claims. In fact, the features of the embodiments are fewer than all the features of the individual embodiments disclosed above.

[0194] In some embodiments, numbers are used to describe components and attribute quantities. It should be understood that such numbers used to describe embodiments are, in some examples, modified by the modifiers "about", "approximate" or "substantially". Unless otherwise stated, "about", "approximate" or "substantially" indicate that the stated number allows a variation of ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, and such approximate values may vary according to the characteristics required by individual embodiments. In some embodiments, the numerical parameters should consider the specified significant digits and adopt the method of retaining the general number of digits. Although the numerical ranges and parameters used in some embodiments of this specification to confirm the breadth of their scope are approximate values, in specific embodiments, the setting of such numerical values is as precise as possible within the feasible range.

[0195] For each patent, patent application, patent application publication, and other materials cited in this specification, such as articles, books, specifications, publications, documents, etc., their entire contents are hereby incorporated into this specification by reference. Except for the application history documents that are inconsistent with or conflict with the content of this specification, and also except for the documents that limit the broadest scope of the claims of this specification (currently or subsequently appended to this specification). It should be noted that if there are inconsistencies or conflicts between the descriptions, definitions, and / or uses of terms in the supplementary materials of this specification and the content described in this specification, the descriptions, definitions, and / or uses of terms in this specification shall prevail.

[0196] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, by way of example and not limitation, alternative configurations of the embodiments of this specification may be considered to be consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A process management system, comprising: An acquisition module configured to obtain process information of an operator performing a target operation, the process information being configured to reflect the completion of the target operation by the operator; A guidance module including a prompting unit and a processing unit; the processing unit being configured to generate guidance information for guiding the operator to perform the target operation; the prompting unit being configured to control a display device to display the guidance information.

2. The system according to claim 1, wherein the guidance module further includes a receiving unit; The receiving unit is configured to receive the process information obtained by the acquisition module and / or input information input by the operator; the input information includes feedback information reflecting the completion of the target operation; The processing unit is further configured to generate guidance information based on the process information and / or the input information received by the receiving unit.

3. The system according to claim 2, wherein the target operation includes a plurality of target sub-operations; The processing unit is further configured to: Determine whether the current target sub-operation is completed based on the process information; In response to the completion of the current target sub-operation, generate guidance information for the next target sub-operation; In response to the current target sub-operation not being completed, generate reminder information to control the display device to display the reminder information based on the prompting unit, the reminder information including the guidance information of the uncompleted current target sub-operation.

4. The system according to claim 3, wherein the processing unit is further configured to: Execute a preset assistance operation in response to the prompt time interval satisfying a preset time condition; the prompt time interval is the time interval between the current time and the time when the prompting unit last issued the guidance information.

5. The system according to claim 2, wherein the target operation includes a plurality of target sub-operations; The processing unit is further configured to: Judge whether the completion of the target operation reflected by the input information and the process information is consistent; In response to the reflected completion being consistent, generate the guidance information for the next target sub-operation; In response to the reflected completion being inconsistent, generate reminder information to control the display device to display the reminder information based on the prompting unit, the reminder information including the guidance information of the current target sub-operation.

6. The system according to claim 1, wherein the target operation includes a plurality of target sub-operations; The guidance information includes first guidance information and / or second guidance information; The first guidance information prompts the next target sub-operation; the second guidance information prompts the operation information of the next target sub-operation.

7. The system according to claim 6, wherein the target sub-operation includes one or more operation points; The operation information includes at least one of the point information of the operation points included in the next target sub-operation, the operation sequence information of the operation points, the operation method information of the operation points, and the process standard information corresponding to the operation points.

8. The system according to claim 1, wherein the acquisition module comprises a sound acquisition unit, an image acquisition unit and an analysis unit; The analysis unit is configured to generate the process information based on the sound information collected by the sound acquisition unit and the image information collected by the image acquisition unit.

9. A process management device, comprising: An input device, a recording device and a processor; The input device is configured to collect the input information of the operator, and the input information includes at least one of the time information of the operator performing the target operation and the personal information of the operator; The recording device is configured to collect the process information of the operator performing the target operation; The processor is configured to: Obtain the process information; Generate guidance information for guiding the operator to perform the target operation based on the process information; Control the display device to display the guidance information.

Images