Process detection method, computing device, storage medium and program product

Abstract

The embodiment of the invention provides a process detection method, computing equipment, a storage medium and a program product. The method comprises the steps of identifying an operation action sequence generated when a target user executes a target process; determining a standard action sequence of the target process; comparing the operation action sequence with the standard action sequence to determine at least one abnormal action in the operation action sequence; and based on the at least one abnormal action, generating an abnormal detection result of the operation action sequence. According to the technical scheme provided by the embodiment of the invention, the process detection accuracy is improved.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of manufacturing technology, and in particular, to a process detection method, a computing device, a storage medium, and a program product. Background Art

[0002] In the manufacturing industry, a process refers to a comprehensive step of continuously performing production activities on production materials, and is the basic unit that makes up the production process. A process usually includes a series of process operations, and a process operation is a production action performed by an operator, aiming to achieve specific processing, assembly, inspection, or other production goals, etc.

[0003] Therefore, ensuring the accuracy of process completion is the key to ensuring production efficiency and quality, and process detection becomes crucial.

[0004] Currently, manual inspection methods are usually adopted, and specialized personnel manually observe and judge the production actions performed by a certain operator to determine whether the process performed by the operator is accurate, etc. However, this method is highly subjective and the detection accuracy is not high. Summary of the Invention

[0005] Multiple aspects of the present application provide a process detection method, a computing device, a storage medium, and a program product to solve the technical problem of low accuracy in process detection in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a process detection method, including:

[0007] Identifying an operation action sequence generated by a target user performing a target process;

[0008] Determining a standard action sequence of the target process;

[0009] Comparing the operation action sequence with the standard action sequence to determine at least one abnormal action in the operation action sequence;

[0010] Generating an abnormal detection result of the operation action sequence based on the at least one abnormal action.

[0011] In a second aspect, an embodiment of the present application provides a computing device, including a processing component and a storage component;

[0012] The storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement the process detection method as described in the first aspect above.

[0013] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processing component, the process detection method described in the first aspect above is implemented.

[0014] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program / instructions. When the computer program / instructions are executed by a processing component, the process detection method described in the first aspect above is implemented.

[0015] In an embodiment of the present application, by identifying the operation action sequence actually generated by a target user when performing a target process, and combining with a pre-configured standard action sequence, the operation action sequence is compared with the standard action sequence, so that at least one abnormal action generated by the target user when performing the target process can be determined. After that, based on the at least one abnormal action and the detection rules pre-configured for the target process, the abnormal detection result of the operation action sequence can be automatically generated. In an embodiment of the present application, by disassembling the process into a series of atomic-level actions and defining the standard action sequence corresponding to the process, the standardization and automation of process detection are realized, which greatly reduces the human bias in the traditional method, ensures the consistency and fairness of the detection result, improves the detection efficiency, and improves the detection accuracy.

[0016] These aspects or other aspects of the present application will be more clearly understood in the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0018] Figure 1 The flowchart of an embodiment of a process detection method provided by the present application is shown;

[0019] Figure 2 The schematic diagram of the scenario interaction in an actual application of an embodiment of the present application is shown;

[0020] Figure 3 The schematic structural diagram of an embodiment of a process detection device provided by the present application is shown;

[0021] Figure 4 The schematic structural diagram of an embodiment of a computing device provided by the present application is shown. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts fall within the scope of protection of this application.

[0023] It should be noted that in cases where the embodiments of this application involve user information, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data that have been authorized by the user or fully authorized by all parties. Additionally, the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or reject. Moreover, various models (including but not limited to language models or large models) involved in this application comply with relevant laws and standards.

[0024] In addition, it should be noted that in cases where the embodiments of this application involve user interaction operations or trigger operations, the user interaction operations or trigger operations involved in the embodiments of this application include but are not limited to: interaction operations in various ways such as touch operations, gesture operations, voice operations, head movement operations, and eye movement operations; among them, touch operations include but are not limited to: click operations, double-click operations, long-press operations, slide operations, pinch operations, or mouse hover operations, etc. Slide operations include but are not limited to: linear slides, curved slides, etc.

[0025] The technical solutions of the embodiments of this application can be applied to production and manufacturing scenarios, such as clothing manufacturing scenarios, automotive manufacturing scenarios, etc.

[0026] For ease of reference, some terms used in this application are defined as follows. It should be noted that the proposed terms and their respective definitions are not strictly limited to these definitions - the terms can be further defined by the use of the term in this application. The term "example" used herein means to be used as an example, instance, or illustration. Any aspect or design described as "exemplary" herein should not necessarily be construed as superior to other aspects or designs.

[0027] Process: The comprehensive steps of continuously carrying out production activities on production materials, which are the basic units constituting the production process. A process usually includes a series of process operations, and process operations are the production actions performed by operators, aiming to achieve specific processing, assembly, inspection, or other production goals, etc. For example, in the automotive manufacturing scenario, a typical process may include stamping (for making body panels), welding (welding the body frame together), painting (providing durability and aesthetics to the body), and finally assembly (installing the engine, tires, and other components).

[0028] Among them, the multiple production actions included in the process usually have a certain sequence.

[0029] Production action: The technical actions performed by production operators, such as feeding, reverse stitching, or shear alignment, etc., can be composed of a set of therbligs, such as picking up, stacking, pushing / pulling, etc.;

[0030] Production equipment: The equipment used by operators during the execution of production actions, such as machine tools, blast furnaces, etc.;

[0031] Production materials: Items related to product production, such as materials, raw materials, auxiliary supplies, etc.

[0032] In the manufacturing industry, the accuracy of process completion is the key to ensuring production efficiency and quality. By detecting a series of production actions of the process performed by operators, the production process can be optimized according to the detection results, improving work efficiency and ensuring product quality, etc. Therefore, process detection is crucial.

[0033] In the traditional method, manual detection methods are usually adopted, such as manual patrol inspection, manual spot inspection, etc. Special personnel, such as coaches familiar with or teaching the production actions involved in the process, can observe the production actions of production operators and conduct manual evaluations, etc. However, these process methods actually have high detection costs, low efficiency, are affected by subjective factors, and due to the differential judgment scales, the detection accuracy is not high and the versatility is poor.

[0034] To solve the above technical problems, the inventor thought that with the development of computer vision technology, vision recognition algorithms are becoming more and more mature. It is possible to automatically recognize the production actions of operators through vision recognition technology, and then compare the production actions with the standard actions in the process to see if action detection can be achieved. However, the inventor found that in this way of comparing single actions, if there are errors in the operator's own operations or misidentifications caused by defects in the vision recognition algorithm itself, the accuracy of the detection results will be affected. Based on this, the inventor conducted a series of studies and proposed the technical solution of this application. In the embodiments of this application, by recognizing the operation action sequence actually generated by the target user when performing the target process, and combining the pre-configured standard action sequence, the operation action sequence is compared with the standard action sequence, so as to determine at least one abnormal action generated by the target user when performing the target process. After that, based on at least one abnormal action, an abnormal detection result of the operation action sequence can be automatically generated. In the embodiments of this application, by disassembling the process into a series of atomic-level actions and defining the standard action sequence corresponding to the process, the standardization and automation of process detection are realized, which greatly reduces the human bias in the traditional method, ensures the consistency and fairness of the detection results, and improves the detection accuracy. In addition, the embodiments of this application recognize the operation action sequence corresponding to the target process, and through sequence comparison rather than single action comparison, so that in the case of misidentification, missed identification or the user's own operation errors, relatively reasonable detection can also be performed through sequence comparison, thereby enhancing the detection stability and reliability to a certain extent.

[0035] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0036] Figure 1 It is a flowchart of an embodiment of a process detection method provided by an embodiment of the present application. The technical solution of this embodiment can be executed by a server, and the method can include the following steps:

[0037] 101: Recognize the operation action sequence generated by the target user when performing the target process.

[0038] Among them, the target user may refer to any production operator, and the target process may refer to any process or a specific process with detection requirements.

[0039] The operation action sequence may include multiple operation production actions, where an operation production action refers to a production action actually performed by the target user during the actual production process. In practical applications, production operators may be responsible for continuously performing a production process on production materials, or may sequentially perform multiple production processes according to the process regulations.

[0040] Among them, a series of standard production actions corresponding to the target process can be predefined, and these standard production actions constitute the standard action sequence of the target process.

[0041] In practical applications, a specialized person can demonstrate each standard production action included in the target process and record it as coach video data. The target user can learn the multiple standard production actions included in the target process by watching the coach video data and perform actual operations accordingly. Among them, the multiple standard production actions included in the target process can have an execution order, and the target user needs to perform the production actions in the execution order to form the operation action sequence corresponding to the target process.

[0042] In one practical application, the technical solution of the embodiment of the present application can be applied to an actual production scenario, where the target user actually performs production operations. Through the technical solution of the embodiment of the present application, the actual production operations of the target user can be detected to determine whether the production actions of the target user need to be optimized, etc. Of course, in another practical application, the technical solution of the embodiment of the present application can also be applied to a production training scenario, where the target user performs production action exercises according to the demonstration actions of specialized personnel. By detecting their practice operations, it can be determined whether the target user has mastered the standard production actions of the target process, so as to decide whether they can perform actual production operations.

[0043] Among them, there are various implementation methods for identifying the operation action sequence generated by the target user when performing the target process:

[0044] As an optional implementation method, identifying the operation action sequence generated by the target user when performing the target process can be to obtain the operation action sequence recorded by a specialized person for the target user when performing the target process. That is, a specialized person can manually observe and record the operations of the target user and report the recording results to the server. The server performs detections, etc.

[0045] As another optional implementation method, in order to further improve the detection efficiency and detection accuracy, etc., the identification of the operation action sequence generated by the target user when performing the target process may include:

[0046] Obtain the operation video collected by the acquisition device; the operation video is obtained by the acquisition device collecting the production actions performed by the target user; identify the operation action sequence corresponding to the target process from the operation video.

[0047] In actual applications, each production operator may correspond to a collection device. When each production operator performs the production actions of the target process through the production device, each collection device may correspond to a production device.

[0048] The operation video of the target user can be collected through the collection device, so that the operation action sequence corresponding to the target process can be identified and obtained from the operation video.

[0049] Wherein, when the production device corresponds to a collection device, in order to distinguish different production operators, the method may further include: obtaining a binding request sent by the user terminal; the binding request includes a first device identifier corresponding to the collection device; establishing an association relationship between the first device identifier and the user identifier of the target user corresponding to the user terminal;

[0050] After obtaining the operation video collected by the collection device as described above, the method may further include: according to the first device identifier of the collection device, searching for the association relationship to determine the corresponding user identifier. Thus, the server can determine the target user corresponding to the operation video, and can identify from the operation the operation action sequence composed of a series of production actions performed by the target user for the target process.

[0051] In addition, the binding request may further include a second device identifier corresponding to the production device, so that an association relationship among the first device identifier, the second device identifier, and the user identifier of the target user can be established.

[0052] 102: Determine the standard action sequence of the target process.

[0053] As can be seen from the above description, the standard action sequence of the target process can be pre-configured, so the standard action sequence can be obtained from the configuration information. The standard action sequence includes multiple standard production actions corresponding to the target process, and the multiple standard production actions may have an execution order. Each standard production action may include reference data such as action elements and action positions.

[0054] The standard action sequence can be pre-configured by a professional, and in addition, it can also be obtained by identifying the coach video data corresponding to the target process. The coach identification data can be collected by collecting multiple standard actions performed by a professional for the target process, etc.

[0055] It should be noted that the production actions involved in the above operation action sequence and standard action sequence may exist in the form of data and are composed of reference data such as action elements and action positions. Action requirements may refer to picking up, stacking, pushing / pulling, etc.

[0056] 103: Compare the operation action sequence with the standard action sequence to determine at least one abnormal action in the operation action sequence.

[0057] By comparing the operation action sequence with the standard action sequence, at least one abnormal action in the operation action sequence can be determined. Of course, the determination result can be empty, that is, there is no abnormal action in the operation action sequence.

[0058] If there is no abnormal action in the operation action sequence, the process can be ended or the operation in step 101 can be continued to continue detecting the operation action sequence generated by the target user for the next execution of the target process. Of course, if there is no abnormal action in the operation action sequence, the normal detection result corresponding to the operation action sequence can also be directly generated, etc. The normal detection result can indicate that the operation action sequence is normal and the target user has no operation errors, etc.

[0059] Among them, the abnormal action can be, for example, a missing action or a redundant action, etc.

[0060] The missing action (Missing Actions) can be, for example, a production action that exists in the standard action sequence but does not appear in the operation action sequence. The missing action may affect the effect and quality of the target process. Therefore, the missing action can be detected to determine the accuracy of the target process executed by the target user.

[0061] The redundant action (Redundant Actions) can be, for example, a production action that appears in the operation action sequence but is not in the standard action sequence. The redundant action may increase the actual operation time of the target process, resulting in a decrease in production efficiency. Therefore, the redundant action can be detected to determine the accuracy of the target process executed by the target user.

[0062] Among them, there are various implementation methods for comparing the operation action sequence with the standard action sequence, such as finding the sequence subsets of the two sequences or finding the longest common subsequence of the two sequences, etc. The sequence subset is composed of the same elements in the two sequences. The longest common subsequence refers to the longest subsequence that appears in the same order between the two sequences, rather than a combination of elements in any order. For example, for the sequence "ABCDGH" and the sequence "AEDCHR", their longest common subsequence is "ADH", and the sequence subset is "ADCH".

[0063] 104: Generate an abnormal detection result of the operation action sequence based on at least one abnormal action.

[0064] The at least one abnormal action may include a missing action, a redundant action, or both a missing action and a redundant action. Based on the at least one abnormal action, an abnormal detection result of the operation action sequence can be generated, and the abnormal detection result can be used to indicate that the operation action sequence is abnormal and the at least one abnormal action, etc.

[0065] In addition, in order to further improve the detection accuracy, detection rules can be pre-configured for the target process, so that an abnormal detection result of the operation action sequence can be generated according to the at least one abnormal action and the detection rules.

[0066] The detection rules can, for example, define the abnormal type or abnormal score corresponding to a missing action or a redundant action, etc., so that the abnormal type of the operation action sequence or the process score, etc. can be determined accordingly.

[0067] In this embodiment, by disassembling the process into a series of atomic-level actions and defining the standard action sequence corresponding to the process, the standardization and automation of process detection are realized, which greatly reduces the human bias in the traditional method, ensures the consistency and fairness of the detection results, and improves the detection accuracy. In addition, the embodiments of the present application identify the operation action sequence corresponding to the target process, and through sequence comparison rather than single-action comparison, so that in the case of mis-identification, missed-identification, or user's own operation error, relatively reasonable detection can also be performed through sequence comparison, thereby enhancing the detection stability and reliability to a certain extent.

[0068] Combined with the foregoing description, based on the at least one abnormal action and the detection rules, an evaluation score of the operation action sequence can be calculated, so that the accuracy of the operation action sequence can be evaluated through the evaluation score to determine whether the target user executes the target process accurately, etc. In some embodiments, the above-mentioned generating an abnormal detection result of the operation action sequence based on the at least one abnormal action may include:

[0069] Determine the total evaluation score defined in the detection rules of the target process; determine the missing scores corresponding to the at least one abnormal action according to the scoring method corresponding to the at least one abnormal action defined in the detection rules; deduct the missing scores corresponding to the at least one abnormal action from the total evaluation score to determine the evaluation score corresponding to the operation action sequence.

[0070] The scoring method can stipulate the scoring methods corresponding to the abnormal actions of different abnormal types. For example, the scoring method can specify the deduction scores corresponding to a single abnormal action of each abnormal type, and then combine the number of abnormal actions corresponding to each abnormal type to determine the missing scores corresponding to the at least one abnormal action, etc.

[0071] In addition, since the abnormal action can include multiple abnormal types, the calculation of the evaluation score can also be obtained according to any of the following implementation methods:

[0072] As an alternative, comparing the above operation action sequence with the standard action sequence to determine at least one abnormal action in the operation action sequence may include:

[0073] Calculating the longest common subsequence of the operation action sequence and the standard action sequence;

[0074] Comparing the standard action sequence with the longest common subsequence, and determining at least one standard production action not included in the longest common subsequence in the standard action sequence as at least one missing action of the operation action sequence.

[0075] Then, generating an abnormal detection result of the operation action sequence based on at least one abnormal action may include:

[0076] Determining the total evaluation score defined in the detection rules of the target process; determining the missing scores corresponding to at least one missing action according to the scoring method corresponding to at least one missing action defined in the detection rules; deducting the missing scores corresponding to at least one missing action from the total evaluation score to determine the evaluation score corresponding to the operation action sequence.

[0077] As another alternative, comparing the above operation action sequence with the standard action sequence to determine at least one abnormal action in the operation action sequence may include:

[0078] Calculating the longest common subsequence of the operation action sequence and the standard action sequence; comparing the operation action sequence with the longest common subsequence, and determining at least one operation production action not included in the longest common subsequence in the operation action sequence as at least one redundant action of the operation action sequence;

[0079] Then, generating an abnormal detection result of the operation action sequence based on at least one abnormal action and the detection rules of the target process may include:

[0080] Determining the total evaluation score defined in the detection rules of the target process; determining the redundant scores corresponding to at least one redundant action according to the scoring method corresponding to at least one redundant action defined in the detection rules; deducting the redundant scores of at least one redundant action from the total evaluation score to determine the evaluation score corresponding to the operation action sequence.

[0081] As yet another alternative, comparing the above operation action sequence with the standard action sequence to determine at least one abnormal action in the operation action sequence may include:

[0082] Calculate the longest common subsequence of the operation action sequence and the standard action sequence; compare the operation action sequence with the longest common subsequence, and determine at least one operation production action in the operation action sequence that is not included in the longest common subsequence as at least one redundant action of the operation action sequence, and determine at least one standard production action in the standard action sequence that is not included in the longest common subsequence as at least one missing action of the operation action sequence.

[0083] The above abnormal detection result of the operation action sequence generated based on at least one abnormal action and the detection rule of the target process may include:

[0084] Determine the total evaluation score defined in the detection rule of the target process; determine the redundant score corresponding to at least one redundant action according to the scoring method corresponding to at least one redundant action defined in the detection rule; determine the missing score corresponding to at least one missing action according to the scoring method corresponding to at least one missing action defined in the detection rule; deduct the redundant score corresponding to at least one redundant action and the redundant score corresponding to at least one missing action from the total evaluation score to determine the evaluation score corresponding to the operation action sequence.

[0085] Among the above three optional methods, it is possible to detect missing actions, redundant actions, or both missing and redundant actions respectively, and deduct the corresponding scores from the total evaluation score according to the scoring methods corresponding to the abnormal actions of different abnormal types to obtain the evaluation score of the operation action sequence. By quantitatively analyzing redundant actions, the efficiency loss caused by unnecessary operations can be reduced. If there are no abnormal actions in the operation action sequence, according to the total evaluation score defined in the detection rule, it can be determined that the average score of the operation action sequence is the average total score.

[0086] Therefore, it is possible to determine whether there are operation errors and whether action adjustment is required based on the evaluation score, which not only helps to improve work efficiency but also guides users to optimize the operation process. In practical applications, the total evaluation score can be adjusted by combining the redundant score and the missing score to obtain the final evaluation score. Through comprehensive processing, the importance of key steps can be considered and the overall operation efficiency can be reflected, further improving the detection accuracy.

[0087] Among them, the missing action can be found by finding the longest common subsequence, and at least one standard production action in the standard action sequence that is not included in the long common subsequence is used as at least one missing action.

[0088] Among them, the redundant action can be found by finding the longest common subsequence, and at least one operation production action in the operation action sequence that is not included in the long common subsequence is used as at least one redundant operation.

[0089] For example, the standard action sequence includes "ABCD", four standard production actions arranged in the execution order, while the operation action sequence includes "BCED", and the longest common subsequence is "BCD". The standard production action in "ABCD" that is not included in "BCD" is A, so the missing action is A. The operation production action in "BCED" that is not included in "BCD" is E, so the redundant action is E.

[0090] By using the longest common subsequence to compare the standard action list and the operation action list, not only the existence of production actions is considered, but also the timing relationship between production actions is taken into account, which can more accurately capture the logical order in the execution of the target process sequence and ensure that the effect of the target process meets the expectations.

[0091] In one possible implementation, the scoring method corresponding to at least one missing action defined in the detection rule may include, for example, a deduction ratio or deduction score corresponding to a missing action. Thus, the deduction score can be calculated based on the deduction ratio, and then the deduction scores corresponding to different missing actions are added together to obtain the missing score corresponding to at least one missing action.

[0092] The scoring method corresponding to at least one redundant action defined in the detection rule may include, for example, a deduction ratio or deduction score corresponding to a redundant action. Thus, the deduction score can be calculated based on the deduction ratio, and then the deduction scores corresponding to different redundant actions are added together to obtain the redundant score corresponding to at least one redundant action.

[0093] Among them, the deduction ratios or deduction scores corresponding to the missing actions and the redundant actions may be the same or different, etc.

[0094] Among them, different missing actions may correspond to different or the same deduction ratios or deduction scores, and different redundant actions may correspond to different or the same deduction ratios or deduction scores, etc. The present application does not limit this.

[0095] In addition, since different production actions have different levels of difficulty, operation durations, and / or importance in the target process, in order to further improve the detection accuracy, the weight value of each standard production action can be predefined in advance, and the weight value is used to represent the importance or difficulty of the standard production action, etc. In another possible implementation, for the missing actions, the determination of the missing score corresponding to at least one missing action according to the scoring method corresponding to at least one missing action defined in the detection rule may include:

[0096] Determine the weight values corresponding to at least one missing action defined in the detection rule; determine the total weight value corresponding to each standard production action in the standard action sequence; determine the missing score corresponding to the missing action according to the first proportion of the weight values of at least one missing action in the total weight value.

[0097] For example, the standard action sequence includes four standard production operations A, B, C, and D, and the corresponding weight values are 10, 30, 40, and 20 respectively. The total weight value is 10 + 30 + 40 + 20 = 100. Suppose at least one missing action includes A and D, then the first proportion is (10 + 20) / 100 = 0.3.

[0098] Among them, the missing scores corresponding to different first proportions can be set in advance, or the missing score can be calculated by multiplying the total evaluation score by the first proportion. Of course, the first proportion can also be directly used as the missing score, and this application does not limit this.

[0099] Through the weight values, the importance of key production actions can be reflected, making the detection results more accurate and reasonable.

[0100] In order to simplify the calculation method, in another possible implementation, the missing score can also be determined according to the number of missing actions of at least one missing action; the redundancy score can be determined according to the number of redundant actions of at least one redundant action.

[0101] Therefore, in some embodiments, the scoring method corresponding to at least one redundant action defined in the above detection rule, determining the redundancy score corresponding to at least one redundant action may include:

[0102] According to the scoring method corresponding to at least one redundant action defined in the detection rule, determine the number of production actions in the standard action sequence and the number of redundant actions of at least one redundant action, and determine the total number of production actions;

[0103] Determine the redundancy score corresponding to at least one redundant action according to the second proportion of the number of redundant actions in the total number of production actions.

[0104] The number of production actions is the total number of standard production actions. For example, the number of production actions is 8 and the number of redundant actions is 2, then the second proportion is: 2 / 10 = 0.2.

[0105] Among them, the redundancy scores corresponding to different second proportions can be set in advance, or the redundancy score can be calculated by multiplying the total evaluation score by the second proportion. Of course, the second proportion can also be directly used as the redundancy score, and this application does not limit this.

[0106] In some embodiments, determining the missing scores corresponding to at least one missing action according to the scoring method defined in the detection rule may include:

[0107] Determine the number of production actions in the standard action sequence and the number of missing actions corresponding to at least one missing action according to the scoring method defined in the detection rule for at least one missing action; determine the missing scores corresponding to at least one missing action according to the third proportion of the number of missing actions in the number of production actions.

[0108] The number of production actions is the total number of standard production actions. For example, if the number of production actions is 8 and the number of missing actions is 2, then the third proportion is: 2 / 8 = 0.25.

[0109] Among them, the missing scores corresponding to different third proportions can be set in advance, or the missing scores can be calculated by multiplying the total evaluation score by the third proportion. Of course, the third proportion can also be directly used as the missing score, and the present application does not limit this.

[0110] In addition, in some embodiments, to ensure the rationality of scoring, determining the redundancy scores corresponding to at least one redundant action according to the scoring method defined in the detection rule may include:

[0111] Determine the redundancy scores corresponding to at least one redundant action according to the scoring method defined in the detection rule for at least one redundant action;

[0112] If the redundancy score exceeds the predetermined score, the redundancy score is updated with the predetermined score.

[0113] That is, there is a certain upper limit constraint on the redundancy score to ensure that the scoring is not too strict and can also reflect the importance of efficiency.

[0114] The predetermined score can be set in combination with the actual situation.

[0115] In addition, the second proportion of at least one redundant action can be determined in the above implementation manner. Optionally, determining the redundancy scores corresponding to at least one redundant action according to the second proportion of the number of redundant actions in the total number of production actions may include: determining the second proportion of the number of redundant actions in the total number of production actions; judging whether the second proportion exceeds the predetermined proportion; if not, determining the redundancy scores corresponding to at least one redundant action according to the second proportion; if so, determining the redundancy scores corresponding to at least one redundant action according to the predetermined proportion.

[0116] The predetermined ratio can be, for example, 20% or the like. By restricting with this predetermined ratio, it is also possible to ensure that there is a certain finite restriction on the redundancy score, ensuring that the scoring is not overly harsh while also reflecting the importance of efficiency. The redundancy score corresponding to the predetermined ratio can be set in advance, or the redundancy score can be calculated by multiplying the total evaluation score by the predetermined ratio. Of course, the predetermined ratio can also be directly used as the redundancy score, and the present application does not limit this.

[0117] Combined with the description above, it can be known that the operation action sequence corresponding to the target process can be recognized from the operation video. In some embodiments, recognizing the operation action sequence corresponding to the target process from the operation video can be implemented as:

[0118] Performing an identification operation of production actions from the operation video; according to the identification result, determining the operation action sequence corresponding to the target process.

[0119] Among them, the identification operation of production actions can be implemented by using a visual recognition algorithm, etc. In addition, an action recognition model can be used for identification. In some embodiments, performing the identification operation of production actions from the operation video can include:

[0120] Using the action recognition model to recognize production actions from the operation recognition;

[0121] Among them, the action recognition model can be obtained by pre-training based on a sample video including sample production actions and the action types of the sample production actions.

[0122] The action recognition model can be implemented as various machine learning models, such as neural network models, deep learning models, and of course, it can also be an artificial intelligence large model, etc. Among them, the artificial intelligence large model refers to a "large parameter" model trained using large-scale data and powerful computing capabilities. These models usually have high generality and generalization capabilities and can be applied to fields such as natural language processing, image recognition, and speech recognition. The present application does not specifically limit the model type of the action recognition model.

[0123] Using the action recognition model can recognize production actions and can determine the action type.

[0124] Among them, there are various implementation manners for determining the operation action sequence corresponding to the target process according to the above recognition result. As an optional manner, determining the operation action sequence corresponding to the target process according to the above recognition result can include:

[0125] According to the recognition result, determining the termination production action of the target process; from the termination production action and at least one production action between the termination production action and the previous termination production action, constituting the operation action sequence of the target process.

[0126] The recognition result may include different production actions and their corresponding action types. In addition, it may also include the production time corresponding to different production operations, and this production time may refer to the start time of each production action, etc.

[0127] The termination production action can be determined from the recognition result according to the action type of the termination production action. Thus, the operation action sequence of the target process can be formed by the termination production action and at least one production action between the termination production action and the previous termination production action.

[0128] In addition, since a certain production action in a process may occur multiple times in different execution orders, in order to improve the recognition accuracy, a unique termination production action can be specified in advance in the standard action sequence. This termination production operation may not participate in the operation of the production materials and only serves as an identification - type marking production operation, so that the termination production action of each process can be determined.

[0129] As another alternative, the above - mentioned determination of the operation action sequence corresponding to the target process according to the recognition result may include:

[0130] According to the recognition result, determine the starting production action and the termination production action of the target process; form the operation action sequence of the target process by the starting production action, the termination production action, and at least one production action between the termination production actions.

[0131] That is, according to the action type of the termination production action and the action type of the starting production action, the starting production action and the termination production action can be determined. Thus, the operation action sequence of the target process can be formed by the starting production action, the termination production action, and at least one production action between the termination production actions.

[0132] Among them, the production time of different production actions can be determined and time - aligned with the equipment operation signal of the production equipment. Thus, the production start time and the production end time of each process can be determined, and then the current starting production action and the termination production action can be determined by combining the production start time and the production end time.

[0133] Among them, the device operation data can refer to data such as temperature, pressure, rotational speed, load, current, etc. generated during the operation of production equipment in the production process. In the actual production process, when different production processes are executed, the device operation data of the production equipment is usually different. Taking the two different production processes of sewing buttons and sewing sleeves as an example, when the production process of sewing buttons is executed, the working frequency, movement trajectory, etc. of the sewing machine needle are different from those when the production process of sewing sleeves is executed, and the corresponding device operation data is also different. In addition, when the same production process is executed cyclically, the device operation data of the production equipment usually shows periodic fluctuation characteristics. For example, when the production process of sewing buttons is executed cyclically, when sewing each button, the working frequency, movement trajectory, etc. of the sewing machine needle are usually the same, and the corresponding device operation data is also the same, and is different from the device operation data corresponding to the sewing interval between two buttons, thus showing periodic fluctuation characteristics. Therefore, whether different production processes are executed or the same production process is executed cyclically, the start and end production times corresponding to the target process can be determined according to the device operation data of the production equipment.

[0134] In addition, as another alternative, the above-mentioned determining the operation action sequence corresponding to the target process according to the recognition result may include:

[0135] Obtain a production termination signal generated by the production equipment; the production termination signal is generated in response to a termination operation triggered by a target user's termination production action at the end of the target process; determine the production termination time corresponding to the production termination signal; according to the recognition result, determine the termination production action corresponding to the production termination time; and form the operation action sequence of the target process from the termination production action and at least one production action between the termination production action and the previous termination production action.

[0136] For example, the production equipment can be deployed with specific operation components, such as specific buttons, etc. to trigger the production termination signal. When the target user executes the target process cyclically, each time the target process ends, the operation component can be triggered, so that the production equipment can sense the trigger operation of the target user to generate the production termination signal and report the production termination signal to the server. The server can determine the production termination time according to the timestamp information of the production termination signal.

[0137] Among them, the recognition result may include multiple production actions and the production times of different production actions, so that according to the production termination time, the corresponding termination production action can be determined from the recognition result. Furthermore, the operation action sequence of the target process can be formed from the termination production action and at least one production action between the termination production action and the previous termination production action.

[0138] As can be seen from the above implementation, since the acquisition device can continuously acquire the production actions performed by the target user to obtain a real-time operation video, the recognition operation of the operation video can be performed in real time to recognize each production action, and then determine the operation action sequence composed of the production actions. In addition, the operation video can also be segmented first to obtain the video segment corresponding to each target process, and then the production action recognition is performed on the video segment, and the multiple production actions obtained by recognition constitute the operation action sequence. Therefore, in some embodiments, recognizing the operation action sequence corresponding to the target process from the operation video includes:

[0139] Segmenting the video segment corresponding to the target process from the operation video; recognizing the operation action sequence corresponding to the target process from the video segment.

[0140] In an alternative way, segmenting the video segment corresponding to the target process from the operation video can be implemented as: obtaining the device operation data of the production device; determining the production start time and production end time corresponding to the target process according to the device operation data corresponding to the target process, and segmenting the video segment corresponding to the target process from the operation video according to the production start time and production end time.

[0141] As can be seen from the above description, the production start time and production end time corresponding to the target process can be determined according to the device operation data. Thus, the video segment corresponding to the target process can be segmented from the operation video in combination with the production start and end times.

[0142] In addition, in another alternative way, the start production action and end production action corresponding to the target process can also be recognized from the operation video; and segmenting the video segment corresponding to the target production process from the operation video according to the start production action and end production action.

[0143] In addition, the target user may perform the target process multiple times during the production process, and multiple operation action sequences are obtained. Each operation action sequence can be detected to obtain a detection result (including a normal detection result or an abnormal detection result), and the detection results of multiple operation action sequences can be combined to obtain the detection result corresponding to the target process.

[0144] Among them, based on the detection result of the operation action sequence and / or the detection result of the target process, corresponding detection prompt information can be generated, and thus the detection prompt information can be sent to the target user so that the target user can understand his own process operation and make improvements, etc. Of course, the detection prompt information can also be sent to relevant personnel, and the relevant personnel can evaluate the work of the target user in combination with the detection prompt information to make rewards and punishments, etc.

[0145] Therefore, in some embodiments, the method may further include:

[0146] Determine the detection results corresponding to multiple operation action sequences of the target user within a predetermined time range; generate detection prompt information for the target user based on the detection results corresponding to the multiple operation sequences; and notify the target user or relevant personnel of the detection prompt information.

[0147] The multiple operation action sequences are generated by the target user executing the target process multiple times within a predetermined time range.

[0148] The predetermined time range can be, for example, one day or one week, etc., and can be set according to the actual situation.

[0149] Among them, the detection prompt information may include the detection results corresponding to the multiple operation sequences. The detection results corresponding to the multiple operation action sequences may include normal detection results and abnormal detection results. For the normal detection results, the evaluation score corresponding to the operation action sequence is also the total evaluation score.

[0150] In addition, when the detection result is an evaluation score, the detection prompt information may include the evaluation score of the target process. The evaluation score may be the average value or sum value of the evaluation scores of the multiple operation action sequences. In addition, the detection prompt information may also include the evaluation score of each operation action sequence to facilitate the target user or relevant personnel to understand the comprehensive evaluation situation of the target user and the evaluation situation at different stages, etc.

[0151] In addition, the detection prompt information can also be generated in combination with at least one abnormal action in each operation action sequence. The detection prompt information may also include at least one abnormal action in each operation action sequence. Or, the abnormal times of different abnormal actions can be determined according to at least one abnormal action in each operation action sequence, and the detection prompt information may also include the abnormal times, so as to facilitate the target user or relevant personnel to focus on which part of the production operation of the target user should be improved, etc. Among them, there are various implementation manners for notifying the target user or relevant personnel of the detection prompt information. For example, it can be sent to the user terminal of the target user or relevant personnel to output the detection prompt information in the user terminal, and the detection prompt information can be sent to the corresponding user terminal based on the user identifier; or the detection prompt information can be sent based on the communication account of the target user or relevant personnel. The communication account can be, for example, an email account, a mobile communication account, or an instant messaging account, etc. The present application does not specifically limit the notification method.

[0152] For the convenience of understanding the technical solution of the present application, Figure 2 The schematic diagram of the scenario interaction in an actual application of the embodiment of the present application is shown. Figure 2 The shown system architecture may include a server 201, a user terminal 202, a collection device 203, and a production device 204.

[0153] Among them, the client 202 can be implemented as a browser, an APP (Application), or a web application such as an H5 (HyperText Markup Language 5) application, or a light application (also known as a mini-program, a lightweight application), or a cloud application, etc. The client 202 can be deployed in an electronic device and needs to rely on the device or certain apps in the device to run, etc. The electronic device can, for example, have a display screen and support information browsing, etc., such as a personal mobile terminal such as a mobile phone, a tablet computer, a personal computer, a desktop computer, a smart speaker, a smart watch, etc. Various other types of applications can usually be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc. The electronic device can refer to a device used by a user and having functions such as computing, Internet access, and communication required by the user, such as a mobile phone, a tablet computer, a personal computer, a wearable device, etc. The electronic device usually can include at least one processing component and at least one storage component. The electronic device may also include basic configurations such as a network card chip, an IO (input / output) bus, and audio-video components, which are not limited in this application. Optionally, according to the implementation form of the electronic device, some peripheral devices may also be included, such as a keyboard, a mouse, an input pen, a printer, etc., which are not limited in this application.

[0154] The server 201 can include servers providing various services, such as a server for background training that provides support for the models used on the client 202, or a service that processes the information sent by the client and / or the acquisition device, etc.

[0155] It should be noted that the server 201 can be implemented as a distributed server cluster composed of multiple servers, or can be implemented as a single server. The server can also be a server of a distributed system, or a server combined with a blockchain. The server can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms, or an intelligent cloud computing server or an intelligent cloud host with artificial intelligence technology.

[0156] It should be noted that Figure 2The presented server 201, client 202, acquisition device 203, and production device 204 are only exemplary descriptions and do not limit their implementation forms. For the convenience of viewing, the illustration of the client is represented by the image of a device, such as the image of a mobile phone in the figure, and the illustration of the acquisition device is also represented by the image of a device, such as the image of a camera in the figure, and the production device is like the image of a sewing machine in the figure. It can be understood that the numbers of the acquisition device 203 and the production device 204 are only illustrative. In actual applications, corresponding acquisition devices 203 and production devices 204 can be configured for each workbench in the production workshop.

[0157] During the actual production process, the target user enters the workbench and starts the production device 204 to perform production actions on the production materials and / or the production device.

[0158] When the acquisition device 203 senses the start of the production device 204 or detects the entry of the target user into the workbench, it can start acquiring the operation video of the target user performing production actions on the production materials and / or the production device, and can upload it to the server 201 in real time.

[0159] Optionally, the client 202 can respond to the scanning operation triggered by the target user for the medium carrying the first device identifier in the acquisition device 203 to obtain the first device identifier of the acquisition device 203, and respond to the scanning operation triggered by the target user for the medium carrying the second device identifier in the production device 204 to obtain the second device identifier of the production device 204, and send a binding request to the server 201. The server 201 can establish an association relationship between the first device identifier, the second device identifier, and the user identifier of the target user corresponding to the client 202.

[0160] The server 201 can pre - establish the standard action sequence of the target process (step 31).

[0161] The server 201 can perform the recognition operation of the production actions from the operation video, and can determine the operation action sequence of the target process according to the recognition result (step 32). The specific implementation method can be seen in the foregoing, and will not be repeated here.

[0162] After that, the server 201 can determine the missing actions and redundant actions in the operation action sequence by comparing the standard action sequence and the operation action sequence (step 33), and then perform scoring in combination with the detection rules (step 34). For example, if the total evaluation score is 100 points, the missing score can be deducted according to the first proportion of the weight value corresponding to the missing action in the total weight value, and then according to the second proportion of the number of redundant actions in the total number of production actions, an additional redundant score not exceeding a predetermined value can be deducted to obtain the evaluation score.

[0163] The server 201 can generate detection prompt information based on the evaluation scores corresponding to multiple operation sequences of the target process within a predetermined time range, and can send the detection prompt information to the client 202. The client 202 can display the detection prompt information in the user interface to prompt the target user whether to make action adjustments, so as to optimize the production process, improve work efficiency, and ensure product quality.

[0164] The technical solution of the embodiment of the present application can achieve standardized and objective evaluation. By defining standard production actions and their weight values, and combining visual recognition or manual disassembly methods, the target process can be disassembled into a series of atomic-level actions, realizing the standardization and automation of process scoring, which greatly reduces the human bias in traditional methods and ensures the consistency and fairness of the evaluation results.

[0165] In addition, the consideration of temporal dependence is realized. The longest common subsequence can be introduced to compare the standard action list with the actual operation action list, not only considering the existence of actions, but also paying attention to the temporal relationship between actions. This can more accurately capture the logical order in the process execution and ensure that the process effect meets the expectations.

[0166] In addition, the identification of redundant actions and efficiency evaluation are realized: through quantitative analysis of redundant actions, a reasonable deduction mechanism is proposed to reduce the efficiency loss caused by unnecessary operations. This method not only helps to improve work efficiency, but also guides users to optimize the operation process.

[0167] In addition, the robust handling of misidentification and missed identification is realized: considering that the visual recognition method may have misidentification or missed identification, with the help of the longest common subsequence, it can effectively accommodate the mistakes and omissions of the actual operation action sequence, ensuring the stability and reliability of the scoring system.

[0168] Finally, a comprehensive scoring mechanism can be adopted. The basic score is calculated according to the weight values of the missing actions, and the final score is further adjusted in combination with the proportion of redundant actions, which can not only reflect the importance of key steps, but also reflect the overall operation efficiency.

[0169] It should be noted that in some of the processes described in the above embodiments and the accompanying drawings, a plurality of operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order in which they appear herein or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions such as "first" and "second" in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit that "first" and "second" are of different types.

[0170] Figure 3 FIG. 4 is a schematic structural diagram of an embodiment of a process detection device provided by an embodiment of the present application. The device may include:

[0171] A first recognition module 301, configured to recognize an operation action sequence generated by a target user performing a target process;

[0172] A determination module 302, configured to determine a standard action sequence of the target process;

[0173] A second recognition module 302, configured to compare the operation action sequence with the standard action sequence to determine at least one abnormal action in the operation action sequence;

[0174] A detection module 303, configured to generate an abnormal detection result of the operation action sequence based on at least one abnormal action.

[0175] In some embodiments, the second recognition module may specifically be configured to: calculate the longest common subsequence of the operation action sequence and the standard action sequence; compare the standard action sequence with the longest common subsequence, and determine at least one standard production action in the standard action sequence that is not included in the longest common subsequence as at least one missing action of the operation action sequence;

[0176] The detection module may specifically be configured to: determine an evaluation total score defined in the detection rule of the target process; determine a missing score corresponding to at least one missing action according to the scoring method corresponding to at least one missing action defined in the detection rule; deduct the missing score corresponding to at least one missing action from the evaluation total score to determine the evaluation score corresponding to the operation action sequence.

[0177] In some embodiments, the first recognition module may specifically be configured to calculate the longest common subsequence of the operation action sequence and the standard action sequence; compare the operation action sequence with the longest common subsequence, and determine at least one operation production action in the operation action sequence that is not included in the longest common subsequence as at least one redundant action of the operation action sequence;

[0178] The detection module can be specifically configured to determine the total evaluation score defined in the detection rules for the target process; determine the redundancy scores corresponding to at least one redundant action according to the scoring method defined for the at least one redundant action in the detection rules; and deduct the redundancy scores corresponding to the at least one redundant action from the total evaluation score to determine the evaluation score corresponding to the operation action sequence.

[0179] In some embodiments, the detection module determining the missing scores corresponding to at least one missing action according to the scoring method defined for the at least one missing action in the detection rules may include:

[0180] Determine the respective weight values corresponding to at least one missing action defined in the detection rules; determine the total weight value corresponding to each standard production action in the standard action sequence; and determine the missing scores corresponding to the missing actions according to the first proportion of the weight values of the at least one missing action in the total weight value.

[0181] In some embodiments, the detection module determining the redundancy scores corresponding to at least one redundant action according to the scoring method defined for the at least one redundant action in the detection rules may include:

[0182] According to the scoring method defined for the at least one redundant action in the detection rules, determine the number of production actions in the standard action sequence and the number of redundant actions of the at least one redundant action, and determine the total number of production actions; and determine the redundancy scores corresponding to the at least one redundant action according to the second proportion of the number of redundant actions in the total number of production actions.

[0183] In some embodiments, the detection module determining the redundancy scores corresponding to at least one redundant action according to the second proportion of the number of redundant actions in the total number of production actions may include:

[0184] Determine the second proportion of the number of redundant actions in the total number of production actions; determine whether the second proportion exceeds a predetermined proportion; if not, determine the redundancy scores corresponding to the at least one redundant action according to the second proportion; if so, determine the redundancy scores corresponding to the at least one redundant action according to the predetermined proportion.

[0185] In some embodiments, the first recognition module can be specifically configured to obtain the operation video collected by the collection device; the operation video is obtained by the collection device collecting the production actions performed by the target user; and recognize the operation action sequence corresponding to the target process from the operation video.

[0186] In some embodiments, the first recognition module recognizing the operation action sequence corresponding to the target process from the operation video may include: performing recognition operations on the production actions from the operation video; and determining the operation action sequence corresponding to the target process according to the recognition result.

[0187] In some embodiments, the first recognition module may determine the operation action sequence corresponding to the target process according to the recognition result, which may include: determining the termination production action of the target process according to the recognition result; and forming the operation action sequence of the target process by the termination production action and at least one production action between the termination production action and the previous termination production action.

[0188] In some embodiments, the first recognition module may determine the operation action sequence corresponding to the target process according to the recognition result, which may include:

[0189] Obtaining a production termination signal generated by the production equipment; the production termination signal is generated in response to a termination operation triggered by the target user during the termination production action of the target process; determining the production termination moment corresponding to the production termination signal;

[0190] Determining the termination production action corresponding to the production termination moment according to the recognition result; and forming the operation action sequence of the target process by the termination production action and at least one production action between the termination production action and the previous termination production action.

[0191] In some embodiments, the operation of the first recognition module to recognize the production action from the operation video may include:

[0192] Recognizing the production action from the operation recognition by using an action recognition model; wherein, the action recognition model is pre-trained based on a sample video including sample production actions and the action types of the sample production actions.

[0193] In some embodiments, the operation of the first recognition module to recognize the operation action sequence corresponding to the target process from the operation video may include: segmenting the video segment corresponding to the target process from the operation video; and recognizing the operation action sequence corresponding to the target process from the video segment.

[0194] In some embodiments, the detection module is further configured to generate a normal detection result when it is determined that there is no abnormal action stored in the operation action sequence.

[0195] In some embodiments, the detection module is further configured to determine the detection results corresponding to multiple operation action sequences of the target user within a predetermined time range; and generate detection prompt information for the target user based on the detection results corresponding to the multiple operation sequences.

[0196] The device may further include:

[0197] A prompt module, configured to notify the target user or relevant personnel of the detection prompt information.

[0198] Figure 3 The described process detection device may execute Figure 1For the process detection method described in the illustrated embodiment, its implementation principle and technical effects will not be elaborated further. For the process detection device in the above embodiment, the specific manners in which each module and unit perform operations have been described in detail in the embodiment related to the method, and will not be elaborated herein.

[0199] Figure 4 FIG. is a schematic structural diagram of a computing device provided by an embodiment of the present application. As Figure 4 shown, in practice, the computing device may include: a storage component 401 and a processing component 402.

[0200] The storage component 401 is used to store computer programs and can be configured to store various other data to support operations on the computing platform. Examples of such data include instructions for any application program or method for operating on the computing platform, data structures, contact data, phone book data, messages, pictures, videos, etc.

[0201] The processing component 402 is coupled to the storage component 401 and is used to execute one or more computer instructions in the storage component to implement the Figure 1 process detection method as shown.

[0202] Furthermore, as Figure 4 shown, the computing device may further include: other components such as a communication component 403, a display component 404, a power supply component 405, an audio component 406, etc. Figure 4 Only some components are schematically shown in, and it does not mean that the computing device only includes the Figure 4 components shown. Additionally, Figure 4 the components within the dashed box in are optional components, rather than mandatory components, and can be determined according to the product form of the working node specifically. The working node of this embodiment can be implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, or an IOT device, or can also be a server device such as a conventional server, a cloud server, or a server array. If the working node of this embodiment is implemented as a terminal device such as a desktop computer, a laptop computer, or a smart phone, it may include the Figure 4 components within the dashed box in; if the working node of this embodiment is implemented as a server device such as a conventional server, a cloud server, or a server array, it may not include the Figure 4 components within the dashed box in.

[0203] The above-mentioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0204] The above-mentioned communication component is configured to facilitate communication, either wired or wireless, between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel.

[0205] The above-mentioned display includes a screen, and the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operations.

[0206] The above-mentioned power supply component provides power to various components of the device where the power supply component is located. The power supply component can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device where the power supply component is located.

[0207] The above-mentioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive external audio signals. The received audio signals can be further stored in the memory or transmitted via the communication component. In some embodiments, the audio component further includes a speaker for outputting audio signals.

[0208] The embodiments of the present application also provide a computer-readable storage medium storing a computer program, which can implement the process detection method of the above-mentioned Figure 1 embodiment shown. The computer-readable medium may be included in the electronic device described in the above embodiment; or it may exist alone without being assembled into the electronic device.

[0209] The embodiments of the present application also provide a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program can implement the process detection method of the above-mentioned Figure 1 embodiment shown when executed by a computer. In such an embodiment, the computer program can be downloaded and installed from the network and / or installed from a removable medium. When the computer program is executed by the processing component, it executes various functions defined in the system of the present application.

[0210] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described system, device, and unit can refer to the corresponding processes in the foregoing method embodiments, and will not be described herein again.

[0211] It should also be noted that the term "including", "comprising", or any other variant thereof is intended to cover a non-exclusive inclusion, so that a process, method, commodity, or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, commodity, or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, commodity, or device including the element.

[0212] Finally, it should be noted that the above are only embodiments of the present application and are not used to limit the present application. For those skilled in the art, the present application may have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A process detection method, characterized in that: include: Identify the operation action sequence generated by the target user executing the target process; Determine a standard action sequence of the target process; Comparing the operation action sequence with the standard action sequence to determine at least one abnormal action in the operation action sequence; Based on the at least one abnormal action, an abnormality detection result of the operation action sequence is generated.

2. The method according to claim 1, characterized in that The comparing the operation action sequence with the standard action sequence to determine at least one abnormal action in the operation action sequence comprises: Calculating the longest common subsequence of the operation action sequence and the standard action sequence; comparing the standard action sequence with the longest common subsequence, and determining at least one standard production action in the standard action sequence that is not included in the longest common subsequence as at least one missing action in the operation action sequence; The generating, based on the at least one abnormal action, an abnormality detection result of the operation action sequence comprises: Determining a total evaluation score defined in the detection rule of the target process; Determining a missing score corresponding to the at least one missing action according to a scoring method corresponding to the at least one missing action defined in the detection rule; The missing score corresponding to the at least one missing action is deducted from the total evaluation score to determine the evaluation score corresponding to the operation action sequence.

3. The method according to claim 1, characterized in that The comparing the operation action sequence with the standard action sequence to determine at least one abnormal action in the operation action sequence comprises: Calculating the longest common subsequence of the operation action sequence and the standard action sequence; Comparing the operation action sequence with the longest common subsequence, and determining at least one operation production action in the operation action sequence that is not included in the longest common subsequence as at least one redundant action of the operation action sequence; The generating, based on the at least one abnormal action, an abnormality detection result of the operation action sequence comprises: Determining a total evaluation score defined in the detection rule of the target process; Determining a redundancy score corresponding to the at least one redundant action according to a scoring method corresponding to the at least one redundant action defined in the detection rule; The redundant score corresponding to the at least one redundant action is deducted from the total evaluation score to determine the evaluation score corresponding to the operation action sequence.

4. The method according to claim 2, characterized in that: The determining, according to the scoring method corresponding to the at least one missing action defined in the detection rule, the missing score corresponding to the at least one missing action comprises: Determine a weight value corresponding to each of the at least one missing action defined in the detection rule; Determine the total weight value corresponding to each standard production action in the standard action sequence; A missing score corresponding to the missing action is determined according to a first proportion of the weight value of the at least one missing action in the total weight value.

5. The method according to claim 3, characterized in that: Determining, according to the scoring method corresponding to the at least one redundant action defined in the detection rule, a redundancy score corresponding to the at least one redundant action includes: Determine the number of production actions in the standard action sequence and the number of redundant actions of the at least one redundant action according to the scoring method corresponding to the at least one redundant action defined in the detection rule, and determine the total number of production actions; A redundancy score corresponding to at least one redundant action is determined according to a second proportion of the number of redundant actions in the total number of production actions.

6. The method according to claim 5, characterized in that The determining, according to the second proportion of the number of redundant actions in the total number of production actions, a redundancy score corresponding to at least one redundant action comprises: Determining a second proportion of the number of redundant actions to the total number of production actions; determining whether the second proportion exceeds a predetermined proportion; If not, determining a redundancy score corresponding to at least one redundant action according to the second proportion; If so, determine a redundancy score corresponding to at least one redundant action according to the predetermined ratio.

7. The method according to claim 1, characterized in that The operation action sequence generated by identifying the target user executing the target process includes: Acquire an operation video collected by a collection device; the operation video is obtained by collecting a production action performed by the collection device on a target user; Identify the operation action sequence corresponding to the target process from the operation video.

8. The method according to claim 7, characterized in that The step of identifying the operation action sequence corresponding to the target process from the operation video comprises: Performing an identification operation of a production action from the operation video; According to the recognition result, the operation action sequence corresponding to the target process is determined.

9. The method according to claim 8, characterized in that Determining the operation action sequence corresponding to the target process according to the recognition result includes: Determining the production termination action of the target process according to the recognition result; The operation action sequence of the target process is composed of the production termination action and at least one production action between the production termination action and the previous production termination action.

10. The method according to claim 8, characterized in that Determining the operation action sequence corresponding to the target process according to the recognition result includes: Acquire a production termination signal generated by a production device; the production termination signal is generated in response to a termination operation triggered by a target user when a target process is terminated and a production termination action is completed; Determine the production termination time corresponding to the production termination signal; Determine, according to the recognition result, the production termination action corresponding to the production termination time; The operation action sequence of the target process is composed of the production termination action and at least one production action between the production termination action and the previous production termination action.

11. The method according to claim 8, characterized in that The recognition operation of performing the production action from the operation video includes: identifying a production action from the operation identification using an action recognition model; The action recognition model is pre-trained based on a sample video including a sample production action and the action type of the sample production action.

12. The method according to claim 1, characterized in that The step of identifying the operation action sequence corresponding to the target process from the operation video comprises: Segmenting the operation video to obtain a video clip corresponding to a target process; Identify the operation action sequence corresponding to the target process from the video clip.

13. The method according to claim 1, characterized in that Also includes: When determining that there is no abnormal action in the operation action sequence, generating a normal detection result of the operation action sequence; Determine detection results corresponding to multiple operation action sequences corresponding to the target user within a predetermined time range; Based on the detection results respectively corresponding to the multiple operation sequences, generating detection prompt information for the target user; Notify the target user or relevant personnel of the detection prompt information.

14. A computing device, characterized in that including a processing component and a storage component; The storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement the process detection method according to any one of claims 1 to 13.

15. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by the processing component, the process detection method according to any one of claims 1 to 13 is implemented.

16. A computer program product, characterized in that It comprises a computer program / instruction, and when the computer program / instruction is executed by a processing component, it implements the process detection method according to any one of claims 1 to 13.

Images