Process action determination method and device, equipment and storage medium

Abstract

The invention provides a process action determination method and device, equipment and a storage medium. The process action determination method comprises the following steps: acquiring image information when a sewing machine worker works; extracting a plurality of pieces of detection information in the image information; dividing a working area corresponding to a sewing machine operation table into a first working area, a second working area, a third working area and a fourth working area; the process action is determined based on the position relation between the detection information and the first working area, the second working area, the third working area and the fourth working area, and the technical bottleneck that the process action of a sewing machine operator cannot be systematically recorded and the corresponding process cannot be automatically associated in the prior art is solved; the working procedure action can be automatically determined, and the working procedure action determination efficiency and accuracy are improved; an efficient and convenient data acquisition and analysis tool is provided for technical training, production process monitoring and deviation correction, the efficiency of technical guidance and quality control is remarkably improved, and the method is far better than a traditional manual mode.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology and relates to a method, device, equipment and storage medium for determining a process action. Background Art

[0002] The global apparel industry, a multi-trillion dollar market employing approximately 120 million people, faces increasingly severe challenges, primarily rising labor costs and a shrinking supply of young workers, leading to widespread labor shortages.

[0003] At the production operations level, the process of training and improving new employees' skills is inefficient. The current widely adopted practice in the industry relies heavily on industrial engineers (IEs) conducting on-site inspections, manually capturing operation videos, and then conducting time-consuming motion analysis. Based on this analysis, they provide improvement recommendations to optimize sewing workers' efficiency. This approach is not only inefficient but also places strict demands on the experience of IE engineers. Furthermore, the limited pool of such professionals limits training support capabilities. Therefore, improving the efficiency and accuracy of motion analysis has become a pressing technical challenge. Summary of the Invention

[0004] The present application provides a process action determination method, device, equipment and storage medium for determining the process actions of a sewing machine worker when working.

[0005] In the first aspect, the present application provides a method for determining process actions, the method comprising: obtaining image information of a sewing machine worker when working; extracting multiple detection information from the image information; dividing the working area corresponding to the sewing machine operating table into a first working area, a second working area, a third working area and a fourth working area; and determining the process actions based on the positional relationship between the detection information and the first working area, the second working area, the third working area and the fourth working area.

[0006] In an implementation of the first aspect, the multiple detection information includes target detection information and key point detection information, and the extraction of the multiple detection information in the image information includes: constructing a yolov8 model for target detection and key point detection; based on the yolov8 model, extracting the target detection information and the key point detection information, wherein the target detection information includes at least one of the position information of the left hand, the position information of the right hand, and the position information of the face, and the key point detection information includes multiple key points on the face.

[0007] In an implementation of the first aspect, a first average duration of the process action of picking up is determined; within the first average duration, based on the position information of the left hand, the position information of the right hand and the position information of the face, the movement information of the left hand or the right hand and the rotation information of the head in the preset frame number image are determined; based on the movement information of the left hand or the right hand and the rotation information of the head, the process action is determined to be picking up.

[0008] In an implementation method of the first aspect, the second average duration of the process action of releasing or sewing is determined; within the second average duration, if the left hand and the right hand are both located in the second working area, the moving speed and moving range of the left hand and the right hand in the second working area are determined; if the moving speed is greater than a preset speed threshold, the moving range is greater than a preset range threshold, and the face is not facing the needle, the corresponding process action is releasing; if the moving speed is less than or equal to the preset speed threshold, the moving range is less than or equal to the preset range threshold, and the time when the face is facing the needle is greater than the first preset time, the corresponding process action is sewing.

[0009] In an implementation of the first aspect, the third average duration of the process action is determined to be whole, and the sorting time when the left hand and the right hand are located in the third working area is determined; if the sorting time is greater than the third average duration and the face is facing the third working area, the corresponding process action is whole.

[0010] In an implementation of the first aspect, a fourth average duration of a process action is determined as the start or end of a process; within the fourth average duration, when the left hand is located in the first working area and the right hand is located in the fourth working area, and the face is located in the first working area, the fourth working area or directly in front, the corresponding process action is the start or end of a process.

[0011] In an implementation of the first aspect, within the fourth average duration, if the left hand moves from the second working area or the third working area to the first working area, and the face is facing the area where the left hand is located or the face is facing forward, the corresponding process action is the end of the process; or, within the fourth average duration, if the right hand moves from the second working area or the third working area to the fourth working area, and the face is facing the area where the right hand is located or the face is facing forward, the corresponding process action is the end of the process.

[0012] An embodiment of the present application provides a method for determining process actions, which achieves a technological breakthrough in the field of clothing manufacturing and effectively solves the technical bottleneck of being unable to systematically record the process actions of sewing machine operators and automatically associate them with corresponding processes in the past; this method can automatically determine process actions without the need for manual comparison and determination, greatly improving the efficiency and accuracy of process action determination.

[0013] In the second aspect, the present application provides a process action determination device, which includes: an image information acquisition module for acquiring image information of a sewing machine worker when working; a detection information extraction module for extracting multiple detection information from the image information; a work area division module for dividing the work area corresponding to the sewing machine operating table into a first working area, a second working area, a third working area and a fourth working area; a process action determination module for determining the process action based on the positional relationship between the detection information and the first working area, the second working area, the third working area and the fourth working area.

[0014] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the process action determination method described in any one of the first aspects of the embodiments of the present application is implemented.

[0015] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a memory storing a computer program; a processor communicatively connected to the memory, and executing the process action determination method described in any one of the first aspects of the embodiment of the present application when calling the computer program.

[0016] As described above, the process action determination method, device, equipment, and storage medium described in this application have the following beneficial effects:

[0017] 1) This process determination method has achieved a technological breakthrough in the field of garment manufacturing, effectively solving the previous technical bottleneck of being unable to systematically record the work actions of sewing machine operators and automatically associate them with corresponding processes.

[0018] 2) By realizing the automated capture of work actions and process action matching analysis, this method provides an efficient and convenient data collection and analysis tool for technical training, production process monitoring and deviation correction, significantly improving the efficiency of technical guidance and quality control, far exceeding traditional manual methods.

[0019] 3) The amplitude of face rotation is accurately measured in degrees, and different degree intervals are determined as different working intervals. The face rotation angles corresponding to the four working intervals are accurately divided, providing a more accurate data basis for the subsequent determination of process actions based on the left / right hand position information and the face rotation position information.

[0020] 4) By setting multiple constraints such as the left / right hand movement speed, movement range, and face orientation, the process action is determined, which improves the accuracy of determining the process action as placing or sewing and enhances the adaptability to complex scenarios.

[0021] 5) Adding a limit on the third average duration avoids misjudgment of process actions when the left and right hands accidentally fall into the third working area, and improves the accuracy of determining whether the process is full.

[0022] 6) The extraction method has reasonable steps, fast execution speed, low hardware performance requirements, and good practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1A Shown is an application scenario diagram corresponding to the process action determination method provided in an embodiment of the present application.

[0024] Figure 1B Shown is a flowchart of a method for determining a process action according to an embodiment of the present application.

[0025] Figure 2 Shown is a structural diagram of the working area corresponding to the sewing machine operating table provided in one embodiment of the present application.

[0026] Figure 3 Shown is a flowchart of extracting detection information provided by an embodiment of the present application

[0027] Figure 4 Shown is a flowchart of another process action determination method provided by an embodiment of the present application.

[0028] Figure 5 Shown is a flowchart of another process action determination method provided by an embodiment of the present application.

[0029] Figure 6 Shown is a flowchart of another process action determination method provided by an embodiment of the present application.

[0030] Figure 7 Shown is a flowchart of another process action determination method provided by an embodiment of the present application.

[0031] Figure 8 The figure shows a process action determination device provided by an embodiment of the present application.

[0032] Figure 9 An electronic device provided by an embodiment of the present application is shown.

[0033] Component number description

[0034] S11-S13 Step 83 Work Area Division Module

[0035] S31-S32 Step 84 Process Action Determination Module

[0036] S41~S43 Step 90 Electronic Equipment

[0037] S51~S53 Step 91 Processor

[0038] S61-S62 Step 92 Non-volatile storage medium

[0039] S71~S72 Step 93 System Bus

[0040] 80 Process action determination device 94 Internal memory

[0041] 81 Image Information Acquisition Module 95 Network Interface

[0042] 82 Detection Information Extraction Module DETAILED DESCRIPTION

[0043] The following describes the embodiments of the present application through specific examples. Those skilled in the art can easily understand the other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0044] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application. Therefore, the illustrations only show components related to the present application and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.

[0045] The following embodiments of the present application provide a process action determination method, apparatus, device, and storage medium, including but not limited to the hardware application scenarios listed in this embodiment. The following description will take the hardware application scenario corresponding to the process action determination method as an example.

[0046] like Figure 1A As shown in FIG, this embodiment provides a hardware scenario diagram corresponding to a method for determining a process action. Figure 1A As shown, the hardware scenario diagram provided in the embodiment of the present application includes a sewing machine and an electronic device. Among them, the working area corresponding to the sewing machine operating table is divided into a first working area, a second working area, a third working area, and a fourth working area. The sewing machine column is equipped with a camera, which is in communication with the electronic device and is used to capture image information of the worker while working. Specifically, the camera can capture image information in real time and send the captured image information to the electronic device. The electronic device extracts multiple detection information from the image information and determines the process action based on the positional relationship between the detection information and the first working area, the second working area, the third working area, and the fourth working area.

[0047] The technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings in the embodiments of the present application.

[0048] like Figure 1B As shown in FIG, this embodiment provides a flow chart of a method for determining a process action. Figure 1B As shown, the process action determination method provided in the embodiment of the present application includes the following steps S11 to S13.

[0049] S11, obtaining image information of a sewing machine worker while working.

[0050] For example, image information of a sewing machine worker at work may be acquired based on a camera device.

[0051] For example, the camera device may be fixedly mounted on the column of the sewing machine, and the camera device fixedly mounted on the column of the sewing machine may be used to collect image information of the worker while he is working in real time.

[0052] It should be noted that the camera device can also be fixedly installed on the desktop of the sewing machine. In actual applications, the camera device can be fixedly installed at any suitable position on the sewing machine according to specific application requirements. It only needs to be fixed that the camera device can obtain clear image information. This application does not impose any restrictions on this.

[0053] Exemplarily, the camera equipment includes: digital camera, high-definition camera, miniature camera, etc.

[0054] It should be noted that the types of cameras listed above are only for illustrative purposes. In actual applications, any other suitable cameras can be selected according to specific application scenarios, and this application does not impose any restrictions on this.

[0055] S12: extracting a plurality of detection information from the image information.

[0056] S13, dividing the working area corresponding to the sewing machine operating table into a first working area, a second working area, a third working area and a fourth working area.

[0057] For example, the sewing machine operating table can be divided into work areas based on the working state of the sewing machine worker. For example, the leftmost part of the sewing machine operating table can be determined as the first work area, the rightmost part of the sewing machine operating table can be determined as the fourth work area, the top part of the sewing machine operating table can be determined as the second work area, and the bottom part of the sewing machine operating table can be determined as the third work area.

[0058] It should be noted that the division of the sewing machine operating table listed in the above example is only for illustrative purposes. In actual applications, the sewing machine operating table can be divided into any other appropriate divisions according to specific application scenarios, and this application does not impose any restrictions on this.

[0059] See also Figure 2 , Figure 2 Shown is a structural diagram of the working area corresponding to the sewing machine operating table provided in one embodiment of the present application. Figure 2 It can be seen that the sewing machine operating table also includes a blind area, and the area corresponding to the blind area is the position where the sewing machine column is located.

[0060] S14 , determining a process action based on the detection information and a positional relationship between the first working area, the second working area, the third working area, and the fourth working area.

[0061] An embodiment of the present application provides a method for determining process actions, which includes obtaining image information of a sewing machine worker at work; extracting multiple detection information from the image information; dividing the working area corresponding to the sewing machine operating table into a first working area, a second working area, a third working area, and a fourth working area; and determining the process actions based on the positional relationship between the detection information and the first working area, the second working area, the third working area, and the fourth working area. The process determination method achieves a technological breakthrough in the field of clothing manufacturing, effectively solving the technical bottleneck of being unable to systematically record the work actions of sewing machine operators and automatically associate them with corresponding processes. By realizing the automated capture of work actions and matching analysis of process actions, the method provides an efficient and convenient data collection and analysis tool for technical training, production process monitoring, and deviation correction, significantly improving the efficiency of technical guidance and quality control, far exceeding traditional manual methods.

[0062] like Figure 3 As shown, this embodiment provides a flow chart for extracting detection information. The plurality of detection information includes target detection information and key point detection information. Figure 3As shown, the process action determination method provided in the embodiment of the present application includes the following steps S31 to S32.

[0063] S31, build the yolov8 model for target detection and key point detection.

[0064] Specifically, the original standard yolov8 model includes a backbone network, a neck network, and a detection head (target detection head).

[0065] Specifically, an independent keypoint detection head is added in parallel to the original standard yolov8 model, enabling it to simultaneously output the bounding box, category, and keypoint coordinates of the detection information. The object detection head and the keypoint detection head can share the features extracted by the backbone network and the neck network, but use independent branches for calculation in the final prediction layer. Compared with the traditional two-stage (first object detection and then keypoint detection) or cascade method, a single forward propagation can simultaneously complete object detection and keypoint detection, maintaining the simplicity of the yolov8 model, making it easy to integrate and deploy in actual systems while significantly improving the detection speed.

[0066] S32, based on the yolov8 model, extracting the target detection information and the key point detection information.

[0067] The target detection information includes at least one of the position information of the left hand, the position information of the right hand and the position information of the face, and the key point detection information includes.

[0068] For example, the key point information includes 68 key points of the face. Specific partitioning of the 68 key points is shown in Table 1.

[0069] Table 168 specific partitions of key points

[0070]

[0071]

[0072] Targeted extraction is performed based on the spatial location characteristics and functional importance of the facial anatomical structure, and different extraction strategies are used in different areas. The specific extraction methods are shown in Table 2.

[0073] Table 2 Extraction methods of key points in different regions

[0074] Key Points Extraction method Original 68-point index left eye Mean coordinates of 6 points in the left eye area [36,37,38,39,40,41] right eye Mean coordinates of 6 points in the right eye area [42,43,44,45,46,47] tip of the nose Directly take the coordinates of the nose tip Index 33 (or 31) Left corner of mouth Directly take the coordinates of the left endpoint of the outer lip Index 48 right corner of mouth Directly take the coordinates of the right endpoint of the outer lip Index 54 jaw Directly take the coordinates of the lowest point of the mandibular contour Index 8

[0075] It should be noted that the specific partitioning of the 68 key points in Table 1 and the method of extracting key points in different areas in Table 2 are only illustrative. In actual applications, any other appropriate number of key points and the corresponding key point extraction method can be selected according to specific application requirements. This application does not impose any restrictions on this.

[0076] The embodiment of the present application provides a method for extracting detection information. By adding an independent key point detection head in parallel on the basis of the original standard YOLOv8 model, a single forward propagation can simultaneously complete target detection and key point detection, thereby maintaining the simplicity and efficiency of the YOLOv8 model, greatly improving the efficiency of extracting detection information, and facilitating integration and deployment in actual systems, thereby reducing the complexity of integration and deployment of the YOLOv8 model in actual systems. By extracting multiple key points on the face, not only the accuracy of facial key point recognition is significantly improved, but also it is ensured that predefined key feature points, such as left eye / right eye, nose tip, corner of mouth, etc., can be stably and accurately located under conditions of complex lighting, posture changes, etc. At the same time, a variety of key point extraction methods are adopted, and specific extraction methods are adopted for different key points, which can meet the needs of high-throughput or low-latency application scenarios.

[0077] like Figure 4 As shown in FIG, this embodiment provides a flowchart of another method for determining a process action. Figure 4 As shown, the process action determination method provided in the embodiment of the present application includes the following steps S41 to S43.

[0078] S41, determining a first average duration of the process action of taking.

[0079] For example, the first average duration of the process action may be determined based on previous working experience of sewing workers.

[0080] It should be noted that this application does not limit the specific data of the first average duration.

[0081] S42, within the first average duration, based on the position information of the left hand, the position information of the right hand and the position information of the face, determine the movement information of the left hand or the right hand and the rotation information of the head in the preset frame number image.

[0082] The preset frame number images may be 10 consecutive frames of images or 20 consecutive frames of images.

[0083] It should be noted that the number of preset frame images can be reasonably determined according to the specific application scenario, and this application does not impose any restrictions on this.

[0084] Specifically, the position information of the left hand, the position information of the right hand and the position information of the face are obtained based on the above-mentioned yolov8 model.

[0085] For example, the Euler angles pitch, yaw, and roll of face rotation are calculated based on the average face size, using the formula Calculate the face rotation range. Based on the left / right hand working area and the face rotation range in Table 3, determine the corresponding process action.

[0086] Table 3 Correspondence between the amplitude of face rotation and the maximum working area

[0087] area Pitch Yaw Roll First working area [-68.20°,-45.00°] [-56.31°,-45.00°] Not obvious Second working area [-70.02°,-68.20°] [-45.00°,63.43°] Not obvious The third working area [-68.20°,0.00°] [-45.00°,63.43°] Not obvious Fourth working area [-70.02°,0.00°] [63.43°,90.00°] Not obvious

[0088] It should be noted that the angles listed in Table 3 are only for illustrative purposes. In actual applications, any other appropriate angle values ​​can be selected according to specific application requirements, and this application does not impose any restrictions on this.

[0089] S43, based on the movement information of the left hand or the right hand and the rotation information of the head, determining that the process action is to take.

[0090] For example, if the left hand moves from the first working area to the second working area or the third working area, and the head turns from the first working area to the second working area or the third working area, leading the face toward the area where the left hand is located or directly forward, the corresponding process action is "holding";

[0091] As another example, if the right hand moves from the fourth working area to the second working area or the third working area, and the head turns from the first working area to the second working area or the third working area, driving the face toward the area where the right hand is located or directly forward, the corresponding process action is to take.

[0092] An embodiment of the present application provides another method for determining process actions, specifically for determining the process action of "grabbing", and determining the process action as "grabbing" based on the movement information of the left hand or right hand and the rotation information of the head. The relationship between the position information of the left hand / right hand and the position information of the face rotation and the four working areas is specifically analyzed, the amplitude of the face rotation is accurately determined to degrees, and different degree intervals are determined as different working intervals. The face rotation angles corresponding to the four working intervals are accurately divided, which provides a more accurate data basis for subsequently determining that the process action is "grabbing" based on the position information of the left hand / right hand and the position information of the face rotation.

[0093] like Figure 5 As shown in FIG, this embodiment provides a flowchart of another method for determining a process action. Figure 5 As shown, the process action determination method provided in the embodiment of the present application includes the following steps S51 to S54.

[0094] S51, determining a second average duration of the process action of placing or sewing.

[0095] S52: If both the left hand and the right hand are located in the second working area during the second average duration, determine the moving speed and moving range of the left hand and the right hand in the second working area.

[0096] For example, the left / right hand position within the second average duration can be detected in real time based on the yolov8 model, the hand motion trajectory can be generated, and the movement time corresponding to the hand motion trajectory can be recorded. Based on the hand motion trajectory / movement time, the displacement change per unit time can be obtained, and then the hand movement speed can be obtained.

[0097] For example, the left / right hand position can be detected in real time based on the yolov8 model, a hand motion trajectory can be generated, the maximum coordinate and the minimum coordinate of the hand motion trajectory in the second working area can be determined, and the movement range can be determined based on the maximum coordinate and the minimum coordinate.

[0098] It should be noted that the methods for determining the moving speed and moving range listed in the above two embodiments are only for illustrative purposes. In actual applications, any other suitable method can be selected to determine the moving speed and moving range according to specific application requirements. This application does not impose any restrictions on this.

[0099] S53, if the moving speed is greater than the preset speed threshold, the moving range is greater than the preset range threshold, and the face is not facing the needle, the corresponding process action is release.

[0100] For example, the preset speed threshold may be set to 0.5 m / s, 0.3 m / s, etc., and the preset range threshold may be set to 30 cm, 25 cm, 35 cm, etc.

[0101] When the moving speed is greater than the preset speed threshold, the moving range is greater than the preset range threshold, and the face is not facing the needle, the corresponding process action is release.

[0102] It should be noted that the specific values ​​of the preset speed threshold and the preset range threshold listed in the above examples are only for illustrative purposes. In actual applications, any suitable setting can be selected as the preset speed threshold or the preset range threshold according to the specific application scenario. This application does not impose any restrictions on this.

[0103] S54, if the moving speed is less than or equal to the preset speed threshold, the moving range is less than or equal to the preset range threshold, and the time when the face is facing the needle is greater than the first preset time, the corresponding process action is sewing.

[0104] For example, the first preset time may be 5s, 10s, 1min, etc.

[0105] It should be noted that the specific values ​​of the first preset time listed above are only for illustrative purposes. In actual applications, the first preset time can also be determined as any other suitable value according to the specific application scenario, and this application does not impose any restrictions on this.

[0106] The present application provides another method for determining process actions, specifically determining the process actions of "releasing" and "sewing." In this method, the movement speed and range of the left and right hands in the second working area are determined; a preset speed threshold and a preset range threshold are set. By comparing the movement speed, movement range, and facial orientation of the sewing machine worker's left / right hands during actual work, the corresponding process action is determined to be "releasing" or "sewing." By setting multiple constraints to determine the process action, the accuracy of determining the process action as "releasing" or "sewing" is improved, and the adaptability to complex scenarios is enhanced.

[0107] like Figure 6 As shown in FIG, this embodiment provides a flow chart of another method for determining a process action. Figure 6 As shown, the process action determination method provided in the embodiment of the present application includes the following steps S61 to S63.

[0108] S61, determining a third average duration of the process action.

[0109] S62, determining the finishing time of the left hand and the right hand in the third working area.

[0110] Exemplarily, the yolov8 model is used to determine whether the left hand and the right hand are located in the third working area. If the yolov8 model detects that the left hand and the right hand are located in the third working area, the hand motion trajectory is generated based on the yolov8 model, and the motion time corresponding to the hand motion trajectory is recorded. The motion time is the sorting time.

[0111] S63: If the tidying time is greater than the third average duration and the face of the person is facing the third working area, the corresponding process action is tidying.

[0112] Exemplarily, the third average duration may be any appropriate time such as 1 minute or 3 minutes.

[0113] It should be noted that the specific values ​​of the third average duration listed above are only used for illustrative purposes. In actual applications, any other appropriate time value can be selected as the third average duration according to specific application requirements, and this application will not go into details.

[0114] An embodiment of the present application provides another method for determining process actions, in which the sorting time of the left hand and the right hand in the third working area is determined; if the sorting time is greater than the third average duration and the face of the person is facing the third working area, the corresponding process action is whole; in this method, the sorting time of the left hand and the right hand in the third working area is limited, and when the sorting time is greater than the third average duration, the corresponding process action is whole, thereby avoiding misjudgment of the process action when the left hand and the right hand mistakenly fall into the third working area, increasing the limit of the third average duration, and improving the accuracy of determining whether the process is whole.

[0115] like Figure 7 As shown in FIG, this embodiment provides a flow chart of another method for determining a process action. Figure 7 As shown, the process action determination method provided in the embodiment of the present application includes the following steps S71 to S72.

[0116] S71, within the fourth average duration, if the left hand moves from the second working area or the third working area to the first working area, and the face is facing the area where the left hand is located or the face is facing forward, the corresponding process action is the end of the process, or.

[0117] S72, within the fourth average duration, if the right hand moves from the second working area or the third working area to the fourth working area, and the face is facing the area where the right hand is located or the face is facing straight ahead, the corresponding process action is process end.

[0118] In some embodiments, the method further includes: determining a fourth average duration of a process action as a process start or a process end; within the fourth average duration, when the left hand is located in the first working area and the right hand is located in the fourth working area, and the face is located in the first working area, the fourth working area or directly in front, the corresponding process action is a process start or a process end.

[0119] An embodiment of the present application provides another method for determining process actions. In this method, by determining the position information of the left hand or the right hand, and the position information of the face, it can be determined that the corresponding process action is the start or end of the process. In addition to the above-mentioned process actions of determining holding, arranging, placing, and sewing, the embodiment of the present application also provides a method for determining the start or end of the process, which ensures the integrity of the process when the sewing machine worker is working, and can clearly define the complete process to ensure the integrity of the determined process actions; by monitoring the rules of process start and process end, abnormal situations can be more easily detected, such as a process being interrupted for no reason, abnormal shortening or extension of the time consumption, etc., and an alarm can be issued in time; it provides relevant technical personnel with a very convenient data collection and analysis tool for product training and in-production correction, which greatly improves work efficiency compared to manual collection and guidance; at the same time, it provides a convenient knowledge base for workers to learn independently, greatly reduces the learning cost, and helps to quickly improve product quality.

[0120] The protection scope of the process action determination method described in the embodiment of the present application is not limited to the execution order of the steps listed in this embodiment. All solutions implemented by adding, reducing, or replacing steps in the existing technology based on the principles of the present application are included in the protection scope of the present application.

[0121] An embodiment of the present application also provides a process action determination device, which can implement the process action determination method described in the present application. However, the implementation device of the process action determination method described in the present application includes but is not limited to the structure of the process action determination device listed in this embodiment. All structural deformations and replacements of the existing technology made according to the principles of the present application are included in the protection scope of the present application.

[0122] like Figure 8 As shown, an embodiment of the present application provides a process action determination device. The process action determination device 80 of the present application includes an image information acquisition module 81, a detection information extraction module 82, a work area division module 83, and a process action determination module 84.

[0123] The image information acquisition module 81 is used to acquire image information of a sewing machine operator at work. The detection information extraction module 82 is used to extract multiple detection information from the image information. The work area division module 83 is used to divide the work area corresponding to the sewing machine operating table into a first work area, a second work area, a third work area, and a fourth work area. The process action determination module 84 is used to determine the process action based on the positional relationship between the detection information and the first work area, the second work area, the third work area, and the fourth work area.

[0124] Among them, the structures and principles of the image information acquisition module 81, the detection information extraction module 82, the work area division module 83, and the process action determination module 84 correspond one to one with the steps in the above-mentioned process action determination method, so they are not repeated here.

[0125] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices or methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules / units is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules or units, which can be electrical, mechanical or other forms.

[0126] The modules / units described as separate components may or may not be physically separate, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present application. For example, the functional modules / units in the various embodiments of the present application may be integrated into a processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into a single module / unit.

[0127] Those skilled in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0128] An embodiment of the present application also provides an electronic device. Figure 9 The diagram shows the structure of an electronic device 90 in one embodiment of the present application. The process action determination method provided in the embodiment of the present application can be applied to Figure 9 The electronic device 90 shown is, but not limited to, Figure 9 As shown, the electronic device 90 includes a processor 91 , a memory, a system bus 93 , and a network interface 95 , wherein the memory may include a non-volatile storage medium 92 and an internal memory 94 .

[0129] The non-volatile storage medium 92 can store an operating system and a computer program. The computer program includes program instructions, which, when executed, can cause the processor to execute any one of the process action determination methods provided in the embodiments of the present application.

[0130] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0131] The internal memory 94 provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any one of the process action determination methods provided in the embodiments of the present application.

[0132] The network interface 99 is used for network communication, such as sending assigned tasks, etc. It will be understood by those skilled in the art that Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0133] It should be understood that the processor 91 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0134] The electronic device 90 in the embodiment of the present application may include terminal devices such as tablet computers, laptop computers, mobile phones, supercomputers, smart wearable devices, etc., and can also be applied to databases, servers, and service response systems based on terminal artificial intelligence. The embodiment of the present application does not impose any restrictions on the specific type of electronic device.

[0135] For example, the electronic device can be a station (STAION, ST) in a WLAN, a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, a computer, a laptop computer, a handheld communication device, a handheld computing device, and / or other devices for communicating on a wireless system and next-generation communication systems, such as mobile terminals in a 5G network, a mobile terminal in a future-evolved Public Land Mobile Network (PLMN), or a mobile terminal in a future-evolved Non-terrestrial Network (NTN).

[0136] The embodiment of the present application also provides a computer-readable storage medium. Those skilled in the art will understand that all or part of the steps in the method for implementing the above embodiment can be completed by instructing the processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state drive, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a digital video disc (DVD)), or a semiconductor medium (for example, a solid-state drive (SSD)), etc.

[0137] The embodiment of the present application may also provide a computer program product, the computer program product including one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the process or function described in the embodiment of the present application is generated in whole or in part. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer or data center to another website, computer or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method.

[0138] When the computer program product is executed by a computer, the computer executes the method described in the above method embodiment. The computer program product can be a software installation package. When the above method is needed, the computer program product can be downloaded and executed on the computer.

[0139] The descriptions of the processes or structures corresponding to the above figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0140] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concepts disclosed in this application shall be covered by the claims of this application.

Claims

1. A method for determining a process action, characterized in that: The method comprises: Obtain image information of sewing machine workers at work; extracting a plurality of detection information from the image information; Divide the working area corresponding to the sewing machine operating table into a first working area, a second working area, a third working area and a fourth working area; A process action is determined based on the positional relationship between the detection information and the first working area, the second working area, the third working area, and the fourth working area.

2. The process action determination method according to claim 1, characterized in that: The plurality of detection information includes target detection information and key point detection information, and the extracting the plurality of detection information from the image information includes: Build the yolov8 model for target detection and key point detection; Based on the yolov8 model, the target detection information and the key point detection information are extracted, wherein the target detection information includes at least one of the position information of the left hand, the position information of the right hand and the position information of the face, and the key point detection information includes multiple key points on the face.

3. The process action determination method according to claim 2, wherein: The method further comprises: Determine the first average duration of the process action to take; During the first averaging duration, determining, based on the position information of the left hand, the position information of the right hand, and the position information of the face, movement information of the left hand or the right hand and rotation information of the head within a preset number of frames of images; Based on the movement information of the left hand or the right hand and the rotation information of the head, the process action is determined to be picking up.

4. The process action determination method according to claim 2, wherein: The method further comprises: Determine the second average duration of the process action being placing or sewing; During the second average duration, if both the left hand and the right hand are located in the second working area, determining the movement speed and movement range of the left hand and the right hand in the second working area; If the moving speed is greater than the preset speed threshold, the moving range is greater than the preset range threshold, and the face is not facing the needle, the corresponding process action is release; If the moving speed is less than or equal to the preset speed threshold, the moving range is less than or equal to the preset range threshold, and the time when the face is facing the needle is greater than the first preset time, the corresponding process action is sewing.

5. The process action determination method according to claim 2, wherein: The method further comprises: Determine the third average duration of the process action; determining the finishing time when the left hand and the right hand are located in the third working area; If the tidying time is greater than the third average duration and the face of the person is facing the third working area, the corresponding process action is tidying.

6. The process action determination method according to claim 2, wherein: The method further comprises: Determine the fourth average duration of the process action as the process start or process end; During the fourth average duration, when the left hand is located in the first working area and the right hand is located in the fourth working area, and the face is located in the first working area, the fourth working area or directly in front, the corresponding process action is process start or process end.

7. The process action determination method according to claim 6, characterized in that: The method further comprises: During the fourth average duration, if the left hand moves from the second working area or the third working area to the first working area, and the face faces the area where the left hand is located or faces straight ahead, the corresponding process action is the end of the process, or During the fourth average duration, if the right hand moves from the second working area or the third working area to the fourth working area, and the face faces the area where the right hand is located or faces straight ahead, the corresponding process action is process completion.

8. A process action determination device, characterized in that: The device comprises: An image information acquisition module is used to acquire image information of a sewing machine worker while he is working; A detection information extraction module, configured to extract a plurality of detection information from the image information; A working area division module, used to divide the working area corresponding to the sewing machine operating table into a first working area, a second working area, a third working area and a fourth working area; The process action determination module is used to determine the process action based on the positional relationship between the detection information and the first working area, the second working area, the third working area, and the fourth working area.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the process action identification method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: The electronic device comprises: a memory storing a computer program; A processor is communicatively connected to the memory, and executes the process action determination method according to any one of claims 1 to 7 when calling the computer program.

Images