A method and system for factory parts abnormal handling identification

Abstract

The application relates to the technical field of artificial intelligence in the production field, in particular to a method for factory part abnormal carrying identification, which comprises the following steps: based on a two-dimensional image coordinate system, a two-dimensional reference vector is preset, the reference vector represents a physical direction or path that needs to be measured and displaced; a video stream is acquired, and frame images are acquired by reading frame by frame at a preset frame rate; based on a preset target detection model, the boundary boxes and unique IDs of all personnel, parts and authorized tools in the frame images are acquired, the displacement vectors corresponding to each qualified rule-violating personnel are analyzed based on time sequence, projection operation is performed on the displacement vectors and the reference vector to analyze whether the displacement vectors are effective, and the rule-violating personnel corresponding to the effective displacement vectors are marked as effective rule-violating personnel; displacement calculation is performed on the effective rule-violating personnel, and abnormal carrying state judgment is performed based on the displacement calculation result; and the application has the advantages that the carrying process of the parts can be automatically and intelligently monitored.

Description

Technical Field

[0001] This application relates to the technical field of artificial intelligence in the production sector, and in particular to a method and system for identifying abnormal handling of factory parts. Background Technology

[0002] In the process of modern industrial production, the operation of discrete manufacturing plants faces many complex challenges. Among these challenges, the orderly and standardized flow of parts and components in assembly lines, warehousing areas, and logistics transit areas plays a decisive role in ensuring production efficiency and product quality. Smooth and compliant parts and components flow is a key link in ensuring the continuity of the production process and improving the stability of product quality.

[0003] Traditional management methods mainly rely on manual inspections, visual supervision, or area control based on sensors such as RFID and infrared beams. However, these methods have significant drawbacks: Firstly, manual supervision is costly and inefficient, requiring dedicated personnel to monitor the screen for extended periods or conduct on-site inspections. This can easily lead to missed reports due to fatigue or negligence, and the labor costs are also high.

[0004] Secondly, traditional sensors have significant limitations. Taking RFID technology as an example, although it can confirm whether an item has passed through a specific gate, it has serious limitations in terms of information acquisition. It cannot identify the identity information of specific handling employees, nor can it determine whether the handling path and distance comply with regulations. This makes it impossible to comprehensively and meticulously control handling activities during the management process.

[0005] Infrared sensors, on the other hand, have a more limited function, only able to detect objects passing through them, and provide extremely limited information dimensions, which cannot meet the needs of monitoring the flow of parts in complex and ever-changing production scenarios.

[0006] Thirdly, post-incident tracing is difficult. Once problems such as lost or misplaced parts occur, current methods typically rely on manual review of massive amounts of video footage. This process is not only time-consuming and labor-intensive, but also, due to the sheer volume of video data, crucial clues may be lost, making it difficult to successfully identify the root cause of the problem and posing significant challenges to problem-solving and responsibility determination.

[0007] Fourthly, there is a lack of intelligent analysis of behavior. Most existing security monitoring systems are still at the stage of "recording" and "presenting" and cannot actively understand the content of the video, especially the connection between "people" and "objects" and the ability to quantify and make compliance judgments on the complex behavior of "moving" that the two together constitute.

[0008] In summary, most existing security monitoring systems remain at the "recording" and "presentation" stage, unable to actively understand the content of the footage. In particular, they cannot associate "people" with "objects," nor can they distinguish between "unauthorized manual handling" and "compliant handling using tools," two behaviors that appear similar but are completely different. This leads to a large number of false alarms in automated monitoring, reducing the reliability and practical value of the system.

[0009] Therefore, the market urgently needs a technological solution that can automatically and intelligently monitor the parts handling process, accurately distinguish between abnormal and normal handling behaviors, and issue timely and effective early warnings. Summary of the Invention

[0010] In order to achieve automatic and intelligent monitoring of the parts handling process, this application provides a method for identifying abnormal parts handling in a factory.

[0011] Firstly, this application provides a method for identifying abnormal handling of factory parts, employing the following technical solution: A method for identifying abnormal handling of factory parts includes: Based on a two-dimensional image coordinate system, a two-dimensional reference vector is preset, which represents the physical direction or path of interest and the displacement to be measured. Acquire the video stream and read it frame by frame at a preset frame rate to obtain frame images; Based on the preset target detection model, obtain the bounding boxes and unique IDs of all personnel, parts and authorized tools in the frame image; Based on the bounding boxes and unique IDs of personnel, parts, and authorized tools in the same frame image, determine the handling status and tool usage status of each personnel; A composite correlation determination is performed on the handling status and tool usage status of each person to obtain all the non-compliant persons. The non-compliant persons are characterized as those whose handling status is yes and tool usage status is no. Based on time series analysis, the displacement vector corresponding to each qualified violator is used to project the displacement vector onto the reference vector to analyze its validity. Violators corresponding to valid displacement vectors are marked as valid violators. Displacement calculations are performed on valid violators, and abnormal handling status is determined based on the displacement calculation results.

[0012] In one embodiment: the target detection model includes a detection model and a tracking model, the detection model being used to process the input frame image and output bounding boxes of all people, parts and authorized tools in the image; The tracking model is used to identify the bounding boxes of input personnel and components and assign or update their unique IDs.

[0013] In one embodiment: the step of determining the handling status and tool usage status of each person based on the bounding boxes and unique IDs of personnel, parts, and authorized tools in the same frame image includes: Based on the bounding boxes of personnel and authorized tools in the same frame image, the tool usage status of each personnel is obtained by calculating the spatial overlap or proximity. When the overlap or proximity of the bounding box of a personnel and at least one authorized tool exceeds a first threshold, the tool usage status is "yes" and otherwise "no". Based on the bounding boxes of personnel and components in the same frame image, the handling status of each personnel is obtained by spatial overlap calculation. When the overlap between the personnel and the bounding box of at least one component exceeds a second threshold, the handling status is "yes" and otherwise "no".

[0014] In one embodiment, the method for determining whether the violator meets the requirements specifically includes: An independent time-series buffer is established for each violator, and the bounding box information and frame information corresponding to the violator are stored in the time-series buffer. The time-series buffer has multiple storage bits. Meeting the requirements means that the time-series buffer corresponding to the violator is filled.

[0015] In one embodiment, the method for obtaining the displacement vector specifically includes: Extract the bounding box information of the first and last two storage bits in the time-series buffer; Calculate the displacement vector based on the position coordinates in the two bounding box information.

[0016] In one embodiment: the step of calculating the displacement of valid violators and judging the abnormal handling status based on the displacement calculation results specifically includes: Based on the bounding box information in the current time-series buffer, calculate the physical distance increment of the corresponding violator in the time-series buffer. Obtain the total cumulative displacement variable corresponding to the previous time-series buffer. If there is no previous time-series buffer, or the time interval between the previous time-series buffer and the current time-series buffer is greater than the time threshold, use the physical distance increment corresponding to the current time-series buffer as the total cumulative displacement variable; otherwise, add the physical distance increment corresponding to the current time-series buffer to the total cumulative displacement variable. When the total cumulative displacement variable exceeds the preset displacement threshold, the corresponding person is determined to be in an abnormal handling state.

[0017] In one embodiment: the step of calculating the physical distance increment of the corresponding violator in the current time-series buffer based on the bounding box information in the current time-series buffer specifically includes: Retrieve all bounding box information in the time-series buffer; Based on the bounding box information, the displacement coefficients in the two-dimensional image coordinate system are obtained; Based on the location coordinates of the bounding box information, calculate the effective pixel displacement components of the violator; The physical distance increment is calculated based on the product of the effective pixel displacement component and the displacement coefficient.

[0018] In one embodiment: the step of obtaining the displacement coefficient in the two-dimensional image coordinate system based on the bounding box information specifically includes: Obtain the preset reference size factor; The average bounding box width is obtained by averaging the bounding box widths in the bounding box information. The displacement coefficient is calculated based on the ratio of the reference size factor to the average bounding box width.

[0019] In one embodiment: the step of calculating the effective pixel displacement component of the violator based on the position coordinates of the bounding box information specifically includes: Calculate the pixel displacement vector between all position coordinates using a pairwise combination method; Based on projection operations, obtain the pixel displacement components of the pixel displacement vector in the direction of the reference vector; The maximum value of the pixel shift component is taken as the effective pixel shift component.

[0020] Secondly, this application provides a system for identifying abnormal handling of factory parts, employing the following technical solution: A system for identifying abnormal handling of factory parts includes: The reference vector preset module is used to preset a two-dimensional reference vector based on a two-dimensional image coordinate system. The reference vector is used to characterize the physical direction or path of the object of interest and the displacement to be measured. The video processing module is used to acquire the video stream and read it frame by frame according to a preset frame rate to obtain frame images; The target detection module is used to detect and obtain the bounding boxes of all personnel, parts and authorized tools in the acquired frame images based on a preset target detection model. The handling status composite association module is used to determine the handling status and tool usage status of each person based on the bounding boxes of personnel, parts and authorized tools in the same frame image; and to perform composite association judgment on the handling status and tool usage status of each person to identify all non-compliant personnel, wherein the non-compliant personnel are those whose handling status is yes and tool usage status is no. The handling distance calculation module analyzes the displacement vector corresponding to each qualified violator based on time series analysis. It then performs projection calculations on the displacement vector and reference vector to analyze its validity, marking violators corresponding to valid displacement vectors as valid violators; and performs displacement calculations on valid violators. The result output module is used to determine whether the object is in an abnormal handling state based on the displacement calculation results.

[0021] In summary, this application has the following beneficial effects: 1. Based on composite correlation judgment, it can automatically and intelligently monitor the handling process of parts and can intelligently distinguish between two states: "abnormal handling over long distances by hand" and "normal long-distance handling using authorized tools"; 2. It can directly perform recognition based on video data from a monocular vision sensor, with lower equipment requirements; 3. By flexibly applying the principles of projection calculation, noise reduction is achieved, and the computational load is reduced. Simultaneously, based on the depth changes of a monocular vision sensor, the actual physical displacement changes are accurately calculated, enabling precise identification of abnormal states. 4. Based on caching and accumulation design, the actions of personnel are decomposed and jointly judged to improve the recognition accuracy. Attached Figure Description

[0022] Figure 1 This is a framework diagram of a system for identifying abnormal handling of factory parts according to this embodiment; Figure 2 This is a flowchart of a method for identifying abnormal handling of factory parts in this embodiment.

[0023] In the diagram, 10 is the reference vector preset module; 20 is the video processing module; 30 is the target detection module; 40 is the transport state composite association module; 50 is the transport distance calculation module; and 60 is the result output module. Detailed Implementation

[0024] The present application will be further described in detail below with reference to the accompanying drawings.

[0025] To better understand the purpose, technical solutions, and advantages of this application, it has been described and illustrated below with reference to the accompanying drawings and embodiments. However, those skilled in the art should understand that this application can be implemented without these details. In some cases, to avoid obscuring various aspects of this application due to unnecessary description, well-known methods, processes, systems, components, and / or circuits already described at a higher level will not be elaborated upon. It will be apparent to those skilled in the art that various modifications can be made to the embodiments disclosed in this application, and the general principles defined in this application can be applied to other embodiments and application scenarios without departing from the principles and scope of this application. Therefore, this application is not limited to the illustrated embodiments, but conforms to the broadest scope consistent with the scope of protection claimed in this application.

[0026] It should be noted that the descriptions of these embodiments are for the purpose of aiding understanding the present invention, but do not constitute a limitation thereof. Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0027] In the description of this application, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0028] In the description of this application, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples.

[0029] like Figure 1 As shown, this embodiment discloses a system for identifying abnormal handling of factory parts. The system achieves overall identification function by combining with a monocular vision sensor. It includes a reference vector preset module 10, a video processing module 20, a target detection module 30, a handling state composite association module 40, a handling distance calculation module 50, and a result output module 60.

[0030] The reference vector preset module 10 is used to preset a two-dimensional reference vector based on the two-dimensional image coordinate system. The reference vector is used to characterize the physical direction or path of the object of interest and the displacement to be measured. The video processing module 20 is used to acquire the video stream and read it frame by frame according to a preset frame rate to obtain frame images; The target detection module 30 is used to detect and obtain the bounding boxes of all personnel, parts and authorized tools in the acquired frame image based on a preset target detection model. The handling status composite association module 40 is used to determine the handling status and tool usage status of each person based on the bounding boxes of personnel, parts and authorized tools in the same frame image; to perform composite association judgment on the handling status and tool usage status of each person to identify all non-compliant personnel, where non-compliant personnel refer to personnel whose handling status is yes and tool usage status is no; The handling distance calculation module 50 analyzes the displacement vector corresponding to each qualified violator based on time series analysis, performs projection calculation on the displacement vector and the reference vector to analyze its validity, marks the violators corresponding to the valid displacement vectors as valid violators, and calculates the displacement of valid violators. The result output module 60 is used to determine whether the object is in an abnormal handling state based on the displacement calculation results.

[0031] like Figure 2 As shown, this embodiment also discloses a method for identifying abnormal handling of factory parts, the specific steps of which include: S100. Based on a two-dimensional image coordinate system, a two-dimensional reference vector is preset. The reference vector represents the physical direction or path of the displacement that is of interest and needs to be measured.

[0032] The reference vector is defined by the starting and ending coordinates in a two-dimensional image coordinate system. In a specific industrial production line monitoring application, monitoring video is acquired by a monocular vision sensor, which uses a fixed viewing angle.

[0033] The operator defines a reference direction on the monitoring screen, such as from the start point P1(x1,y1) to the end point P2(x2,y2) of the production line, and the system calculates the reference vector Vref=P2-P1 accordingly.

[0034] S200: Acquire the video stream and read it frame by frame at a preset frame rate to obtain frame images.

[0035] In this step, the system captures video streams in real time or offline using a monocular vision sensor and reads images frame by frame at a preset frame rate.

[0036] S300: Based on a preset target detection model, obtain the bounding boxes and unique IDs of all personnel, components, and authorized tools in the frame image.

[0037] In this step, the target detection model includes a detection model and a tracking model.

[0038] The detection models include a personnel detection model, a component detection model, and an authorized handling tool detection model. All three models were pre-trained using YOLOv11 before loading.

[0039] Thus, by processing the input frame image through the detection model, the bounding boxes, categories, and confidence scores of all personnel, components, and authorized tools in the image can be directly output. Confidence filtering and non-maximum suppression are then applied to the three types of detection results to obtain a precise set of bounding boxes for personnel, components, and authorized tools.

[0040] In this embodiment, the tracking model is a ByteTrack algorithm module with multi-target tracking capabilities, used to assign a unique and persistent identifier to the same object in consecutive video frames. Thus, after the detection model outputs bounding boxes, the tracking model can identify the bounding box sets of input personnel and components, and assign or update their unique IDs, thereby achieving multi-target tracking and identification.

[0041] In order to improve the recognition speed, the tracking model can also identify only the bounding box set of people.

[0042] S400 determines the handling status and tool usage status of each person based on the bounding boxes and unique IDs of personnel, parts, and authorized tools in the same frame image.

[0043] In this step, multiple bounding boxes within the same image frame are identified to determine whether a person is handling parts by hand or using authorized tools. Since it is not necessary to determine whether the person is handling parts when using tools, it is only necessary to determine whether parts are being handled when tools are not being used. Therefore, it is only necessary to determine the handling status and tool usage status of each person.

[0044] In this embodiment, based on a unique ID, the carrying status and tool usage status of each person in the image are determined. Specifically, the carrying status and tool usage status are identified in the following ways: Based on the bounding boxes of personnel and authorized tools in the same frame image, the tool usage status of each personnel is obtained by calculating the spatial overlap or proximity. When the overlap or proximity of the bounding box of a personnel with at least one authorized tool exceeds a first threshold, the tool usage status is "yes" and otherwise "no".

[0045] Based on the bounding boxes of personnel and components in the same frame image, the handling status of each personnel is obtained by spatial overlap calculation. When the overlap between the personnel and the bounding box of at least one component exceeds a second threshold, the handling status is "yes" and otherwise "no".

[0046] S500: Perform a composite correlation determination on the handling status and tool usage status of each person to obtain all violating personnel. Violating personnel are characterized as those whose handling status is "yes" and tool usage status is "no".

[0047] In this step, the composite association determination refers to simultaneously judging the handling status and tool usage status of each person, filtering out the persons whose handling status is "yes" and tool usage status is "no", that is, selecting the persons who are handling parts but not using authorized tools, and then marking the filtered persons as non-compliant persons.

[0048] If the handling status is "Yes" and the tool usage status is "Yes", it is considered compliant tool handling. If the handling status is "No", then the person did not handle any parts, and therefore it is not a violation.

[0049] S600: Based on time series analysis, the displacement vector corresponding to each qualified violator is analyzed. The displacement vector is projected onto the reference vector to analyze its validity. The violator corresponding to the valid displacement vector is marked as a valid violator.

[0050] The purpose of this step is to filter out invalid motions perpendicular to the reference vector, thereby reducing unnecessary computation.

[0051] The specific methods for determining whether a person who has violated the rules meets the requirements include: An independent time-series buffer is established for each violator, and the bounding box information and frame information corresponding to the violator are stored in the time-series buffer. The time-series buffer has multiple storage bits. Meeting the requirements means that the time-series buffer corresponding to the violator is filled.

[0052] The aforementioned time-series buffer is stored in frames. It can be configured to only store consecutive frames, or it can be configured to allow adjacent frames to be stored within a preset time interval. Under required conditions, if the time-series buffer is not full, the user's buffer is cleared or updated using a sliding window method to prepare for the next round of computation.

[0053] In this embodiment, the bounding box information includes the length and width data of the bounding box and its position coordinates. The position coordinates can be the midpoint of the bounding box, one of the endpoints of the bounding box, or the midpoint of the bottom edge or the top edge. Preferably, the midpoint of the bottom edge is used as the position coordinates.

[0054] In this embodiment, the setting of a time-series buffer can be used to further filter out violators.

[0055] Furthermore, the specific methods for obtaining the displacement vector in this step include: Extract the bounding box information of the first and last two storage bits in the time-series buffer; Calculate the displacement vector based on the position coordinates in the two bounding box information.

[0056] S700: Calculate the displacement of valid violators and determine the abnormal handling status based on the displacement calculation results.

[0057] In this step, the motion on the two-dimensional coordinate system is converted into the actual physical movement distance, and then the abnormal handling state is judged. When the displacement calculation result exceeds the threshold, it is judged as an abnormal handling state, and based on the judgment result, an alarm is triggered, the event is recorded, or other downstream tasks are executed.

[0058] In one embodiment, step S700, which involves calculating the displacement of valid violators and determining the abnormal handling status based on the displacement calculation results, specifically includes: Based on the bounding box information in the current time-series buffer, calculate the physical distance increment of the corresponding violator in the time-series buffer. Obtain the total cumulative displacement variable corresponding to the previous time-series buffer. If there is no previous time-series buffer, or the time interval between the previous time-series buffer and the current time-series buffer is greater than the time threshold, use the physical distance increment corresponding to the current time-series buffer as the total cumulative displacement variable; otherwise, add the physical distance increment corresponding to the current time-series buffer to the total cumulative displacement variable. When the total cumulative displacement variable exceeds the preset displacement threshold, the corresponding person is determined to be in an abnormal handling state.

[0059] In this embodiment, the abnormal transport status is determined by the total cumulative displacement. This avoids the situation where the physical distance increment within a time buffer does not meet the requirements, which would cause the abnormal transport to fail to be successfully identified.

[0060] Once the physical distance increment is calculated in the current time-series buffer, the system automatically clears the time-series location buffer for that person or updates it using a sliding window method.

[0061] In another embodiment, the step of calculating the physical distance increment of the corresponding violator in the current time-series buffer based on the bounding box information specifically includes: Retrieve all bounding box information in the time-series buffer; Based on the bounding box information, the displacement coefficients in the two-dimensional image coordinate system are obtained; Based on the location coordinates of the bounding box information, calculate the effective pixel displacement components of the violator; The physical distance increment is calculated based on the product of the effective pixel displacement component and the displacement coefficient.

[0062] In this embodiment, the step of obtaining the displacement coefficient in the two-dimensional image coordinate system based on the bounding box information specifically includes: Obtain the preset reference size factor; The average bounding box width is obtained by averaging the bounding box widths in the bounding box information. The displacement coefficient is calculated based on the ratio of the reference size factor to the average bounding box width.

[0063] In the above steps, the reference size factor is related to the physical dimensions of the target object. For example, for a typical pedestrian, the average shoulder width can be assumed to be 0.5-0.8 meters. The reference size factor can be a uniform value, or it can be a preset independent value for each person based on a unique ID.

[0064] By calculating with the average bounding box width, the reference size coefficient can be dynamically adjusted based on the depth estimation of the monocular vision sensor, thereby improving the final calculation accuracy.

[0065] Furthermore, in another embodiment, the step of calculating the effective pixel displacement components of the violator based on the position coordinates of the bounding box information specifically includes: Calculate the pixel displacement vector between all position coordinates using a pairwise combination method; Based on projection operations, obtain the pixel displacement components of the pixel displacement vector in the direction of the reference vector; The maximum value of the pixel shift component is taken as the effective pixel shift component.

[0066] The embodiments described in this specific implementation are preferred embodiments of this application and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A method for identifying abnormal handling of factory parts, characterized in that, include: Based on a two-dimensional image coordinate system, a two-dimensional reference vector is preset, which represents the physical direction or path of interest and the displacement to be measured. Acquire the video stream and read it frame by frame at a preset frame rate to obtain frame images; Based on the preset target detection model, obtain the bounding boxes and unique IDs of all personnel, parts and authorized tools in the frame image; Based on the bounding boxes and unique IDs of personnel, parts, and authorized tools in the same frame image, determine the handling status and tool usage status of each personnel; A composite correlation determination is performed on the handling status and tool usage status of each person to obtain all the non-compliant persons. The non-compliant persons are characterized as those whose handling status is yes and tool usage status is no. An independent time-series buffer is established for each violator, and the bounding box information and frame information corresponding to the violator are stored in the time-series buffer. The time-series buffer has multiple storage slots; compliance means that the time-series buffer corresponding to the violator is filled. Extract the bounding box information of the first and last two storage bits in the time buffer, and calculate the displacement vector based on the position coordinates in the two bounding box information; The displacement vector is projected onto the reference vector to analyze its validity, and the violators corresponding to the valid displacement vectors are marked as valid violators. Displacement calculations are performed on valid violators, and abnormal handling status is determined based on the displacement calculation results; The step of calculating the displacement of valid violators and determining the abnormal handling status based on the displacement calculation results specifically includes: Based on the bounding box information in the current time-series buffer, calculate the physical distance increment of the corresponding violator in the time-series buffer. Obtain the total cumulative displacement variable corresponding to the previous time-series buffer. If there is no previous time-series buffer, or the time interval between the previous time-series buffer and the current time-series buffer is greater than the time threshold, use the physical distance increment corresponding to the current time-series buffer as the total cumulative displacement variable; otherwise, add the physical distance increment corresponding to the current time-series buffer to the total cumulative displacement variable. When the total cumulative displacement variable exceeds the preset displacement threshold, the corresponding person is determined to be in an abnormal handling state.

2. The method for identifying abnormal handling of factory parts according to claim 1, characterized in that: The target detection model includes a detection model and a tracking model. The detection model is used to process the input frame image and output the bounding boxes of all personnel, parts and authorized tools in the image. The tracking model is used to identify the input bounding boxes of personnel and parts and assign or update their unique IDs.

3. The method for identifying abnormal handling of factory parts according to claim 1, characterized in that, The steps for determining the handling status and tool usage status of each person based on the bounding boxes and unique IDs of personnel, parts, and authorized tools in the same frame image include: Based on the bounding boxes of personnel and authorized tools in the same frame image, the tool usage status of each personnel is obtained by calculating the spatial overlap or proximity. When the overlap or proximity of the bounding box of a personnel and at least one authorized tool exceeds a first threshold, the tool usage status is "yes" and otherwise "no". Based on the bounding boxes of personnel and components in the same frame image, the handling status of each personnel is obtained by spatial overlap calculation. When the overlap between the personnel and the bounding box of at least one component exceeds a second threshold, the handling status is "yes" and otherwise "no".

4. The method for identifying abnormal handling of factory parts according to claim 1, characterized in that, The step of calculating the physical distance increment of the corresponding violator in the current time-series buffer based on the bounding box information in the current time-series buffer specifically includes: Retrieve all bounding box information in the time-series buffer; Based on the bounding box information, the displacement coefficients in the two-dimensional image coordinate system are obtained; Based on the location coordinates of the bounding box information, calculate the effective pixel displacement components of the violator; The physical distance increment is calculated based on the product of the effective pixel displacement component and the displacement coefficient.

5. The method for identifying abnormal handling of factory parts according to claim 4, characterized in that, The step of obtaining the displacement coefficients in the two-dimensional image coordinate system based on the bounding box information specifically includes: Obtain the preset reference size factor; The average bounding box width is obtained by averaging the bounding box widths in the bounding box information. The displacement coefficient is calculated based on the ratio of the reference size factor to the average bounding box width.

6. The method for identifying abnormal handling of factory parts according to claim 4, characterized in that, The step of calculating the effective pixel displacement components of the violator based on the position coordinates of the bounding box information specifically includes: Calculate the pixel displacement vector between all position coordinates using a pairwise combination method; Based on projection operations, obtain the pixel displacement components of the pixel displacement vector in the direction of the reference vector; The maximum value of the pixel displacement component is taken as the effective pixel displacement component.

7. A system for identifying abnormal handling of factory parts, applied to the method for identifying abnormal handling of factory parts as described in any one of claims 1-6, characterized in that, include: The reference vector preset module is used to preset a two-dimensional reference vector based on a two-dimensional image coordinate system. The reference vector is used to characterize the physical direction or path of the object of interest and the displacement to be measured. The video processing module is used to acquire the video stream and read it frame by frame according to a preset frame rate to obtain frame images; The target detection module is used to detect and obtain the bounding boxes of all personnel, parts and authorized tools in the acquired frame images based on a preset target detection model. The handling status composite association module is used to determine the handling status and tool usage status of each person based on the bounding boxes of personnel, parts and authorized tools in the same frame image; and to perform composite association judgment on the handling status and tool usage status of each person to identify all non-compliant personnel, wherein the non-compliant personnel are those whose handling status is yes and tool usage status is no. The transport distance calculation module analyzes the displacement vector of each qualified violator based on time-series analysis. It projects the displacement vector onto a reference vector to analyze its validity, marking violators with valid displacement vectors as qualified violators. Displacement calculation is then performed on qualified violators. Specifically, an independent time-series buffer is established for each violator, storing their corresponding bounding box information and frame information. The time-series buffer has multiple storage slots; qualification means the buffer corresponding to the violator is full. Bounding box information from the first and last storage slots of the time-series buffer is extracted, and the displacement vector is calculated based on the position coordinates in the two bounding box information. The displacement calculation for qualified violators includes: calculating the physical distance increment of the corresponding violator in the current time-series buffer based on the bounding box information; obtaining the total cumulative displacement variable corresponding to the previous time-series buffer; if no previous time-series buffer exists, or the time interval between the previous and current time-series buffers is greater than a time threshold, the physical distance increment corresponding to the current time-series buffer is used as the total cumulative displacement variable; otherwise, the physical distance increment corresponding to the current time-series buffer is added to the total cumulative displacement variable. The result output module is used to determine whether the handling is in an abnormal state based on the displacement calculation results; when the total cumulative displacement variable is greater than the preset displacement threshold, the corresponding person in violation is determined to be in an abnormal handling state.

Images