Method and system for recognizing personnel behavior in a power tower production workshop

Abstract

The application discloses a power iron tower production workshop personnel behavior recognition method and system, the method comprises the following steps: collecting video data of different areas in the power iron tower production workshop based on each video acquisition device, and analyzing each video data based on a preset visual model to obtain an analysis result; determining whether there is an abnormal result in each analysis result, if there is an abnormal result, determining an abnormal area and an abnormal behavior personnel corresponding to the abnormal result; determining a reminding device corresponding to the abnormal area, and outputting early warning reminding information to the abnormal behavior personnel based on the reminding device. The application sets video acquisition devices and reminding devices in different areas of the power iron tower production workshop, once it is analyzed by the preset visual model that the personnel in a certain area have behavior abnormalities, the reminding device in the area is controlled to start reminding the abnormal behavior personnel, realizing real-time supervision of the safety of the personnel behavior in the power iron tower production workshop.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method and system for recognizing personnel behavior in a power tower production workshop. Background Technology

[0002] Power transmission towers are an important component of power transmission lines in power systems, manufactured using angle steel in production workshops. The production of power transmission towers is a typical example of heavy equipment manufacturing, encompassing multiple processes such as raw material loading and unloading, cutting, hole making, welding, galvanizing, and trial assembly. Each process carries inherent risks, making safety management of workshop personnel particularly crucial.

[0003] Currently, monitoring systems are typically installed in power tower production workshops to manage safety-related behaviors such as whether personnel are wearing safety helmets and reflective clothing, and whether their operations are in accordance with regulations. However, these monitoring systems record and store evidence in multiple dimensions, rather than providing real-time supervision, and therefore cannot fundamentally meet the requirements of production safety, easily leading to safety accidents. Summary of the Invention

[0004] The main objective of this invention is to provide a method and system for identifying personnel behavior in a power tower production workshop, aiming to solve the technical problem of how to monitor the safety of personnel behavior in a power tower production workshop in real time and fundamentally meet the requirements of production safety.

[0005] To achieve the above objectives, the present invention provides a method for identifying personnel behavior in a power tower production workshop, the method comprising: Based on the video acquisition devices, video data of different areas in the power tower production workshop are collected, and the video data is analyzed based on a preset visual model to obtain analysis results. Determine whether there are any abnormal results in each of the analysis results. If there are abnormal results, identify the abnormal area and the person with abnormal behavior corresponding to the abnormal result. Identify the alert device corresponding to the abnormal area, and output early warning alert information to the person exhibiting abnormal behavior based on the alert device.

[0006] Preferably, the step of outputting a warning reminder to the person exhibiting abnormal behavior based on the reminder device includes the following afterward: Locate the personnel identifier corresponding to the abnormal behavior in the preset storage unit, and add the abnormal result to the preset storage unit based on the personnel identifier; Locate the personnel icon corresponding to the personnel identifier and the area icon corresponding to the abnormal area in the preset 3D dashboard, and update the stored abnormal results to the dashboard display information corresponding to the personnel icon and the area icon respectively.

[0007] Preferably, the step of updating the stored abnormal results to the dashboard display information corresponding to the personnel icon and the area icon includes: When a command to view the personnel icon is received, the dashboard display information corresponding to the personnel icon is obtained and displayed. The dashboard display information corresponding to the personnel icon includes at least the historical abnormal results, skills and qualifications, and safety points corresponding to the abnormal behavior personnel. When a viewing trigger command for the area icon is received, the dashboard display information corresponding to the area icon is obtained and displayed. The dashboard display information corresponding to the area icon includes at least the risk level of the abnormal area and the temperature and humidity data of each device in the abnormal area. The risk level is generated by the number of abnormal results contained in the abnormal area.

[0008] Preferably, the step of analyzing each of the video data based on a preset visual model to obtain the analysis results includes the following steps before proceeding: Generate positive sample pairs and negative sample pairs, and train a preset initial model based on the positive sample pairs and negative sample pairs; After the preset initial model has been trained for a preset duration, a loss function value for the preset initial model is generated based on the positive sample pairs, and a visual base model is generated based on the loss function value. Based on the images of personnel behavior corresponding to the power tower production workshop, the visual basic model is adjusted and trained, and based on the adjustment and training, the visual basic model is generated into the preset visual model.

[0009] Preferably, the step of generating positive sample pairs and negative sample pairs includes: Multiple different positive sample images are obtained, and for each positive sample image, multiple new images are generated based on data augmentation. The positive sample images and the multiple new images together constitute a positive sample pair. After generating a positive sample pair for each positive sample image, a sample image is randomly extracted from each positive sample pair, and the extracted sample images are combined to form the negative sample pair.

[0010] Preferably, the step of generating a visual base model based on the loss function value includes: Determine whether the loss function value is less than a preset threshold. If it is less than the preset threshold, then generate the preset initial model as the visual base model. If the loss function value is greater than or equal to a preset threshold, the model parameters of the preset initial model are updated, and the preset initial model is trained based on the positive sample pairs and the negative sample pairs for the updated preset initial model.

[0011] Preferably, the step of outputting early warning reminder information to the person exhibiting abnormal behavior based on the reminder device includes: Determine the anomaly type of the abnormal result and find the warning data corresponding to the anomaly type. The anomaly type includes at least the following: not wearing a safety helmet, not wearing a safety helmet correctly, not wearing a reflective vest, not wearing gloves, being in a prohibited area of ​​the workshop, not wearing a safety helmet and being in a risk area of ​​the workshop, and operation that does not comply with the specifications. The abnormal results and early warning data are used to generate early warning reminder information, which is then output to the individuals exhibiting abnormal behavior based on the reminder device.

[0012] Preferably, each area of ​​the power tower production workshop is equipped with an infrared acquisition device, and the infrared acquisition device corresponds one-to-one with the video acquisition device; The step of collecting video data from different areas within the power tower production workshop based on the video acquisition devices includes: For each of the aforementioned areas, when an infrared acquisition device within the area detects the presence of a person within the area, a time start marker is generated; When an infrared sensor detects that a person has left the area, a time-end marker is generated. The system receives raw video data collected by a video acquisition device within the area, and determines the start point of the video data from the raw video data based on the time start identifier, and determines the end point of the video data based on the time end identifier.

[0013] Furthermore, to achieve the above objectives, the present invention also provides a personnel behavior recognition system for a power tower production workshop. The power tower production workshop is equipped with multiple video acquisition devices. The personnel behavior recognition system for the power tower production workshop includes a storage unit, a processor, a communication bus, and a control program stored in the storage unit. The communication bus is used to enable communication between the processor and the memory. The processor is used to execute the control program to implement the steps of the power tower production workshop personnel behavior recognition method as described above.

[0014] Furthermore, to achieve the above objectives, the present invention also provides a medium, which is a readable storage medium storing a control program. When the control program is executed by a processor, it implements the steps of the power tower production workshop personnel behavior recognition method described above.

[0015] This invention discloses a method and system for identifying personnel behavior in a power tower production workshop. The workshop is equipped with multiple video acquisition devices that collect video data from different areas. These data are then analyzed using a pre-trained visual model to obtain analysis results. The system determines if any abnormal results exist. If abnormal results are found, the corresponding abnormal area and personnel exhibiting abnormal behavior are identified. A corresponding alert device is then identified for each abnormal area, and warning messages are sent to the individuals exhibiting abnormal behavior. By installing video acquisition and alert devices in different areas of the power tower production workshop, and activating the alert device in that area to alert the individuals once the pre-trained visual model detects abnormal behavior, real-time monitoring of personnel behavior in the power tower production workshop is achieved, fundamentally meeting production safety requirements. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the first embodiment of the method for identifying personnel behavior in a power tower production workshop according to the present invention. Figure 2 This is a flowchart illustrating the second embodiment of the method for identifying personnel behavior in a power tower production workshop according to the present invention. Figure 3 This is a flowchart illustrating the third embodiment of the method for identifying personnel behavior in a power tower production workshop according to the present invention. Figure 4 This is a schematic diagram of the hardware operating environment involved in an embodiment of the personnel behavior recognition system for power tower production workshops of the present invention.

[0017] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0018] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0019] This invention provides a method for identifying personnel behavior in a power tower manufacturing workshop. Please refer to [the relevant documentation]. Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the method for identifying personnel behavior in a power tower production workshop according to the present invention.

[0020] This invention provides an embodiment of a method for recognizing personnel behavior in a power tower production workshop. It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order. Specifically, the method for recognizing personnel behavior in a power tower production workshop in this embodiment includes:

[0021] Step S10: Collect video data from different areas within the power tower production workshop using the video acquisition devices, and analyze the video data based on a preset visual model to obtain analysis results.

[0022] This embodiment of the personnel behavior recognition method in a power tower production workshop is applied to the system server of the power tower production workshop. The power tower production workshop is divided into multiple different areas, each equipped with a video acquisition device. Each video acquisition device is connected to the system server to analyze the personnel behavior captured by the video acquisition devices within their respective areas, determining whether any safety hazards exist. Each video acquisition device can collect video data from its own area in real time and upload the collected video data to the system server for analysis and processing.

[0023] Furthermore, to avoid excessive video data processing by the system server, it can be configured to collect video data only when personnel are present in the area, or to collect video data in real-time but only process video data during the time periods when personnel are present. Considering the real-time monitoring needs of the power tower production workshop, this embodiment preferably collects video data in real-time but only processes video data during the time periods when personnel are present. Specifically, corresponding to each area of ​​the power tower production workshop, in addition to video acquisition devices, each area is also equipped with corresponding infrared acquisition devices. The steps of collecting video data from different areas within the power tower production workshop based on each of the aforementioned video acquisition devices include:

[0024] Step S11: For each area, when an infrared acquisition device in the area detects the presence of a person in the area, a time start identifier is generated. Step S12: When the infrared acquisition device in the area detects that the person has left the area, a time end marker is generated; Step S13: Receive the raw video data collected by the video acquisition device in the area, and determine the start point of the video data from the raw video data based on the time start identifier, and determine the end point of the video data based on the time end identifier.

[0025] Furthermore, for each area, once an infrared sensor detects someone entering the area, the system server automatically generates a time start identifier, which can be generated by combining the current time with the area number. Similarly, when an infrared sensor detects someone leaving the area, the system server automatically generates a time end identifier, in the same way as the time start identifier. The person entering the area is the first person to enter, and the person leaving the area is the last person to leave, ensuring that personnel are monitored and identified whenever they are present within the area.

[0026] Furthermore, to monitor the behavior of personnel within the power tower production workshop in real time, video capture devices within the area transmit the raw video data to the system server. This raw video data may contain footage that does not include personnel. The backend server then identifies the raw video data based on a generated time start marker, determining the time point corresponding to that marker. This time point is the starting point for the video data showing personnel within the area.

[0027] Furthermore, the system server is pre-trained with a preset visual model for video analysis. Starting from this point, the system server segments the video data at specific time intervals (e.g., 10, 20, or 30 seconds). The preset visual model then analyzes these segments to determine if any personnel in the segment exhibit any safety hazards, obtaining the corresponding analysis results. These safety hazards include, but are not limited to, not wearing a safety helmet, not wearing a helmet correctly, not wearing reflective clothing, not wearing gloves, and improper operation.

[0028] Furthermore, while segmenting the video data, the system server also checks whether the time corresponding to the segmented video data is the time point corresponding to the time end marker. If it is, the time point is determined as the end point of the video data, and segmentation stops. In this way, it ensures that the preset visual model only processes video data containing people, improves data processing efficiency, and achieves efficient monitoring of personnel behavior in the power tower production workshop.

[0029] Step S20: Determine whether there are any abnormal results in each of the analysis results. If there are abnormal results, determine the abnormal area and abnormal behavior personnel corresponding to the abnormal results.

[0030] Furthermore, a pre-defined visual model analyzes video data from each area, generating analysis results that fall into at least two categories: one for individuals exhibiting abnormal behavior within the area, and the other for individuals not exhibiting abnormal behavior. These two categories are distinguished by broad category labels, with the former labeled as abnormal results and the latter as normal results. Additionally, abnormal results are further differentiated by subcategories based on the specific abnormal behavior and individuals, such as individuals not wearing helmets or reflective vests.

[0031] Furthermore, the system server uses the major category identifier to determine whether there are any abnormal results in each analysis result, i.e., whether there are analysis results for individuals with abnormal behavior. If such analysis results exist, the system searches for the video data from which the analysis result originated, and then uses the region identifier carried by the video data to determine the region where the abnormal result is located. This region is the abnormal region corresponding to the abnormal result. At the same time, the system identifies the individual with abnormal behavior based on the minor category identifier; this individual is the one with abnormal behavior corresponding to the abnormal result.

[0032] Step S30: Determine the alerting device corresponding to the abnormal area, and output early warning alert information to the person exhibiting abnormal behavior based on the alerting device.

[0033] Furthermore, each area of ​​the power tower production workshop is pre-equipped with warning devices, such as broadcasts and audible and visual alarms. After identifying abnormal areas and individuals exhibiting abnormal behavior, the warning devices within those areas are located, and warning messages are sent to the individuals exhibiting abnormal behavior through these devices, prompting them to correct their behavior promptly. In addition, considering the diverse range of abnormal behaviors that could lead to safety hazards in the power tower production workshop, different types of warning messages can be sent based on the specific type of abnormal behavior for more precise alerts. Specifically, the step of sending warning messages to individuals exhibiting abnormal behavior based on the warning devices includes:

[0034] Step S31: Determine the anomaly type of the abnormal result and find the warning data corresponding to the anomaly type. The anomaly type includes at least the following: not wearing a safety helmet, not wearing a safety helmet correctly, not wearing a reflective vest, not wearing gloves, being in a prohibited area of ​​the workshop, not wearing a safety helmet and being in a risk area of ​​the workshop, and operation that does not comply with the specifications. Step S32: Generate early warning information from the abnormal results and early warning data, and output the early warning information to the person exhibiting abnormal behavior based on the early warning device.

[0035] Furthermore, the pre-defined visual model, in the analysis results generated from analyzing video data, includes information indicating the type of abnormal behavior for abnormal results. The system server uses this information to determine the type of abnormal result. This type of abnormality includes, but is not limited to, not wearing a safety helmet, wearing a safety helmet incorrectly, not wearing reflective clothing, not wearing gloves, being in prohibited areas of the workshop (e.g., leaning against an operating table or a vehicle), not wearing a safety helmet and being in a high-risk area of ​​the workshop (e.g., not wearing a safety helmet and being in a hoisting operation area, not wearing a safety helmet correctly and being in a hoisting operation area), and non-standard operations (e.g., non-standard hoisting operations, non-standard cutting operations), and other behaviors that are likely to cause safety accidents in the power tower production workshop.

[0036] Furthermore, the system server pre-sets corresponding early warning data for various anomaly types. For example, the early warning data for personnel not wearing safety helmets is to activate a broadcast announcement, and the early warning data for personnel being in prohibited areas of the workshop is to activate an audible and visual alarm. After determining the anomaly type of the abnormal result, the system searches for the corresponding early warning data based on the correspondence between the anomaly type and the early warning data. The found early warning data and the anomaly result are then combined to generate an early warning reminder message. This message can be combined with information about the abnormal behavior and the personnel involved in the abnormal result to form an early warning reminder message, which is then output to the personnel exhibiting the abnormal behavior via a reminder device. This serves to remind specific personnel to correct specific abnormal behaviors, such as broadcasting the personnel's names and the fact that they are not wearing safety helmets to remind them to wear them.

[0037] This implementation of a personnel behavior recognition method in a power tower production workshop involves multiple video acquisition devices within the workshop. These devices collect video data from different areas of the workshop, which is then analyzed using a pre-trained visual model. The analysis results are used to determine if any anomalies exist. If anomalies are found, the corresponding abnormal areas and individuals exhibiting abnormal behavior are identified. Furthermore, the corresponding alert devices are identified, and warning messages are sent to the individuals exhibiting abnormal behavior. By deploying video acquisition and alert devices in different areas of the power tower production workshop, and activating the alert devices in those areas to warn individuals of abnormal behavior once the pre-trained visual model detects such behavior, real-time monitoring of personnel behavior in the power tower production workshop is achieved, fundamentally meeting production safety requirements.

[0038] Further, please refer to Figure 2Based on the first embodiment of the method for identifying personnel behavior in a power tower production workshop according to the present invention, a second embodiment of the method for identifying personnel behavior in a power tower production workshop according to the present invention is proposed.

[0039] The difference between the second embodiment of the method for identifying personnel behavior in a power tower production workshop and the first embodiment of the method for identifying personnel behavior in a power tower production workshop is that, after the step of outputting early warning reminder information to the personnel exhibiting abnormal behavior based on the reminder device, the method includes: Step S40: Locate the personnel identifier corresponding to the abnormal behavior in the preset storage unit, and add the abnormal result to the preset storage unit based on the personnel identifier; Step S50: Locate the personnel icon corresponding to the personnel identifier and the area icon corresponding to the abnormal area in the preset 3D dashboard, and update the stored abnormal results to the dashboard display information corresponding to the personnel icon and the area icon respectively.

[0040] Furthermore, the power tower production workshop is equipped with pre-set 3D dashboards and pre-set storage units that communicate with the system server. The pre-set 3D dashboards display equipment operation data, on-site environmental data, and personnel behavior data from various areas of the power tower production workshop, achieving simultaneous display of virtual and real data. The pre-set storage units store various types of data related to the power tower production workshop, supporting the display of the pre-set 3D dashboards. Personnel information within the production workshop is stored in the pre-set storage units, and each person's information is distinguished by a unique personnel identifier, such as an ID card number or employee ID. For personnel exhibiting abnormal behavior, the corresponding personnel identifier is retrieved from the pre-set storage units, and the abnormal result is added to the data group corresponding to that personnel identifier in the pre-set storage units, stored along with other information about the person exhibiting abnormal behavior.

[0041] Furthermore, all information about each person and each area in the production workshop is displayed in a preset 3D dashboard to facilitate refined management of personnel and areas within the workshop. For identified individuals and areas exhibiting abnormal behavior, the system searches the preset 3D dashboard for the corresponding personnel icon and the corresponding area icon. The stored anomaly results are then updated in the dashboard display information corresponding to the personnel icon to allow for viewing of the individual's abnormal behavior. Simultaneously, the stored anomaly results are updated in the dashboard display information corresponding to the area icon to allow for viewing of any abnormal behavior occurring in that area. Both personnel and area icons are clickable for clearer viewing of various information. Specifically, the step of updating the stored anomaly results in the dashboard display information corresponding to the personnel and area icons includes:

[0042] Step S60: When a viewing trigger command for the personnel icon is received, the dashboard display information corresponding to the personnel icon is obtained and displayed. The dashboard display information corresponding to the personnel icon includes at least the historical abnormal results, skill qualifications, and safety points corresponding to the abnormal behavior personnel. Step S70: When a viewing trigger command for the area icon is received, the dashboard display information corresponding to the area icon is obtained and displayed. The dashboard display information corresponding to the area icon includes at least the risk level of the abnormal area and the temperature and humidity data of each device in the abnormal area. The risk level is generated by the number of abnormal results contained in the abnormal area.

[0043] Furthermore, when a request to view a personnel icon is received, the system server retrieves the corresponding dashboard display information from a preset storage unit and transmits the retrieved dashboard display information to the display interface corresponding to the personnel icon in the preset 3D dashboard for display. The displayed dashboard information includes at least the historical abnormal results, skills and qualifications, and safety points of the abnormal behavior personnel corresponding to the personnel icon, indicating the skills, safety awareness, and historical abnormal behaviors possessed by the abnormal behavior personnel.

[0044] Furthermore, when a request to view a region icon is received, the system server retrieves the corresponding dashboard display information from a preset storage unit and transmits the retrieved information to the display interface corresponding to the region icon in a preset 3D dashboard. This displayed information includes at least the risk level of the abnormal region corresponding to the region icon, and the temperature and humidity data of each device within that abnormal region. The risk level is determined by the number of times abnormal behavior occurs in each abnormal region. The more times abnormal behavior occurs, the more abnormal results are generated by the preset visual model, resulting in a higher risk level and indicating a higher risk for the abnormal region; conversely, a lower risk level indicates a lower risk for the abnormal region. The temperature and humidity data of each device within the abnormal region are obtained by pre-setting temperature and humidity detection devices for each device in the abnormal region. These devices detect the temperature and humidity of each device and store the values ​​in a preset storage unit, which are then displayed on the preset 3D dashboard. This allows managers to easily check whether the temperature and humidity values ​​of each device are within a suitable range by clicking on the region icon.

[0045] This embodiment displays information about personnel and areas in the power tower production workshop through a pre-set 3D dashboard. It can conveniently display abnormal behavior of personnel and risk levels of each area, which is conducive to achieving precise control over personnel and areas in the power tower production workshop.

[0046] Further, please refer to Figure 3Based on the first and second embodiments of the method for identifying personnel behavior in power tower production workshops of the present invention, a third embodiment of the method for identifying personnel behavior in power tower production workshops of the present invention is proposed.

[0047] The difference between the third embodiment of the method for recognizing personnel behavior in a power tower production workshop and the first and second embodiments of the method for recognizing personnel behavior in a power tower production workshop is that, before the step of analyzing the video data based on a preset visual model to obtain the analysis results, the following steps are included: Step S80: Generate positive sample pairs and negative sample pairs, and train the preset initial model based on the positive sample pairs and negative sample pairs; Furthermore, for the preset visual model, this embodiment employs self-supervised training to avoid labeling large amounts of training data, improve training efficiency, and facilitate the real-time deployment of the preset visual model. Specifically, positive and negative sample pairs are generated by a generator, and these generated positive and negative sample pairs are used to train the preset initial model. Here, a positive sample pair is a different version of an image, such as a new image formed by rotating the same image at different angles, while a negative sample pair is a completely different image. By training the preset initial model with these positive and negative sample pairs, the preset initial model can distinguish between different versions of the same image and different images. Variants of the same image are close to each other in the model feature space, while variants of different images are far apart. Specifically, the steps for generating positive and negative sample pairs include:

[0048] Step S81: Obtain multiple different positive sample images, and for each positive sample image, generate multiple new images based on data augmentation. The positive sample images and the multiple new images together constitute a positive sample pair. Step S82: After generating a positive sample pair for each positive sample image, randomly extract a sample image from each positive sample pair and combine the extracted sample images to form the negative sample pair.

[0049] Furthermore, multiple different images are collected as positive sample images. For each positive sample image, multiple new images are generated through data augmentation. Data augmentation can be any one of Gaussian blur, color dithering, or random cropping, or a combination of these methods. Gaussian augmentation uses a distance-weighted average of the colors of pixels surrounding the original image, and then replaces the original color of each pixel to obtain a new image. Specifically, Gaussian augmentation can be represented by the formula (1).

[0050] (1); Where P(x, y) represents the pixel value of the pixel at pixel coordinates (x, y) after replacement, k represents the radius parameter for pixel calculation, i and j represent the offsets in the row and column directions of the pixel coordinates, respectively, q(x, y) represents the pixel value of the pixel after offset (i, j) from pixel coordinates (x, y), and G[i+k+1, j+k+1] represents the Gaussian kernel weight matrix pre-generated by the two-dimensional Gaussian function.

[0051] Furthermore, color dithering can be used to add random noise to the color values ​​of the original image, disrupting the original smooth gradients to create a new image. Alternatively, random cropping can be used to randomly select a sub-region from the original image according to preset rules to create a new image. Then, for each positive sample image, it is combined with the generated new image to form a positive sample pair. After each positive sample pair has been data-enhanced to generate its own positive sample pair, a sample image is randomly selected from each positive sample pair, with each selected sample image being different from the others, and these are combined to form a negative sample pair.

[0052] Step S90: After training the preset initial model for a preset duration, generate the loss function value of the preset initial model based on the positive sample pairs, and generate the visual basic model based on the loss function value. Furthermore, during the training of the preset initial model using positive and negative sample pairs, the training time is recorded. Once the training time reaches the preset time, the loss function value of the preset initial model is generated using positive sample pairs. Then, based on the loss magnitude represented by the loss function value, the preset initial model is generated as the visual base model. Specifically, the loss function value can be generated using the following formula (2).

[0053] (2); Where Zu and Zv represent the feature vectors of the positive sample pairs after being encoded by the encoder in the preset initial model, sim(Zu, Zv) represents the cosine similarity, exp represents the exponential function, T represents the temperature coefficient, which takes a value between 0.5 and 1, and N represents the number of positive sample pairs.

[0054] Furthermore, the step of generating the visual base model based on the loss function value includes: Step S91: Determine whether the loss function value is less than a preset threshold. If it is less than the preset threshold, generate the preset initial model as the visual base model. Step S92: If the loss function value is greater than or equal to a preset threshold, the model parameters of the preset initial model are updated, and the preset initial model is trained based on the positive sample pairs and the negative sample pairs for the updated preset initial model.

[0055] Furthermore, a preset threshold representing the magnitude of the loss is pre-set. The calculated loss function value is compared with this preset threshold to determine whether the loss function value is less than the preset threshold. If it is less, it indicates that the loss is small, and the preset initial model is generated as the visual base model. Conversely, if the comparison determines that the loss function value is greater than or equal to the preset threshold, the model parameters of the preset initial model are updated according to the preset update formula. The updated preset initial model is then trained again using positive and negative sample pairs to generate new loss function values ​​for comparison until the generated loss function value is less than the preset threshold. The specific update formula used to update the model parameters of the preset initial model can be found in the following formula (3).

[0056] (3); Where w represents the updated model parameters, w0 represents the original model parameters, h represents the model learning rate, mt represents the bias correction of the first-order momentum of the model, Vt represents the bias correction of the second-order momentum of the model, e represents the model numerical stability term, and r represents the model weight decay coefficient.

[0057] Step S100: Based on the personnel behavior images corresponding to the power tower production workshop, adjust and train the visual basic model, and generate the preset visual model based on the adjusted and trained visual basic model.

[0058] Understandably, the visual baseline model generated through training using positive and negative sample pairs can distinguish different versions of the same image and different images in general, but it fails to accurately identify personnel behavior scenarios in power tower production workshops. Therefore, the visual baseline model needs to be adjusted using personnel behavior images related to power tower production workshops. Specifically, personnel behavior images corresponding to power tower production workshops need to be acquired. These images should include both positive and negative images of each type of abnormal behavior—an abnormal image of each type of abnormal behavior and a normal image of the corresponding normal behavior. For example, an image of someone not wearing a safety helmet correctly and an image of someone wearing a safety helmet correctly.

[0059] Furthermore, the visual baseline model is adjusted and trained using images of such personnel behavior, and the effectiveness of the adjustment and training can also be determined by the magnitude of the loss function value. The loss function value can be referred to in the following formula (4).

[0060] (4); Where L2 represents the loss function value, M represents the number of images of human behavior, yo represents the reference value corresponding to the o-th image of human behavior, and Po represents the prediction value generated by the visual base model for the o-th image of human behavior.

[0061] Furthermore, if the generated loss function value is small, it indicates that the adjusted and trained visual base model has high accuracy in recognizing personnel behavior in the power tower production workshop, thus the visual base model can be generated as the preset visual model. Conversely, if the generated loss function value after training is large, it indicates that the adjusted and trained visual base model has low accuracy in recognizing personnel behavior in the power tower production workshop, thus requiring continued adjustment and training of the visual base model until the generated loss function value is small, at which point the visual base model can be generated as the preset visual model.

[0062] In this embodiment, the preset visual model is trained through self-supervision and fine-tuning. Self-supervision avoids the need for manual standardization of a large amount of training data, while fine-tuning ensures the applicability of the model in the power tower production workshop scenario. This improves the model training efficiency while ensuring the model's accuracy, which is conducive to the rapid and effective deployment of the preset visual model.

[0063] Furthermore, this invention also provides a personnel behavior recognition system for a power tower production workshop. Please refer to... Figure 4 , Figure 4 This is a schematic diagram of the hardware operating environment of the equipment involved in the embodiment of the personnel behavior recognition system for power tower production workshop of the present invention.

[0064] like Figure 4 As shown, the personnel behavior recognition system for the power tower production workshop may include: a processor 1001, such as a CPU; a communication bus 1002; a user interface 1003; a network interface 1004; and a storage device 1005. The communication bus 1002 is used to establish communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The storage device 1005 may be a high-speed RAM or a stable, non-volatile memory, such as a disk drive. Optionally, the storage device 1005 may also be a storage device independent of the aforementioned processor 1001.

[0065] Those skilled in the art will understand that Figure 4 The hardware structure of the personnel behavior recognition system in the power tower production workshop shown in the figure does not constitute a limitation on the personnel behavior recognition system in the power tower production workshop. It may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0066] like Figure 4As shown, the storage device 1005, as a storage medium, may include an operating system, a network communication module, a user interface module, and a control program. The operating system is a program that manages and controls the personnel behavior recognition system and software resources in the power tower production workshop, and supports the operation of the network communication module, user interface module, control program, and other programs or software. The network communication module manages and controls the network interface 1004; the user interface module manages and controls the user interface 1003.

[0067] exist Figure 4 In the hardware structure of the personnel behavior recognition system for the power tower production workshop shown, the network interface 1004 is mainly used to connect to other system servers and communicate data with them; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with it; the processor 1001 can call the control program stored in the memory 1005 and perform the following operations: Based on the video acquisition devices, video data of different areas in the power tower production workshop are collected, and the video data is analyzed based on a preset visual model to obtain analysis results. Determine whether there are any abnormal results in each of the analysis results. If there are abnormal results, identify the abnormal area and the person with abnormal behavior corresponding to the abnormal result. Identify the alert device corresponding to the abnormal area, and output early warning alert information to the person exhibiting abnormal behavior based on the alert device.

[0068] Furthermore, after the step of outputting a warning reminder to the person exhibiting abnormal behavior based on the reminder device, the processor 1001 can call the control program stored in the storage 1005 and perform the following operations: Locate the personnel identifier corresponding to the abnormal behavior in the preset storage unit, and add the abnormal result to the preset storage unit based on the personnel identifier; Locate the personnel icon corresponding to the personnel identifier and the area icon corresponding to the abnormal area in the preset 3D dashboard, and update the stored abnormal results to the dashboard display information corresponding to the personnel icon and the area icon respectively.

[0069] Furthermore, after the step of updating the stored abnormal results to the dashboard display information corresponding to the personnel icon and the area icon respectively; the processor 1001 can call the control program stored in the storage 1005 and perform the following operations: When a command to view the personnel icon is received, the dashboard display information corresponding to the personnel icon is obtained and displayed. The dashboard display information corresponding to the personnel icon includes at least the historical abnormal results, skills and qualifications, and safety points corresponding to the abnormal behavior personnel. When a viewing trigger command for the area icon is received, the dashboard display information corresponding to the area icon is obtained and displayed. The dashboard display information corresponding to the area icon includes at least the risk level of the abnormal area and the temperature and humidity data of each device in the abnormal area. The risk level is generated by the number of abnormal results contained in the abnormal area.

[0070] Furthermore, before the step of analyzing each video data based on a preset visual model to obtain the analysis results, the processor 1001 can call the control program stored in the storage 1005 and perform the following operations: Generate positive sample pairs and negative sample pairs, and train a preset initial model based on the positive sample pairs and negative sample pairs; After the preset initial model has been trained for a preset duration, a loss function value for the preset initial model is generated based on the positive sample pairs, and a visual base model is generated based on the loss function value. Based on the images of personnel behavior corresponding to the power tower production workshop, the visual basic model is adjusted and trained, and based on the adjustment and training, the visual basic model is generated into the preset visual model.

[0071] Furthermore, the step of generating positive sample pairs and negative sample pairs includes: Multiple different positive sample images are obtained, and for each positive sample image, multiple new images are generated based on data augmentation. The positive sample images and the multiple new images together constitute a positive sample pair. After generating a positive sample pair for each positive sample image, a sample image is randomly extracted from each positive sample pair, and the extracted sample images are combined to form the negative sample pair.

[0072] Furthermore, the step of generating the visual base model based on the loss function value includes: Determine whether the loss function value is less than a preset threshold. If it is less than the preset threshold, then generate the preset initial model as the visual base model. If the loss function value is greater than or equal to a preset threshold, the model parameters of the preset initial model are updated, and the preset initial model is trained based on the positive sample pairs and the negative sample pairs for the updated preset initial model.

[0073] Furthermore, the step of outputting early warning reminder information to the person exhibiting abnormal behavior based on the reminder device includes: Determine the anomaly type of the abnormal result and find the warning data corresponding to the anomaly type. The anomaly type includes at least the following: not wearing a safety helmet, not wearing a safety helmet correctly, not wearing a reflective vest, not wearing gloves, being in a prohibited area of ​​the workshop, not wearing a safety helmet and being in a risk area of ​​the workshop, and operation that does not comply with the specifications. The abnormal results and early warning data are used to generate early warning reminder information, which is then output to the individuals exhibiting abnormal behavior based on the reminder device.

[0074] Furthermore, each area of ​​the power tower production workshop is equipped with an infrared acquisition device, and the infrared acquisition device corresponds one-to-one with the video acquisition device. The step of collecting video data from different areas within the power tower production workshop based on the video acquisition devices includes: For each of the aforementioned areas, when an infrared acquisition device within the area detects the presence of a person within the area, a time start marker is generated; When an infrared sensor detects that a person has left the area, a time-end marker is generated. The system receives raw video data collected by a video acquisition device within the area, and determines the start point of the video data from the raw video data based on the time start identifier, and determines the end point of the video data based on the time end identifier.

[0075] The specific implementation of the personnel behavior recognition system in the power tower production workshop of the present invention is basically the same as the various embodiments of the personnel behavior recognition method in the power tower production workshop described above, and will not be repeated here.

[0076] This invention also proposes a medium. The medium is a readable storage medium storing a control program. When executed by a processor, the control program implements the steps of the personnel behavior recognition method for power tower production workshops as described above.

[0077] The storage medium of the present invention can be a computer-readable storage medium, and its implementation is basically the same as the embodiments of the above-described method for identifying personnel behavior in power tower production workshops, and will not be described again here.

[0078] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many modifications under the guidance of the present invention without departing from the spirit and scope of the claims. All equivalent structural or procedural transformations made using the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are within the protection scope of the present invention.

Claims

1. A method for identifying personnel behavior in a power tower production workshop, wherein the power tower production workshop is equipped with multiple video acquisition devices, characterized in that, The method for identifying personnel behavior in the power tower production workshop includes: Based on the video acquisition devices, video data of different areas in the power tower production workshop are collected, and the video data is analyzed based on a preset visual model to obtain analysis results. Determine whether there are any abnormal results in each of the analysis results. If there are abnormal results, identify the abnormal area and the person with abnormal behavior corresponding to the abnormal result. Identify the alerting device corresponding to the abnormal area, and output early warning alert information to the person exhibiting abnormal behavior based on the alerting device.

2. The method for identifying personnel behavior in a power tower production workshop as described in claim 1, characterized in that, Following the step of outputting a warning alert to the person exhibiting abnormal behavior based on the alerting device, the following is included: Locate the personnel identifier corresponding to the abnormal behavior in the preset storage unit, and add the abnormal result to the preset storage unit based on the personnel identifier; Locate the personnel icon corresponding to the personnel identifier and the area icon corresponding to the abnormal area in the preset 3D dashboard, and update the stored abnormal results to the dashboard display information corresponding to the personnel icon and the area icon respectively.

3. The method for identifying personnel behavior in a power tower production workshop as described in claim 2, characterized in that, The step of updating the stored abnormal results to the dashboard display information corresponding to the personnel icon and the area icon includes: When a command to view the personnel icon is received, the dashboard display information corresponding to the personnel icon is obtained and displayed. The dashboard display information corresponding to the personnel icon includes at least the historical abnormal results, skills and qualifications, and safety points corresponding to the abnormal behavior personnel. When a viewing trigger command for the area icon is received, the dashboard display information corresponding to the area icon is obtained and displayed. The dashboard display information corresponding to the area icon includes at least the risk level of the abnormal area and the temperature and humidity data of each device in the abnormal area. The risk level is generated by the number of abnormal results contained in the abnormal area.

4. The method for identifying personnel behavior in a power tower production workshop as described in claim 1, characterized in that, Before the step of analyzing each video data based on a preset visual model to obtain the analysis results, the following steps are included: Generate positive sample pairs and negative sample pairs, and train a preset initial model based on the positive sample pairs and negative sample pairs; After the preset initial model has been trained for a preset duration, a loss function value for the preset initial model is generated based on the positive sample pairs, and a visual base model is generated based on the loss function value. Based on the images of personnel behavior corresponding to the power tower production workshop, the visual basic model is adjusted and trained, and based on the adjustment and training, the visual basic model is generated into the preset visual model.

5. The method for identifying personnel behavior in a power tower production workshop as described in claim 4, characterized in that, The steps for generating positive and negative sample pairs include: Multiple different positive sample images are obtained, and for each positive sample image, multiple new images are generated based on data augmentation. The positive sample images and the multiple new images together constitute a positive sample pair. After generating a positive sample pair for each positive sample image, a sample image is randomly extracted from each positive sample pair, and the extracted sample images are combined to form the negative sample pair.

6. The method for identifying personnel behavior in a power tower production workshop as described in claim 4, characterized in that, The step of generating a visual base model based on the loss function value includes: Determine whether the loss function value is less than a preset threshold. If it is less than the preset threshold, then generate the preset initial model as the visual base model. If the loss function value is greater than or equal to a preset threshold, the model parameters of the preset initial model are updated, and the preset initial model is trained based on the positive sample pairs and the negative sample pairs for the updated preset initial model.

7. The method for identifying personnel behavior in a power tower production workshop as described in any one of claims 1-6, characterized in that, The step of outputting early warning reminder information to the person exhibiting abnormal behavior based on the reminder device includes: Determine the anomaly type of the abnormal result and find the warning data corresponding to the anomaly type. The anomaly type includes at least the following: not wearing a safety helmet, not wearing a safety helmet correctly, not wearing a reflective vest, not wearing gloves, being in a prohibited area of ​​the workshop, not wearing a safety helmet and being in a risk area of ​​the workshop, and operation that does not comply with the specifications. The abnormal results and early warning data are generated into early warning reminder information, and the early warning reminder information is output to the person exhibiting abnormal behavior based on the reminder device.

8. The method for identifying personnel behavior in a power tower production workshop as described in any one of claims 1-6, characterized in that, Infrared acquisition devices are installed in each area of ​​the power tower production workshop, and each infrared acquisition device corresponds to a video acquisition device. The step of collecting video data from different areas within the power tower production workshop based on each of the video acquisition devices includes: For each of the aforementioned areas, when an infrared acquisition device within the area detects the presence of a person within the area, a time start marker is generated; When an infrared sensor detects that a person has left the area, a time-end marker is generated. The system receives raw video data collected by a video acquisition device within the area, and determines the start point of the video data from the raw video data based on the time start identifier, and determines the end point of the video data based on the time end identifier.

9. A personnel behavior recognition system for a power tower production workshop, wherein the power tower production workshop is equipped with multiple video acquisition devices, characterized in that, The personnel behavior recognition system in the power tower production workshop includes a storage device, a processor, a communication bus, and a control program stored in the storage device. The communication bus is used to enable communication between the processor and the memory. The processor is used to execute the control program to implement the steps of the method for identifying personnel behavior in a power tower production workshop as described in any one of claims 1-8.

10. A medium, characterized in that, The medium is a readable storage medium, on which a control program is stored. When the control program is executed by a processor, it implements the steps of the personnel behavior recognition method in the power tower production workshop as described in any one of claims 1-8.

Images