Safety detection method and device for electric power operation site, electronic equipment and storage medium

Abstract

The invention discloses a safety detection method and device for an electric power operation site, electronic equipment and a storage medium. The method comprises the following steps: acquiring video data of a power operation site, and performing image recognition on video frames at multiple moments in the video data to determine target objects in the video frames; joint detection is carried out on the target object to obtain joint information of a plurality of joint points of the target object, and the joint information at least comprises joint coordinates; according to the joint information, an object state corresponding to the target object is determined, under the condition that the object state is an abnormal state, alarm information corresponding to the abnormal state is generated and displayed, and the abnormal state comprises at least one of posture abnormity, action abnormity and environment interaction abnormity. The safety detection precision of an electric power operation site is effectively improved, the safety of a target object is guaranteed, and the operation efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a safety detection method, device, electronic equipment and storage medium for an electric power operation site. Background Art

[0002] There are usually certain dangers at the power operation site, and safe operating procedures must be strictly followed. Real-time detection of the status information of operators is one of the key measures to ensure safety.

[0003] In related technologies, safety detection mainly relies on on-site safety personnel or staff at the testing center to observe through video. However, this method is limited by the personnel's field of vision, concentration, and personal subjective judgment. In addition, in order to assist manual detection, video analysis technology based on predefined rules is also used, such as using human motion detection to trigger alarms. However, this method has limited ability to understand complex behaviors and is prone to false alarms or missed alarms. Summary of the invention

[0004] The present invention provides a safety detection method, device, electronic equipment and storage medium for an electric power operation site, so as to improve the safety detection accuracy and efficiency of the electric power operation site and reduce false alarms and missed alarms.

[0005] According to one aspect of the present invention, a safety detection method for an electric power operation site is provided, and the method may include:

[0006] Acquire video data of an electric power operation site, and perform image recognition on video frames at multiple moments in the video data to determine a target object in the video frame;

[0007] Performing joint detection on the target object to obtain joint information of multiple joint points of the target object, wherein the joint information at least includes joint coordinates;

[0008] The object state corresponding to the target object is determined according to the joint information. When the object state is an abnormal state, an alarm message corresponding to the abnormal state is generated and displayed, wherein the abnormal state includes at least one of abnormal posture, abnormal movement and abnormal environment interaction.

[0009] According to another aspect of the present invention, a safety detection device for an electric power operation site is provided, the method comprising:

[0010] A video acquisition module is used to acquire video data of the power operation site and perform image recognition on video frames at multiple moments in the video data to determine the target object in the video frame;

[0011] A joint detection module, used to perform joint detection on the target object to obtain joint information of multiple joint points of the target object, wherein the joint information at least includes joint coordinates;

[0012] An abnormal alarm module is used to determine the object state corresponding to the target object based on the joint information, and when the object state is an abnormal state, generate and display alarm information corresponding to the abnormal state, wherein the abnormal state includes at least one of abnormal posture, abnormal movement and abnormal environment interaction.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the safety detection method for the power operation site described in any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the safety detection method for a power operation site described in any embodiment of the present invention when executed.

[0018] The technical solution of the embodiment of the present invention is as follows: first, by acquiring video data of the power operation site, potential safety risks can be discovered in time, and image recognition is performed on video frames at multiple moments in the video data to determine the target object in the video frame, so as to detect whether the target object complies with safety regulations and ensure that the operation activities meet industry safety standards; then, joint detection is performed on the target object to obtain joint information of multiple joint points of the target object, wherein the joint information at least includes joint coordinates, so as to accurately capture the coordinates of each joint point of the target object, record and analyze the object state in detail, and discover potential safety risks in time; finally, the object state corresponding to the target object is determined according to the joint information, and when the object state is an abnormal state, an alarm information corresponding to the abnormal state is generated and displayed, wherein the abnormal state includes at least one of abnormal posture, abnormal movement and abnormal environment interaction, and by real-time detection of the posture, movement and interaction of the target object with the environment, possible safety hazards can be quickly identified and physical injuries caused by improper posture or dangerous movement can be prevented, and warnings can be issued in time to avoid accidents. The technical solution of the embodiment of the present invention uses real-time image recognition and joint detection technology to accurately determine whether the object state of the target object is abnormal. Once an abnormal situation is detected, an alarm is immediately issued, which solves the problem that the power operation site relies on manual detection and video analysis technology in the related technology, which is inefficient and prone to false alarms and missed alarms. It can effectively improve the safety detection accuracy of the power operation site, ensure the safety of the target object, and improve work efficiency.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 is a flow chart of a safety detection method for an electric power operation site provided according to Embodiment 1 of the present invention;

[0022] Figure 2 is a flow chart of a safety detection method for an electric power operation site provided according to a second embodiment of the present invention;

[0023] Figure 3is a schematic diagram of the structure of a safety detection device for an electric power operation site provided according to a third embodiment of the present invention;

[0024] Figure 4 It is a structural schematic diagram of an electronic device for implementing the safety detection method for an electric power operation site according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0028] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0029] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0030] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0031] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0032] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0033] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0034] Embodiment 1

[0035] Figure 1 A flowchart of a safety detection method for an electric power operation site is provided for the first embodiment of the present invention. This embodiment can be applicable to detecting the working status of a target object at an electric power operation site. The method can be performed by a safety detection device at the electric power operation site. The safety detection device at the electric power operation site can be implemented in the form of hardware and / or software, and optionally, can be implemented by an electronic device, which can be a mobile terminal, a PC or a server, etc.

[0036] like Figure 1 As shown, the method may specifically include:

[0037] S110, acquiring video data of a power operation site, and performing image recognition on video frames at multiple moments in the video data to determine a target object in the video frames.

[0038] The target object may be understood as a person performing power operation on site, and the target object may include an electrician or a construction worker, etc.

[0039] It can be understood that obtaining video data of an electric power operation site and performing image recognition on video frames at multiple moments in the video data to determine the target object in the video frame can include: collecting video data of the operation site through video acquisition equipment deployed at the electric power operation site (such as a camera or other video acquisition equipment, etc.); transmitting the video data to a central server or edge computing device through a wired or wireless network; extracting video frames from a continuous video stream, and the video frames can be selected at fixed time intervals or based on event triggers (such as extraction when a state change is detected); using image recognition technology (such as convolutional neural network CNN, etc.) to process the extracted video frames to identify the target object in the video frame.

[0040] S120. Perform joint detection on the target object to obtain joint information of multiple joint points of the target object, wherein the joint information at least includes joint coordinates.

[0041] The joint points may be understood as the joint positions identified for the target object. The joint points may include but are not limited to the head, shoulders, elbows, wrists, hips, knees and ankles. The joint information may be understood as the data of the joint positions of the target object. The joint information may at least include joint coordinates. The joint coordinates may be the position of each joint point.

[0042] S130. Determine the object state corresponding to the target object according to the joint information, and if the object state is an abnormal state, generate and display alarm information corresponding to the abnormal state, wherein the abnormal state includes at least one of abnormal posture, abnormal movement, and abnormal environment interaction.

[0043] Among them, the object state can be understood as the state of the target object obtained by analyzing the joint information. The object state is associated with the posture, action and interaction of the target object with the environment. The object state may include an abnormal state. The abnormal state may include at least one of abnormal posture, abnormal action and abnormal environment interaction. The abnormal posture can be understood as the posture of the target object does not meet the requirements of safety or normal work. The abnormal posture may include at least one of abnormal working posture, stagnant state and falling state. The abnormal action can be understood as the action of the target object does not meet the standard operating procedures or safety guidelines. The abnormal action may include irregular motion state. The abnormal environment interaction can be understood as the interaction between the target object and the surrounding environment does not meet the safety specifications or expected behavior patterns. The abnormal environment interaction may include the state of staying in a dangerous area or the state of dangerous behavior.

[0044] Optionally, in the case where the posture abnormality is an abnormal working posture, determining the object state corresponding to the target object based on the joint information includes: determining at least three target joint points from the multiple joint points, and determining the target angle between the at least three target joint points; when the target angle is not within a preset angle range, determining the object state corresponding to the target object as an abnormal working posture.

[0045] Among them, the target joint point can be understood as a joint point used to determine the object state corresponding to the target object. It can be understood that, in the case where the posture abnormality is an abnormal working posture, the target joint point can be a group of joint points selected from multiple joint points, which are used to determine the target angle, and then determine whether the working posture is abnormal. The target angle can be understood as the angle formed between at least three target joint points. By calculating the angle between the target joint points, it can be determined whether the working posture of the target object is abnormal. The preset angle range can be understood as a pre-set angle interval, which is used to determine whether the target angle is abnormal. If the target angle is within the preset angle range, it is determined that the working posture of the target object is normal; otherwise, it is confirmed that the working posture is abnormal.

[0046] Through the above technical solution, it is possible to detect the working posture of the target object in real time at the power operation site, and take timely measures when an abnormality is detected, thereby improving the safety of the operation site.

[0047] On the basis of the above scheme, the joint information includes confidence; when the posture abnormality is a stagnant state, determining the object state corresponding to the target object according to the joint information includes:

[0048] Determine a target joint point from the plurality of joint points, and determine the instantaneous displacement of the target joint point at a plurality of moments within a preset time period according to the joint coordinates of the target joint point;

[0049] Multiplying the confidence corresponding to the target joint point at each moment with the instantaneous displacement to obtain a displacement product, and summing the displacement products corresponding to multiple moments to obtain a stage displacement of the target object;

[0050] When the stage displacement is less than or equal to a preset displacement threshold, the object state corresponding to the target object is determined to be a stagnant state.

[0051] Among them, the stagnant state can be understood as a state in which the target object has almost no movement within a preset time period. The confidence can be understood as the degree of credibility of the detected target joint point position. The value of the confidence is usually between 0 and 1, wherein 1 indicates that the position of the target joint point is very certain, and 0 indicates that the position of the target joint point is completely uncertain. The higher the confidence value, the more reliable the position of the target joint point. The preset time period can be understood as a time interval for determining the movement of the target joint point. The instantaneous displacement can be understood as an indicator for measuring the distance moved by the target joint point within a preset time period, and specifically can be the displacement change of the target joint point relative to the previous moment at a certain moment. The displacement product can be understood as the result obtained by multiplying the instantaneous displacement of the target joint point at each moment with its corresponding confidence, which is used to ensure that only displacement data with high confidence is used first. The stage displacement can be understood as a total displacement change for determining the target joint point within a preset time period, which can be determined by the sum of all displacement products passed by the target joint point within the preset time period. The preset displacement threshold can be understood as a pre-set value used to determine whether the stage displacement of the target joint point is small enough. When the stage displacement is less than or equal to the preset displacement threshold, the object state corresponding to the target object is determined to be a stagnant state.

[0052] Through the above method, it is possible to accurately judge the movement of the target object within a preset time period and determine whether the target object is in a stagnant state. This is of great significance for detecting personnel activities at the power operation site and can help to promptly discover possible safety risks.

[0053] On the basis of the above solution, optionally, in the case where the abnormal posture is a falling state, determining the object state corresponding to the target object according to the joint information includes:

[0054] Determine a target joint point from the plurality of joint points, and determine the instantaneous height difference of the target joint point at a plurality of moments within a preset time period according to the joint coordinates of the target joint point;

[0055] The confidence corresponding to the target joint point at each moment is multiplied by the instantaneous height difference to obtain a height product, and the height products corresponding to multiple moments are summed to obtain a stage height difference of the target object;

[0056] When the stage height difference is less than or equal to a preset height threshold, the object state corresponding to the target object is determined as a falling state, wherein the preset height threshold is a negative value.

[0057] Among them, the instantaneous height difference can be understood as an indicator used to measure the position change of the target joint point in the vertical direction within a preset time period, and specifically can be the height change of the target joint point relative to the previous moment at a certain moment. The height product can be understood as the result obtained by multiplying the instantaneous height difference of the target joint point at each moment with its corresponding confidence, which is used to ensure that only high-confidence height difference data is used preferentially. The preset height threshold can be understood as a pre-set value, which is used to determine whether the stage height difference of the target joint point is small enough. When the stage height difference is less than or equal to the preset height threshold, it is determined that the object state corresponding to the target object is a falling state.

[0058] Through the above method, it is possible to accurately judge the vertical movement of the target object within a preset time period, and judge whether it is in a falling state based on this. This is of great significance for detecting the activities of the target object at the power operation site, and can help to timely discover possible safety risks.

[0059] On the basis of the above solution, optionally, in the case where the abnormal motion is an irregular motion state, determining the object state corresponding to the target object according to the joint information includes:

[0060] Determine a target joint point from the plurality of joint points, and determine a velocity time domain signal corresponding to the target joint point according to the joint coordinates of the target joint point at a plurality of moments;

[0061] The speed time domain signal is converted into a speed frequency domain signal by fast Fourier transform, and the amplitudes in the speed frequency domain signal are summed to obtain a total speed amplitude;

[0062] When the total velocity amplitude exceeds a preset amplitude threshold, the object state corresponding to the target object is determined to be an irregular motion state.

[0063] Wherein, the speed time domain signal can be understood as a signal indicating that the speed of the target joint point changes with time, that is, data describing the speed of the target joint point in the time domain. Correspondingly, the speed time domain signal can be understood as a signal after the speed time domain signal is transformed by fast Fourier transform, which is used to characterize the signal of the target joint point speed changing with frequency, that is, the speed amplitude at different frequencies. The amplitude can be understood as the intensity of each frequency component in the frequency domain signal. In this embodiment, the amplitude can be the intensity of different frequency components in the speed frequency domain signal. The total speed amplitude can be understood as the result obtained by adding the amplitudes of all speed frequency domain signals, which is used to characterize the energy distribution of the target joint point at each frequency. The preset amplitude threshold can be understood as a pre-set value, which is used to determine whether the total speed amplitude of the target joint point exceeds the range of normal movement. When the total speed amplitude exceeds the preset amplitude threshold, it is confirmed that the target object is in an irregular motion state.

[0064] Through the above method, it is possible to accurately judge the movement mode of the target object and determine whether it is in an irregular movement state. This is of great significance for detecting the activities of the target object at the power operation site and can help to promptly discover possible safety risks.

[0065] Based on the above solution, optionally, when the environmental interaction abnormality is a state of staying in a dangerous area, determining the object state corresponding to the target object according to the joint information includes:

[0066] When the joint coordinates of at least one joint point are within a preset danger zone, determining a residence time of the joint point within the preset danger zone;

[0067] When the stay time is greater than or equal to a preset time threshold, the object state corresponding to the target object is determined as a state of staying in a dangerous area.

[0068] Among them, the preset danger zone can be understood as a predefined unsafe area in the power operation site. The preset danger zone can be marked with polygons or other shapes in a map or image coordinate system. Exemplarily, the danger zone may include but is not limited to high-voltage equipment, flammable and explosive substances or other potential sources of danger. The preset time threshold can be understood as a pre-set value, which is used to determine whether the time the target joint point stays in the danger zone exceeds the allowed safety time. When the stay time is greater than or equal to the preset time threshold, it is determined that the target object is in a state of staying in the danger zone.

[0069] Through the above method, it is possible to accurately judge the stay of the target object in the dangerous area, and judge whether it is in the state of staying in the dangerous area. It is of great significance to detect the activities of the target object at the power operation site, and can help to timely discover possible safety risks.

[0070] Based on the above solution, optionally, the abnormal environment interaction includes a state where a dangerous behavior occurs; and determining the object state corresponding to the target object according to the joint information includes:

[0071] Determine a first joint point corresponding to the first part and a second joint point corresponding to the second part from the plurality of joint points;

[0072] Determine a height difference between the first part and the second part according to the joint coordinates of the first joint point and the second joint point;

[0073] When the height difference of the parts is greater than or equal to a preset difference threshold, the object state corresponding to the target object is determined as a state in which a dangerous behavior occurs.

[0074] Among them, the first part and the second part can be understood as the parts to be detected and related to the state of dangerous behavior of the target object, and the first part and the second part are different. The first joint point can be understood as the joint point corresponding to the first part. Exemplarily, in the case of detecting the movement of the hand, the first joint point can be the position of the hand. The second joint point can be understood as the joint point corresponding to the second part, and the second joint point is different from the first joint point. Exemplarily, if the movement of the foot is to be detected, the second joint point can be the position of the foot. The part height difference refers to the vertical height difference between the first part and the second part, which can be obtained by calculating the vertical distance between the first joint point and the second joint point. The preset difference threshold can be understood as a pre-set value, which is used to determine whether the part height difference exceeds the safety range. When the part height difference exceeds the preset difference threshold, it is confirmed that the target object is in a state of dangerous behavior.

[0075] Through the above method, it is possible to accurately judge the actions of the target object and determine whether it is in a state of dangerous behavior. This is of great significance for detecting the activities of the target object at the power operation site and can help to promptly discover possible safety risks.

[0076] The technical solution of the embodiment of the present invention is as follows: first, by acquiring video data of the power operation site, potential safety risks can be discovered in time, and image recognition is performed on video frames at multiple moments in the video data to determine the target object in the video frame, so as to detect whether the target object complies with safety regulations and ensure that the operation activities meet industry safety standards; then, joint detection is performed on the target object to obtain joint information of multiple joint points of the target object, wherein the joint information at least includes joint coordinates, so as to accurately capture the coordinates of each joint point of the target object, and the object state can be recorded and analyzed in detail to discover potential safety risks in time; finally, the object state corresponding to the target object is determined according to the joint information, and when the object state is an abnormal state, an alarm information corresponding to the abnormal state is generated and displayed, wherein the abnormal state includes at least one of abnormal posture, abnormal movement and abnormal environment interaction, and by real-time detection of the posture, movement and interaction of the target object with the environment, possible safety hazards can be quickly identified and physical injuries caused by improper posture or dangerous movement can be prevented, and warnings can be issued in time to avoid accidents. The technical solution of the embodiment of the present invention uses real-time image recognition and joint detection technology to accurately determine whether the state of the target object is abnormal. Once an abnormal situation is detected, an alarm is immediately issued, which solves the problem that the power operation site in the related technology relies on manual detection and video analysis technology, which is inefficient and prone to false alarms and missed alarms. It can effectively improve the safety detection accuracy of the power operation site, ensure the safety of the target object, and improve work efficiency.

[0077] Embodiment 2

[0078] Figure 2 The safety detection method for a power operation site provided in the second embodiment of the present invention is an optional embodiment of the above embodiment. The target object is a construction worker, the first object is a hand, and the second object is a foot.

[0079] like Figure 2 As shown, the method may specifically include:

[0080] First, video data of an electric power operation site is acquired, and image recognition is performed on video frames at multiple moments in the video data to determine construction personnel in the video frames.

[0081] Perform joint detection on the detected construction workers to obtain joint information of multiple joint points of the construction workers, where the joint information may include joint point coordinates and confidence levels.

[0082] Finally, the object state corresponding to the construction personnel is determined according to the joint information, and when the object state is an abnormal state, alarm information corresponding to the abnormal state is generated and displayed.

[0083] Abnormal states include abnormal posture (abnormal working posture, stagnant state and falling state), abnormal movement (irregular movement state), and abnormal environmental interaction (staying in a dangerous area and engaging in dangerous behavior).

[0084] The specific process of determining the object state corresponding to the construction personnel according to the joint information may include:

[0085] Collect joint information of a total number of joint points T and N in a total number of video frames;

[0086] Assuming that the joint point is i, the coordinates of the joint point i at time t are p i (t) = (x i (t),y i (t)).

[0087] Among them, x i (t) represents the horizontal coordinate of joint point i at time t (usually represents the horizontal position), y i (t) represents the ordinate of joint point i at time t (usually indicating the vertical position). Correspondingly, the coordinate p of joint point j at time t can be obtained similarly. j (t) and the coordinates p of the joint point k at time t k (t).

[0088] The confidence of joint point i at time t is conf i (t);

[0089] The instantaneous displacement Δp of joint point i at time t i (t) is: Δp i (t) = p i (t)-p i (t-Δt);

[0090] Wherein, Δt is the sampling time interval;

[0091] The modulus of the displacement vector of joint point i at time t is: ||Δp i (t)||.

[0092] Then, the detection process of (1) abnormal posture as abnormal working posture is:

[0093] Select three joint points i, j and k, and calculate the target angle θ of joint points i, j and k at time t ijk (t):

[0094]

[0095] If the target angle θ ijk(t) If it is not within the predefined normal working posture angle range, it is determined to be an abnormal working posture.

[0096] (2) The detection process of abnormal posture being in a stagnant state is as follows:

[0097] Calculate the instantaneous displacement Δp of joint point i at time t i (t);

[0098] Stagnation detection formula:

[0099]

[0100] When the stage displacement (the sum of the product of the confidence of the joint point and the instantaneous displacement at each moment) is less than the preset displacement threshold ε, it is determined to be a stagnant state.

[0101] (3) The detection process of abnormal posture as falling state is as follows:

[0102] Calculate the instantaneous height difference Δy of the body center (such as the hip joint) hip (t);

[0103] Δy hip (t) = y hip (t)-y hip (t-Δt);

[0104] Among them, y hip (t) is the height of the joint point at time t, y hip (t-Δt) is the height of the joint point at the moment t-Δt.

[0105] The detection formula for the fall state is:

[0106]

[0107] Among them, conf hip (t) is the confidence of the height of the joint point at time t.

[0108] When the stage height difference (the sum of the product of the confidence level and the instantaneous height difference) is less than the preset height threshold -γ, it is determined to be a fall state.

[0109] (4) The detection process of abnormal motion as irregular motion state is as follows:

[0110] Calculate the velocity v of joint point i at time t i (t):

[0111]

[0112] For speed v i (t) Perform fast Fourier transform to obtain the velocity V in the frequency domaini (f), each value of f in the frequency domain represents the amplitude of the velocity:

[0113] V i (f) = FFT(v i (t));

[0114] Detection formula for irregular motion state:

[0115]

[0116] When the total velocity amplitude (the sum of the amplitudes in the frequency domain) exceeds the preset amplitude threshold η, it is determined to be an irregular motion state.

[0117] (5) The detection process of the abnormal environment interaction state of staying in the dangerous area is as follows:

[0118] The danger zone is polygon R, and the personnel position is L(t);

[0119] Detection formula for staying in the danger zone:

[0120]

[0121] When the stay time is greater than or equal to the preset time threshold δ, it is determined as a state of staying in a dangerous area.

[0122] (6) The process of detecting the state where the environmental interaction anomaly leads to dangerous behavior is as follows:

[0123] Detect hand height y hand (t) and foot height y foot (t) Height difference of the parts:

[0124] y hand (t)-y foot (t)>λ;

[0125] When the height difference of the parts is greater than or equal to the preset difference threshold λ, it is determined that a dangerous behavior has occurred.

[0126] The technical solution of this embodiment can effectively detect the safety status of construction workers through real-time analysis of video data at the power operation site. First, the position of the construction workers is determined by performing image recognition on the video frame, and then the joint detection technology is used to obtain their joint information, including joint point coordinates and confidence. Based on the joint information, a variety of abnormal states can be automatically detected, including but not limited to abnormal working posture, stagnant state, fall state, irregular motion state, and abnormal environmental interaction, etc., which can timely discover potential safety hazards, such as construction workers staying in dangerous areas for a long time or taking working postures that may cause injuries. When an abnormal state is detected, a corresponding alarm message will be generated and displayed to remind relevant personnel to take measures to avoid accidents. It not only improves the safety management level of the operation site, but also reduces the cost of manual detection, improves work efficiency, and helps to build a safer and more reliable working environment.

[0127] Embodiment 3

[0128] Figure 3 This is a schematic diagram of the structure of a safety detection device for a power operation site provided by the third embodiment of the present invention. Figure 3 As shown, the device includes: a video acquisition module 310, a joint detection module 320, and an abnormal alarm module 330.

[0129] Among them, the video acquisition module 310 is used to acquire video data of the power operation site, and perform image recognition on video frames at multiple moments in the video data to determine the target object in the video frame; the joint detection module 320 is used to perform joint detection on the target object to obtain joint information of multiple joint points of the target object, wherein the joint information at least includes joint coordinates; the abnormal alarm module 330 is used to determine the object state corresponding to the target object according to the joint information, and when the object state is an abnormal state, generate and display alarm information corresponding to the abnormal state, wherein the abnormal state includes at least one of posture abnormality, movement abnormality and environmental interaction abnormality.

[0130] The technical solution of the embodiment of the present invention is as follows: first, the video data of the power operation site is acquired through the video acquisition module 210, so that potential safety risks can be discovered in time, and image recognition is performed on the video frames at multiple moments in the video data to determine the target object in the video frame, so as to detect whether the target object complies with safety regulations and ensure that the operation activities meet the industry safety standards; then, the joint detection module 220 performs joint detection on the target object to obtain joint information of multiple joint points of the target object, wherein the joint information at least includes joint coordinates, so as to accurately capture the coordinates of each joint point of the target object, and the object state can be recorded and analyzed in detail to discover potential safety risks in time; finally, the abnormal alarm module 230 determines the object state corresponding to the target object according to the joint information, and generates and displays alarm information corresponding to the abnormal state when the object state is an abnormal state, wherein the abnormal state includes at least one of abnormal posture, abnormal action and abnormal environment interaction, and by real-time detection of the posture, action and interaction of the target object with the environment, it is possible to quickly identify possible safety hazards and prevent physical injuries caused by improper posture or dangerous action, and issue warnings in time to avoid accidents. The technical solution of the embodiment of the present invention uses real-time image recognition and joint detection technology to accurately determine whether the state of the target object is abnormal. Once an abnormal situation is detected, an alarm is immediately issued, which solves the problem that the power operation site in the related technology relies on manual detection and video analysis technology, which is inefficient and prone to false alarms and missed alarms. It can effectively improve the safety detection accuracy of the power operation site, ensure the safety of the target object, and improve work efficiency.

[0131] On the basis of the above scheme, optionally, the posture abnormality includes a working posture abnormality, and the abnormal alarm module can be used to determine at least three target joint points from the multiple joint points, and determine the target angle between at least three target joint points; when the target angle is not within a preset angle range, the object state corresponding to the target object is determined as a working posture abnormality.

[0132] Based on the above scheme, optionally, the joint information includes confidence; the posture abnormality includes a stagnant state; the abnormal alarm module can be used to determine a target joint point from a plurality of the joint points, and determine the instantaneous displacement of the target joint point at multiple moments within a preset time period according to the joint coordinates of the target joint point; multiply the confidence corresponding to the target joint point at each moment by the displacement information to obtain a displacement product, sum the target products corresponding to multiple moments to obtain the stage displacement of the target object; when the stage displacement is less than or equal to a preset displacement threshold, determine the object state corresponding to the target object as a stagnant state.

[0133] On the basis of the above scheme, optionally, the posture abnormality includes a fall state; the abnormal alarm module can be used to determine a target joint point from a plurality of the joint points, and determine the instantaneous height difference of the target joint point at multiple moments within a preset time period according to the joint coordinates of the target joint point; multiply the confidence corresponding to the target joint point at each moment and the instantaneous height difference to obtain a height product, and sum the height products corresponding to multiple moments to obtain the stage height difference of the target object; when the stage height difference is less than or equal to a preset height threshold, the object state corresponding to the target object is determined as a fall state, wherein the preset height threshold is a negative value.

[0134] On the basis of the above scheme, optionally, the abnormal motion includes an irregular motion state; the abnormal alarm module can be used to determine the target joint point from the multiple joint points, and determine the speed time domain signal corresponding to the target joint point according to the joint coordinates of the target joint point at multiple moments; the speed time domain signal is converted into a speed frequency domain signal through fast Fourier transform, and the amplitude in the speed frequency domain signal is summed to obtain the total speed amplitude; when the total speed amplitude exceeds a preset amplitude threshold, the object state corresponding to the target object is determined as an irregular motion state.

[0135] On the basis of the above scheme, optionally, the environmental interaction anomaly includes a state of staying in a dangerous area; the abnormal alarm module can be used to determine the residence time of the joint point in the preset dangerous area when the joint coordinates of at least one joint point are within the preset dangerous area; when the residence time is greater than or equal to a preset time threshold, the object state corresponding to the target object is determined as a state of staying in the dangerous area.

[0136] On the basis of the above scheme, optionally, the environmental interaction anomaly includes a state in which a dangerous behavior occurs; the abnormal alarm module determines a first joint point corresponding to the first part and a second joint point corresponding to the second part from the multiple joint points; determines the part height difference between the first part and the second part according to the joint coordinates of the first joint point and the second joint point; when the part height difference is greater than or equal to a preset difference threshold, determines the object state corresponding to the target object as a state in which a dangerous behavior occurs.

[0137] The safety detection device for the power operation site provided by the embodiment of the present invention can execute the safety detection method for the power operation site provided by any embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method.

[0138] Example 4

[0139] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0140] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0141] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0142] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a safety detection method for a power operation site.

[0143] In some embodiments, the safety detection method for the power operation site may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the safety detection method for the power operation site described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the safety detection method for the power operation site in any other appropriate manner (e.g., by means of firmware).

[0144] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0145] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0146] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0147] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0148] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0149] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0150] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0151] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A safety detection method for an electric power operation site, characterized in that: include: Acquire video data of an electric power operation site, and perform image recognition on video frames at multiple moments in the video data to determine a target object in the video frame; Performing joint detection on the target object to obtain joint information of multiple joint points of the target object, wherein the joint information at least includes joint coordinates; The object state corresponding to the target object is determined according to the joint information. When the object state is an abnormal state, an alarm message corresponding to the abnormal state is generated and displayed, wherein the abnormal state includes at least one of abnormal posture, abnormal movement and abnormal environment interaction.

2. The method according to claim 1, characterized in that The abnormal posture includes abnormal working posture; and determining the object state corresponding to the target object according to the joint information includes: Determine at least three target joint points from the plurality of joint points, and determine a target angle between the at least three target joint points; When the target angle is not within the preset angle range, the object state corresponding to the target object is determined as an abnormal working posture.

3. The method according to claim 1, characterized in that: The joint information includes confidence; the posture abnormality includes a stagnant state; and determining the object state corresponding to the target object according to the joint information includes: Determine a target joint point from the plurality of joint points, and determine the instantaneous displacement of the target joint point at a plurality of moments within a preset time period according to the joint coordinates of the target joint point; Multiplying the confidence corresponding to the target joint point at each moment with the instantaneous displacement to obtain a displacement product, and summing the displacement products corresponding to multiple moments to obtain a stage displacement of the target object; When the stage displacement is less than or equal to a preset displacement threshold, the object state corresponding to the target object is determined to be a stagnant state.

4. The method according to claim 1, characterized in that: The abnormal posture includes a falling state; and determining the object state corresponding to the target object according to the joint information includes: Determine a target joint point from the plurality of joint points, and determine the instantaneous height difference of the target joint point at a plurality of moments within a preset time period according to the joint coordinates of the target joint point; The confidence corresponding to the target joint point at each moment is multiplied by the instantaneous height difference to obtain a height product, and the height products corresponding to multiple moments are summed to obtain a stage height difference of the target object; When the stage height difference is less than or equal to a preset height threshold, the object state corresponding to the target object is determined as a falling state, wherein the preset height threshold is a negative value.

5. The method according to claim 1, characterized in that The abnormal motion includes an irregular motion state; and determining the object state corresponding to the target object according to the joint information includes: Determine a target joint point from the plurality of joint points, and determine a velocity time domain signal corresponding to the target joint point according to the joint coordinates of the target joint point at a plurality of moments; The speed time domain signal is converted into a speed frequency domain signal by fast Fourier transform, and the amplitudes in the speed frequency domain signal are summed to obtain a total speed amplitude; When the total velocity amplitude exceeds a preset amplitude threshold, the object state corresponding to the target object is determined to be an irregular motion state.

6. The method according to claim 1, characterized in that The abnormal environment interaction includes a state of staying in a dangerous area; and determining the object state corresponding to the target object according to the joint information includes: When the joint coordinates of at least one joint point are within a preset danger zone, determining a residence time of the joint point within the preset danger zone; When the stay time is greater than or equal to a preset time threshold, the object state corresponding to the target object is determined as a state of staying in a dangerous area.

7. The method according to claim 1, characterized in that The abnormal environment interaction includes a state where a dangerous behavior occurs; and determining the object state corresponding to the target object according to the joint information includes: Determine a first joint point corresponding to the first part and a second joint point corresponding to the second part from the plurality of joint points; Determine a height difference between the first part and the second part according to the joint coordinates of the first joint point and the second joint point; When the height difference of the parts is greater than or equal to a preset difference threshold, the object state corresponding to the target object is determined as a state in which a dangerous behavior occurs.

8. A safety detection device for an electric power operation site, characterized in that: include: A video acquisition module is used to acquire video data of the power operation site and perform image recognition on video frames at multiple moments in the video data to determine the target object in the video frame; A joint detection module, used to perform joint detection on the target object to obtain joint information of multiple joint points of the target object, wherein the joint information at least includes joint coordinates; An abnormal alarm module is used to determine the object state corresponding to the target object based on the joint information, and when the object state is an abnormal state, generate and display alarm information corresponding to the abnormal state, wherein the abnormal state includes at least one of abnormal posture, abnormal movement and abnormal environment interaction.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the safety detection method for the power operation site according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the safety detection method for a power operation site according to any one of claims 1 to 7 when executed.

Images