Method, device and terminal equipment for detecting operation behavior norms based on target detection

By using the target detection model to analyze synchronous video frames during the gas transportation company during the delivery and distribution company, the efficiency and accuracy of human work standard detection are solved, efficient and standardized detection of staff behavior is achieved, and the detection speed is improved. and accuracy.

CN113780077BActive Publication Date: 2024-12-03SHENZHEN GAS CORP +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110897147.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-08-05
Publication Date
2024-12-03
Estimated Expiration
2041-08-05

AI Technical Summary

Technical Problem

During the gas transportation process of the transportation and distribution company, it is difficult to achieve efficient and accurate human work standard inspection, and the labor intensity is high, the work efficiency is low, and the personnel safety is poor, especially in severe weather and complex terrain.

Method used

Using the operation behavioral norm detection method based on object detection, by obtaining the synchronous video frame of the station and using the trained object detection model, it determines whether all target objects in the target object set is carried. If carried, it is determined that the behavior meets the specification. If not carried, the violation will be determined based on the preset duration and behavioral norms.

Benefits of technology

The speed and accuracy of personnel posture detection are improved, and the human posture estimation test model of deep convolutional neural network is realized efficient and standardized detection of staff working behaviors.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113780077B_ABST
    Figure CN113780077B_ABST
Patent Text Reader

Abstract

The present application discloses a method, device and terminal device for detecting work behavior norms based on target detection, the method comprising obtaining a number of synchronous video frames corresponding to a station; determining whether the obtained number of synchronous video frames carry all target objects in a target object set based on a trained target detection model; if all target objects in a target object set are carried, it is determined that the work behavior of the staff meets the norms. The present application synchronously obtains a number of video frames, and uses a target detection model to determine whether a number of video frames carry all target objects in a target object set to detect the work behavior of the staff. In this way, a human posture estimation test model based on a deep convolutional neural network has a powerful computing capability, which can greatly improve the speed of personnel posture detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of image analysis and behavior recognition, and in particular to a method, device and terminal equipment for detecting operating behavior norms based on target detection. Background Art

[0002] The gas transportation process of the transmission and distribution company has the following operating specifications: (1) Before the tank truck enters the station, the station staff needs to walk around the tank truck to check the vehicle's condition; (2) After the tank truck stops at the loading area, before loading and unloading operations, the station staff needs to place anti-skid mats on the wheels to prevent the vehicle from accidentally moving; (3) After the tank truck stops at the unloading area, before loading and unloading operations, the station staff needs to place a warning sign in front of the vehicle to remind unauthorized personnel not to approach. However, the manual operation method has the characteristics of high labor intensity, low work efficiency, and poor personnel safety, which is more obvious under conditions such as bad weather and complex terrain. How to judge whether the staff is operating in accordance with the specifications is an urgent problem to be solved by technical personnel in this field. Summary of the invention

[0003] The technical problem to be solved by the present application is to provide a method, device and terminal equipment for detecting operation behavior norms based on target detection in view of the deficiencies of the existing technology.

[0004] In order to solve the above technical problems, the first aspect of the embodiments of the present application provides a method for detecting operation behavior specifications based on target detection, the method comprising:

[0005] Obtain several synchronized video frames corresponding to the station;

[0006] Based on the trained object detection model, determining whether the acquired multiple synchronized video frames carry all the target objects in the target object set;

[0007] If all target objects in the target object set are carried, it is determined that the staff member's operating behavior complies with the specifications.

[0008] The method for detecting the operation behavior specification based on target detection, wherein the method further comprises:

[0009] If all target objects in the target object set are not carried, then the time interval between the acquisition time of several synchronous video frames and the entry time of the tank truck into the preset area of ​​the station is obtained;

[0010] If the time interval reaches the preset duration, the corresponding behavior information of the staff is determined based on the trained target recognition model and the acquired synchronized video frames;

[0011] Based on the behavior information and preset behavior norms, the staff's illegal operations are determined.

[0012] The method for detecting the operation behavior specification based on target detection, wherein the method further comprises:

[0013] If the time interval does not reach the preset duration, the acquisition of several synchronous video frames corresponding to the station continues until the acquired several synchronous video frames carry all target objects in the target object set or the time interval reaches the preset duration.

[0014] In the method for detecting operating behavior specifications based on target detection, the plurality of synchronous video frames are obtained by synchronously shooting with a plurality of imaging devices, and the plurality of synchronous video frames correspond one-to-one to the plurality of imaging devices.

[0015] The method for detecting operating behavior specifications based on target detection, wherein the plurality of imaging devices themselves include a front camera of the tank truck, a rear camera of the tank truck, and fixed cameras and / or mobile cameras within the station.

[0016] The method for detecting the operation behavior specification based on target detection, wherein the step of determining whether the acquired plurality of synchronized video frames carry all target objects in the target object set based on the trained target detection model specifically comprises:

[0017] Inputting each of the plurality of synchronized video frames into a trained object detection model, and outputting a reference object carried by each synchronized video frame through the object detection model;

[0018] All acquired reference objects are matched with the target object set to determine whether a number of synchronized video frames carry all target objects in the target object set.

[0019] The method for detecting operating behavior specifications based on target detection, wherein the target object set includes tank trucks, staff, anti-slip mats and warning signs.

[0020] A second aspect of an embodiment of the present application provides a device for detecting operating behavior norms based on target detection, the device comprising:

[0021] An acquisition module, used to acquire a number of synchronous video frames corresponding to the station;

[0022] A determination module, used for determining whether the acquired plurality of synchronized video frames carry all target objects in the target object set based on a trained target detection model;

[0023] The determination module is used to determine whether the operation behavior of the staff meets the specification when carrying all the target objects in the target object set.

[0024] A third aspect of an embodiment of the present application provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in any of the target detection-based operation behavior specification detection methods described above.

[0025] A fourth aspect of the embodiments of the present application provides a terminal device, comprising: a processor, a memory, and a communication bus; the memory stores a computer-readable program that can be executed by the processor;

[0026] The communication bus realizes the connection and communication between the processor and the memory;

[0027] When the processor executes the computer-readable program, the steps in any of the target detection-based operation behavior specification detection methods described above are implemented.

[0028] Beneficial effects: Compared with the prior art, the present application provides a method, device and terminal device for detecting work behavior specifications based on target detection, the method comprising obtaining a number of synchronous video frames corresponding to the station; based on a trained target detection model, determining whether the obtained number of synchronous video frames carry all target objects in the target object set; if all target objects in the target object set are carried, it is determined that the work behavior of the staff meets the specifications. The present application synchronously obtains a number of video frames, and uses a target detection model to determine whether a number of video frames carry all target objects in the target object set to detect the work behavior of the staff. In this way, the human posture estimation test model based on a deep convolutional neural network has powerful computing power and can greatly improve the speed of personnel posture detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without inventive work.

[0030] Figure 1 A flowchart of the target detection-based operation behavior specification detection method provided in this application.

[0031] Figure 2 This is a schematic diagram of the structure of the target detection-based operating behavior standard detection device provided in this application.

[0032] Figure 3 This is a schematic diagram of the structure of the terminal device provided in this application. DETAILED DESCRIPTION

[0033] The present application provides a method, device and terminal device for detecting operation behavior norms based on target detection. In order to make the purpose, technical solution and effect of the present application clearer and more specific, the present application is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0034] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0035] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0036] In a specific implementation, the terminal device described in the embodiments of the present application includes, but is not limited to, other portable devices such as mobile phones with touch-sensitive surfaces (e.g., touch display screens and / or touch pads), laptop computers, or tablet computers. It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., touch display screens and / or touch pads).

[0037] In the following discussion, a terminal device including a display and a touch-sensitive surface is described. However, it should be understood that the terminal device may also include one or more other physical user interface devices such as a physical keyboard, a mouse and / or a joystick.

[0038] The terminal device supports various applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, video conferencing applications, disk burning applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, exercise support applications, photo management applications, data camera applications, digital video camera applications, web browsing applications, digital music player applications and / or digital video playback applications, etc.

[0039] Various applications that can be executed on the terminal device can use at least one common physical user interface device such as a touch-sensitive surface. The first or multiple functions of the touch-sensitive surface and the corresponding information displayed on the terminal can be adjusted and / or changed between applications and / or within the corresponding application. In this way, the common physical framework of the terminal (e.g., the touch-sensitive surface) can support various applications with user interfaces that are intuitive and transparent to the user.

[0040] It should be understood that the sequence numbers and sizes of the steps in this embodiment do not mean the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0041] After research, the inventor found that the gas transportation process of the transmission and distribution company has the following operating specifications: (1) Before the tank truck enters the station, the station staff needs to walk around the tank truck to check the vehicle condition; (2) After the tank truck stops at the loading area, before loading and unloading operations, the station staff needs to place anti-skid mats on the wheels to prevent the vehicle from accidentally moving; (3) After the tank truck stops at the unloading area, before loading and unloading operations, the station staff needs to place a warning sign in front of the vehicle to remind unauthorized personnel not to approach. However, the manual operation method has the characteristics of high labor intensity, low work efficiency, and poor personnel safety, which is more obvious under conditions such as bad weather and complex terrain. In addition, how to judge whether the staff is operating in accordance with the specifications is an urgent problem to be solved by technical personnel in this field.

[0042] In order to solve the above problems, in an embodiment of the present application, several synchronous video frames corresponding to the station are obtained; based on the trained target detection model, it is determined whether the several synchronous video frames obtained carry all the target objects in the target object set; if all the target objects in the target object set are carried, it is determined that the staff's work behavior meets the specifications. The present application synchronously obtains several video frames, and uses the target detection model to determine whether several video frames carry all the target objects in the target object set to detect the staff's work behavior. In this way, the human posture estimation test model based on the deep convolutional neural network has powerful computing power and can greatly improve the speed of personnel posture detection.

[0043] The application content is further explained below through the description of embodiments in conjunction with the accompanying drawings.

[0044] This embodiment provides a method for detecting operation behavior norms based on target detection, such as Figure 1 and Figure 2 As shown, the method includes:

[0045] S10. Obtain several synchronous video frames corresponding to the station.

[0046] Specifically, the shooting time corresponding to each of the several synchronous video frames is the same, that is, the several synchronous video frames are obtained by synchronous shooting of several imaging devices, and the several synchronous video frames correspond one to one to the several imaging devices. It can be understood that several imaging devices receive shooting instructions at the same time, and each shoots their respective shooting ranges based on the shooting instructions received to obtain their respective corresponding video frames, so as to obtain several synchronous video frames. Among them, the shooting scene corresponding to each video frame in the several synchronous video frames is contained in the station, and the shooting scene corresponding to each synchronous video frame may be different. In other words, each imaging device shoots different areas in the station, so that the several synchronous video frames obtained by shooting can obtain image information of each area of ​​the station.

[0047] In one implementation of this embodiment, the plurality of imaging devices themselves include a front camera of the tank truck, a rear camera of the tank truck, and a fixed camera and / or a mobile camera in the station. The fixed camera and the mobile camera in the station are used to capture regional images in a preset area, and the preset area includes a station area in the station for parking tank trucks. For example, the area center of the preset area is the same as the center of the station area for parking tank trucks, the area shape is the same as the area shape of the station area for parking tank trucks, and the area size is twice the area size of the station area for parking tank trucks.

[0048] S20: Based on the trained target detection model, determine whether the acquired multiple synchronous video frames carry all target objects in the target object set.

[0049] Specifically, the target detection model is a trained network model, and the reference object carried by the video frame can be detected by the target detection model, wherein the target detection model can be a deep learning module, such as a deep convolutional neural network. In addition, the target object set is pre-set and is used as a basis for measuring whether the staff's work behavior complies with the specification. Since the gas transportation process of the transmission and distribution company has the following work specifications: (1) Before the tank truck enters the station, the station staff needs to walk around the tank truck to check the vehicle condition; (2) After the tank truck stops in the loading area, before loading and unloading operations, the station staff needs to place anti-skid mats on the wheels to prevent the vehicle from moving accidentally; (3) After the tank truck stops in the unloading area, before loading and unloading operations, the station staff needs to place a warning sign in front of the vehicle to remind unrelated personnel not to approach. Therefore, the target object set can include tank trucks, staff, anti-skid mats and warning signs, so as to determine whether the staff's work behavior complies with the specifications by detecting whether the tank truck, staff, anti-skid mats and warning signs are carried in a number of synchronized video frames.

[0050] In addition, during the gas transportation process of the transmission and distribution company, tank trucks, anti-skid mats and warning signs all need to be placed in their respective designated positions. Therefore, when determining whether the acquired several synchronous video frames carry all the target objects in the target object set, it is also necessary to determine whether each target object in the target object set is located at the designated position. When each target object in the target object set is located at the designated position, it is determined that several synchronous video frames carry all the target objects in the target object set. Therefore, when the target detection model detects the reference object in the synchronous video frame, it also detects the position information of the reference object in the synchronous video frame, so as to determine the reference object and the position information corresponding to the reference object based on the position information, so as to determine whether several synchronous videos carry all the target objects in the target object set.

[0051] In an implementation of this embodiment, determining whether the acquired plurality of synchronized video frames carry all target objects in the target object set based on the trained target detection model specifically includes:

[0052] Inputting each of the plurality of synchronized video frames into a trained object detection model, and outputting a reference object carried by each synchronized video frame through the object detection model;

[0053] All acquired reference objects are matched with the target object set to determine whether a number of synchronized video frames carry all target objects in the target object set.

[0054] Specifically, the input item of the target detection model is a synchronized video frame, and the target detection model is a reference object carried by the synchronized video frame, wherein the reference object is included in the target object set. It can be understood that the target detection model is used to detect the target object in the target object set in the synchronized video frame, and the synchronized video frame may include one target object in the target object set, may include multiple target objects in the target object set, or may not include the target object in the target object set. Correspondingly, the target detection model can detect one reference object, may detect multiple reference objects, or may not detect the reference object. For example, the target object set includes a tank truck, a staff member, an anti-slip mat, and a warning sign, and the synchronized video frame carries a staff member, then the target detection model can detect a reference object, which is a staff member in the target object set.

[0055] In one implementation of the present embodiment, each of the several synchronized video frames may carry a different reference object, so that after acquiring the several synchronized video frames, the reference objects carried by each synchronized video frame are acquired respectively through the target detection model, and then all the acquired reference objects are deduplicated to remove duplicate reference objects among all the acquired reference objects, thereby obtaining a reference object set; then the reference object set is compared with the target object set, and if each target object in the target object set is included in the reference object set, then the several synchronized video frames carry all target objects in the target object set; conversely, if there is at least one target object included in the reference object set, then the several synchronized video frames do not carry all target objects in the target object set.

[0056] In one implementation of this embodiment, the training process of the target detection model may include obtaining a training sample set, and training a preset detection network based on each training sample in the training sample set to obtain a target detection model. Wherein, the training sample set includes a number of training video frames, each of the training video frames carries at least one target object, and the default object configured by the preset detection network is all target objects in the target object set. That is, the preset detection network configures all target objects in the target object set with a default object, and the preset detection network is trained through the training sample set so that the trained preset network model can identify the target object carried in the video frame, that is, the target detection model can detect the target object carried by the synchronous video frame. For example, the target object set includes a tank truck, a staff member, an anti-slip mat, and a warning sign, and the default object configured by the preset detection network includes a tank truck, a staff member, an anti-slip mat, and a warning sign. Each training video frame in the training sample set carries at least one of the tank truck, the staff member, the anti-slip mat, and the warning sign, such as training video frame A carries a tank truck, training video frame B carries an anti-slip mat, etc.

[0057] In one implementation of this embodiment, during the gas transportation process of the transmission and distribution company, the tank truck needs to be parked at its corresponding designated position before the staff will place the anti-skid mat and warning sign. That is to say, when detecting all objects in the target object set, it can be determined whether a tank truck is carried in a number of synchronized videos. When the tank truck is detected, it will be determined whether the tank truck is located at the designated position corresponding to the tank truck. When the tank truck is located at its corresponding designated position, the parking time of the tank truck at the designated position is determined. When the parking time reaches the target time, the staff, anti-skid mat and warning sign will be detected.

[0058] Based on this, based on the trained target detection model, the process of determining whether the acquired synchronized video frames carry all the target objects in the target object set can be:

[0059] Determine all reference objects carried by a number of synchronized video frames based on a trained object detection model, and determine whether a tank truck is included in all the reference objects;

[0060] If a tank truck exists, the time interval between the synchronized video frame and the candidate video frame in which the tank truck is first detected is used to determine the parking time of the tank truck;

[0061] When the parking time does not reach the specified time, the step of acquiring a plurality of synchronized video frames is continued until the parking time reaches the specified time;

[0062] When the parking time reaches the specified time, all reference objects are detected to see whether they carry staff, anti-slip mats and warning signs, so as to determine whether several synchronized video frames carry all target objects in the target object set.

[0063] S30. If all target objects in the target object set are carried, it is determined that the staff member's operating behavior complies with the specification.

[0064] Specifically, the said carrying all target objects in the target object set refers to a video frame set formed by a number of synchronous video frames carrying all target objects in the target object set, and each target object is located at a corresponding designated position, wherein each target object can exist in one synchronous video frame, or can be stored in multiple synchronous video frames, and for each target object in the target image set, there is at least one synchronous video frame in a number of synchronous video frames, and the target object is included in the synchronous video frame. In addition, when all target objects are carried in a number of synchronous video frames, it means that the staff has placed the target object at the designated position of the target object, thereby determining the staff's operating behavior norms. For example, the target objects include a tank truck, a staff member, an anti-skid mat, and a warning sign. When it is determined based on a number of synchronous video frames that the tank truck, the anti-skid mat, and the warning sign are all located at their corresponding designated positions, it means that the staff's operating behavior meets the norms.

[0065] In an implementation of this embodiment, the detection method further includes:

[0066] S40, if all target objects in the target object set are not carried, then obtaining the time interval between the acquisition time of several synchronous video frames and the entry time of the tank truck into the preset area of ​​the station;

[0067] S50, if the time interval does not reach the preset duration, continue to obtain a number of synchronous video frames corresponding to the station until the obtained number of synchronous video frames carry all target objects in the target object set or the time interval reaches the preset duration;

[0068] S60: If the time interval reaches a preset time length, the corresponding behavior information of the staff is determined based on the trained target recognition model and the acquired synchronized video frames, and the illegal operation of the staff is determined based on the behavior information and the preset behavior norms.

[0069] Specifically, the preset area is a pre-set area in the station for parking tank trucks, and the time interval is used to reflect the time the tank truck stays in the preset area of ​​the station, wherein the premise for obtaining the time interval between the acquisition time of several synchronous video frames and the entry time of the tank truck into the preset area of ​​the station is that the several synchronous video frames carry the tank truck, that is, when the several synchronous video frames carry the tank truck but do not carry all the target objects in the target object set, the time interval between the acquisition time of several synchronous video frames and the entry time of the tank truck into the preset area of ​​the station is obtained. In addition, the acquisition time is the shooting time of several synchronous video frames. Since the several synchronous video frames are shot synchronously, the shooting time corresponding to each synchronous video frame is the same. Therefore, when obtaining the time interval between the acquisition time of several synchronous video frames and the entry time of the tank truck into the preset area of ​​the station, the shooting time of any synchronous video frame in the several synchronous video frames and the entry time of the tank truck into the preset area of ​​the station can be obtained; and then the time interval is calculated based on the acquired shooting time and entry time.

[0070] In addition, the preset duration is pre-set and is used to limit the maximum duration for the staff to place the target object at the designated location corresponding to the target object. When the preset interval reaches the preset duration, it means that the staff has placed the target object at the designated location in accordance with the specification; conversely, when the preset interval does not reach the preset duration, it means that the staff has not violated the work specification and may be in the process of placing the target object at the designated location. Therefore, at this time, the acquisition of several synchronous video frames corresponding to the station is continued until the acquired several synchronous video frames carry all the target objects in the target object set or the time interval reaches the preset duration. The preset behavior specification is a preset correct behavior specification. By comparing the preset behavior specification with the identified behavior information, the behavior of the staff that does not meet the behavior specification can be determined.

[0071] In one implementation of this embodiment, the target recognition model is trained to identify the behavior information of the staff in the synchronous video frame, wherein the behavior information may include standing guard, looking out, holding the gate, playing with the mobile phone, lying down to sleep, and swinging the arm, etc. It can be understood that the input item of the target recognition model is the synchronous video frame, and the output item is the behavior information of the staff in the synchronous video frame, for example, standing guard, playing with the mobile phone, etc. The target recognition model can adopt a regression-based neural network model, so that the detection problem of the staff behavior of the station is solved as a regression problem, and end-to-end optimization can be achieved, so that the work behavior information of the staff can be efficiently detected.

[0072] In one implementation of this embodiment, the training process of the target recognition model may be: extracting frames from the collected historical synchronous video data, and annotating each extracted synchronous data frame to form a training data set, wherein the annotation may include block diagram location information and behavior categories for characterizing the behavior of the staff, and the behavior categories corresponding to each synchronous data frame are all preset behavior categories in the preset behavior category set; iteratively training the preset neural network based on regression using the training data set until the detection accuracy and detection speed of the preset neural network meet the preset requirements to obtain the target recognition model. In addition, the preset behavior category set is a collection of work behavior categories that do not meet safety standards, wherein the preset behavior category set may include standing guard, looking out, holding the gate, playing with mobile phones, lying down to sleep, and swinging arms, etc.

[0073] In one implementation of this embodiment, the target recognition model may include a number of convolution units, each of which includes a number of convolution layers and a pooling layer, and the number of convolution layers are cascaded in sequence and connected to the pooling layer. For example, the target recognition model includes 4 convolution units, each of which includes two convolution layers and a pooling layer, and the two convolution layers are cascaded and then cascaded with the pooling layer, wherein the number of convolution kernels from the first convolution layer to the sixth convolution layer is 64, 128, 256, 256, 512 and 512 respectively, and the convolution kernel size of each convolution kernel is 3x3, the stride is 1, and the padding is 1. The seventh convolution layer is composed of 512 3x3 convolution kernels with a stride of 1, and the eighth convolution layer is composed of 512 2x2 convolution kernels with a stride of 1. In addition, each convolution layer uses the following formula to normalize the input data:

[0074]

[0075] Among them, E[x (k) ] represents the average value of neurons x(k) in each batch of training data; Represents a standard deviation of the activation of neurons x(k) in each batch of data.

[0076] In addition, when training the target recognition model, batch stochastic gradient descent can be used for training. The loss function in the training process can use the AM-softmax loss function, which can be expressed as:

[0077]

[0078] Among them, cosθ yi Indicates the area of ​​the calculated sample on the behavior category, cosθ yi -m requires that the area between the behavior categories has at least a spacing of hyperparameter m, and s is a hyperparameter used to expand the interval of cos values ​​by s times; y i represents the behavior category, c represents the number of behavior categories, and j represents the jth behavior category; Represents the behavior category y i The corresponding weight coefficient.

[0079] In one implementation of this embodiment, the detection method further includes: when it is determined that the staff's operating behavior complies with the specification, an alarm message can be sent to the cloud server, so that after receiving the alarm message, the cloud server records the violation log and sends the message to the terminal device of the security manager of the site within a first preset time. Of course, when the staff's violation is detected, the violation can also be sent to the cloud server so that the cloud server can obtain the staff's violation.

[0080] In summary, this embodiment provides a method, device and terminal device for detecting work behavior specifications based on target detection, the method comprising obtaining a number of synchronous video frames corresponding to the site; based on a trained target detection model, determining whether the obtained number of synchronous video frames carry all target objects in the target object set; if all target objects in the target object set are carried, it is determined that the staff's work behavior complies with the specification. This application synchronously obtains a number of video frames, and uses a target detection model to determine whether a number of video frames carry all target objects in the target object set to detect the staff's work behavior. In this way, the human posture estimation test model based on a deep convolutional neural network has powerful computing power and can greatly improve the speed of personnel posture detection.

[0081] Based on the above-mentioned target detection-based work behavior specification detection method, this embodiment provides a target detection-based work behavior specification detection device, such as Figure 2 As shown, the detection device comprises:

[0082] An acquisition module 100 is used to acquire a number of synchronous video frames corresponding to the station;

[0083] A determination module 200 is used to determine whether the acquired synchronized video frames carry all target objects in the target object set based on a trained target detection model;

[0084] The determination module 300 is used to determine whether the worker's operation behavior complies with the specification when carrying all target objects in the target object set.

[0085] Based on the above-mentioned target detection-based job behavior specification detection method, this embodiment provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the target detection-based job behavior specification detection method as described in the above-mentioned embodiment.

[0086] Based on the above-mentioned target detection-based operation behavior specification detection method, the present application also provides a terminal device, such as Figure 3As shown, it includes at least one processor (processor) 20; display screen 21; and memory (memory) 22, and may also include a communications interface (Communications Interface) 23 and a bus 24. Among them, the processor 20, the display screen 21, the memory 22 and the communication interface 23 can communicate with each other through the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setting mode. The communication interface 23 can transmit information. The processor 20 can call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0087] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0088] The memory 22 is a computer-readable storage medium that can be configured to store software programs, computer executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions or modules stored in the memory 22, that is, implementing the methods in the above embodiments.

[0089] The memory 22 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, a variety of media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, may also be a transient storage medium.

[0090] In addition, the working process of the above-mentioned target detection-based operation behavior specification detection device, the specific process of loading and executing the above-mentioned storage medium and multiple instruction processors in the terminal device have been described in detail in the above-mentioned method and will not be stated one by one here.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting operation behavior norms based on target detection, characterized in that: The method comprises: Obtain several synchronized video frames corresponding to the station; Based on the trained object detection model, determining whether the acquired multiple synchronized video frames carry all the target objects in the target object set; If all target objects in the target object set are carried, it is determined that the staff member's operating behavior meets the specifications; The target object set includes tank trucks, staff, anti-slip mats and warning signs; Based on the trained target detection model, the process of determining whether the acquired synchronized video frames carry all target objects in the target object set specifically includes: Determine all reference objects carried by a number of synchronized video frames based on a trained object detection model, and determine whether a tank truck is included in all the reference objects; If a tank truck exists, the time interval between the synchronized video frame and the candidate video frame in which the tank truck is first detected is used to determine the parking time of the tank truck; When the parking time does not reach the specified time, the step of acquiring a plurality of synchronized video frames is continued until the parking time reaches the specified time; When the parking time reaches the specified time, all reference objects are detected to see whether they carry staff, anti-slip mats and warning signs, so as to determine whether a number of synchronous video frames carry all target objects in the target object set.

2. The method for detecting operating behavior norms based on target detection according to claim 1, characterized in that: The method further comprises: If all target objects in the target object set are not carried, then the time interval between the acquisition time of several synchronous video frames and the entry time of the tank truck into the preset area of ​​the station is obtained; If the time interval reaches the preset duration, the corresponding behavior information of the staff is determined based on the trained target recognition model and the acquired synchronized video frames; Based on the behavior information and preset behavior norms, the staff's illegal operations are determined.

3. The method for detecting operating behavior norms based on target detection according to claim 2, characterized in that: The method further comprises: If the time interval does not reach the preset duration, the acquisition of several synchronous video frames corresponding to the station continues until the acquired several synchronous video frames carry all target objects in the target object set or the time interval reaches the preset duration.

4. The method for detecting operating behavior norms based on target detection according to claim 1, characterized in that: The plurality of synchronous video frames are obtained by synchronously shooting a plurality of imaging devices, and the plurality of synchronous video frames correspond one to one to the plurality of imaging devices.

5. The method for detecting operating behavior norms based on target detection according to claim 4 is characterized in that: The plurality of imaging devices themselves include a tank truck front camera, a tank truck rear camera, and fixed cameras and / or mobile cameras within the terminal.

6. The method for detecting operating behavior norms based on target detection according to claim 1, characterized in that: The determining, based on the trained target detection model, whether the acquired plurality of synchronized video frames carry all target objects in the target object set specifically includes: Inputting each of the plurality of synchronized video frames into a trained object detection model, and outputting a reference object carried by each synchronized video frame through the object detection model; All acquired reference objects are matched with the target object set to determine whether a number of synchronized video frames carry all target objects in the target object set.

7. A device for detecting operating behavior norms based on target detection, characterized in that: The detection device comprises: An acquisition module, used to acquire a number of synchronous video frames corresponding to the station; A determination module, used for determining whether the acquired plurality of synchronized video frames carry all target objects in the target object set based on a trained target detection model; A determination module, used to determine whether the worker's operation behavior complies with the specification when carrying all target objects in the target object set; The target object set includes tank trucks, staff, anti-slip mats and warning signs; Based on the trained target detection model, the process of determining whether the acquired synchronized video frames carry all target objects in the target object set specifically includes: Determine all reference objects carried by a number of synchronized video frames based on a trained object detection model, and determine whether a tank truck is included in all the reference objects; If a tank truck exists, the time interval between the synchronized video frame and the candidate video frame in which the tank truck is first detected is used to determine the parking time of the tank truck; When the parking time does not reach the specified time, the step of acquiring a plurality of synchronized video frames is continued until the parking time reaches the specified time; When the parking time reaches the specified time, all reference objects are detected to see whether they carry staff, anti-slip mats and warning signs, so as to determine whether a number of synchronous video frames carry all target objects in the target object set.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the target detection-based operation behavior specification detection method as described in any one of claims 1-6.

9. A terminal device, characterized in that: include: Processor, memory and communication bus; The memory stores a computer-readable program executable by the processor; The communication bus realizes the connection and communication between the processor and the memory; When the processor executes the computer-readable program, the steps in the target detection-based operation behavior specification detection method as described in any one of claims 1 to 6 are implemented.