Target tracking-based dangerous behavior detection method, related method and device

Abstract

The invention discloses a dangerous behavior detection method based on target tracking, a related method and a related device, and the method comprises the steps: carrying out the mapping transformation of a pedestrian video based on an improved Gaussian model, and obtaining a transformation image sequence; extracting a frame from the pedestrian video as an initial background image; dividing the pedestrian video and the transformed image sequence into a plurality of fragment videos and a plurality of fragment transformed image sequences; analyzing the plurality of fragment videos and the plurality of fragment transformation image sequences based on the initial background image and a target tracking algorithm, and splicing paths to obtain a first target tracking path and a second target tracking path; judging whether the distance between the first target tracking path and the second target tracking path is greater than a preset threshold value or not: if yes, updating the background image to obtain a new first target tracking path, and executing the judgment step again; if not, taking the first target tracking path as a target path; and obtaining a dangerous behavior detection result based on the preset safe action path and the target path.

Description

Technical Field

[0001] The present invention relates to a method for detecting dangerous behaviors based on target tracking, related methods and devices. Background Art

[0002] The construction of engineering projects is a high-risk industry. With the gradual deepening of people's understanding of the importance of safety work, the overall safety management level has been improved to some extent, but the situation of work safety is still not optimistic. Due to the continuous expansion of the scale of engineering project construction, the construction technology has become increasingly complex, the construction technical requirements have become higher and higher, and the difficulty has increased, which often brings some problems and hidden dangers that have never been encountered in safety management work. Coupled with the fact that large-scale projects are often subcontracted layer by layer from the general contractor to the subcontractor, to the professional subcontractor and the labor subcontractor, it has increased the difficulty of safety supervision. In this situation, the risk of the project area is determined according to the risk value of the work permit to analyze and evaluate the status of HSE (Health, Safety, and Environment), and a risk warning is given in real time. Through the real-time collection of HSE data work permits, a dynamic programming method for establishing an evaluation model corresponding to high, medium, and low risks in the area is used to monitor the warning index data of dynamic monitoring. Currently, dangerous behavior detection is mainly implemented based on the target tracking algorithm. Summary of the Invention

[0003] In order to obtain more accurate dangerous behavior detection results, embodiments of the present invention provide a method for detecting dangerous behaviors based on target tracking, related methods and devices.

[0004] In a first aspect, an embodiment of the present invention provides a method for detecting dangerous behaviors based on target tracking, the method comprising:

[0005] Performing mapping transformation on the acquired pedestrian video based on an improved Gaussian model to obtain a sequence of transformed images;

[0006] Extracting a frame from the pedestrian video as an initial background image;

[0007] Based on a preset window length and a preset step size, dividing the pedestrian video and the sequence of transformed images into a plurality of segment videos and a plurality of segment transformed image sequences respectively; the preset window length is equal to the preset step size plus one;

[0008] Analyzing the plurality of segment videos and the plurality of segment transformed image sequences respectively based on the initial background image and a target tracking algorithm to obtain a plurality of segment video paths and a plurality of segment transformed image paths;

[0009] Based on the plurality of segment video paths and the plurality of segment transformed image paths, splicing to obtain a first target tracking path and a second target tracking path respectively;

[0010] Determine whether the distance between the first target tracking path and the second target tracking path is greater than a first preset threshold:

[0011] If so, based on the initial background image, obtain a new background image corresponding to the multiple segment videos;

[0012] Based on each segment video and the corresponding new background image, obtain a new segment video path, splice the obtained multiple new segment video paths to obtain a new first target tracking path, and re - execute the above determination step;

[0013] If not, use the first target tracking path as the target path;

[0014] Based on the obtained preset safe action path and the target path, obtain a dangerous behavior detection result.

[0015] In one or some alternative embodiments of the embodiments of the present application, the obtaining a new background image corresponding to the multiple segment videos based on the initial background image includes:

[0016] For every two adjacent segment videos, perform average calculation respectively to obtain a corresponding average image;

[0017] Perform background difference operations on the two average images and the initial background image respectively to obtain two background difference images;

[0018] Determine whether the similarity between the two background difference images is less than a second preset threshold:

[0019] If so, fuse the average image corresponding to the segment video with a later time sequence in the two adjacent segment videos with the initial background image to obtain a new background image corresponding to this segment video;

[0020] If not, use the initial background image as the new background image corresponding to this segment video.

[0021] In one or some alternative embodiments of the embodiments of the present application, the mapping transformation of the obtained pedestrian video based on the improved Gaussian model to obtain a transformed image sequence includes:

[0022] For each frame image of the pedestrian video, calculate the Gaussian kernel of each pixel point;

[0023] Normalize the Gaussian kernel of each pixel point;

[0024] Multiply the Gaussian kernel of each pixel point element - by - element with the regional pixel value corresponding to the pixel point and then sum to obtain the value of the pixel point after transformation;

[0025] The transformed image sequence is obtained according to the transformed values of each pixel point in each frame image of the pedestrian video.

[0026] In one or some optional embodiments of the embodiments of the present application, calculating the Gaussian kernel for each pixel point for each frame image of the pedestrian video includes:

[0027] For each frame image of the pedestrian video, based on the following formula 1, calculate each value in the Gaussian kernel for each pixel point in the Gaussian kernel:

[0028]

[0029] In the formula, g(x, y, I) represents the value of the pixel point (x, y) in the Gaussian kernel, and σ x,y represents the pixel mapping parameter of the pixel point (x, y), and I represents the pixel value of the pixel point (x, y);

[0030] Among them, the pixel mapping parameter of the pixel point (x, y) is calculated based on formula 2:

[0031]

[0032] In the formula, σ x,y represents the pixel mapping parameter of the pixel point (x, y), and N*M represents the size of the Gaussian kernel.

[0033] In one or some optional embodiments of the embodiments of the present application, the distance between the first target tracking path and the second target tracking path is calculated by the following method:

[0034] Take a preset number of feature points from the first target tracking path and the second target tracking path respectively to obtain a preset number of feature point pairs;

[0035] Calculate the mean value of the distances between each pair of feature points to obtain the distance between the first target tracking path and the second target tracking path.

[0036] In a second aspect, an HSE management method based on risk items and hazard sources provided by an embodiment of the present invention includes:

[0037] Based on multi-modal data collection, a pedestrian video is obtained;

[0038] According to the pedestrian video, the dangerous behavior detection result obtained by the dangerous behavior detection method according to any one of claims 1-5;

[0039] According to the dangerous behavior detection result, implement a risk warning action.

[0040] Thirdly, an embodiment of the present invention provides a dangerous behavior detection device based on target tracking, and the device includes:

[0041] A first transformation module, configured to perform mapping transformation on the acquired pedestrian video based on an improved Gaussian model to obtain a sequence of transformed images;

[0042] A first extraction module, configured to extract one frame from the pedestrian video as an initial background image;

[0043] A first partitioning module, configured to partition the pedestrian video and the sequence of transformed images into a plurality of segment videos and a plurality of segment transformed image sequences respectively based on a preset window length and a preset step length; the preset window length is equal to the preset step length plus one;

[0044] A first analysis module, configured to analyze the plurality of segment videos and the plurality of segment transformed image sequences respectively based on the initial background image and a target tracking algorithm to obtain a plurality of segment video paths and a plurality of segment transformed image paths;

[0045] A first splicing module, configured to splice the plurality of segment video paths and the plurality of segment transformed image paths respectively to obtain a first target tracking path and a second target tracking path;

[0046] A first judgment module, configured to judge whether the distance between the first target tracking path and the second target tracking path is greater than a first preset threshold:

[0047] A first update module, configured to, if the distance between the first target tracking path and the second target tracking path is greater than the first preset threshold, obtain a new background image corresponding to the plurality of segment videos based on the initial background image; obtain new segment video paths based on each segment video and the corresponding new background image, splice the obtained plurality of new segment video paths to obtain a new first target tracking path, and re - execute the above judgment step;

[0048] A second update module, configured to, if the distance between the first target tracking path and the second target tracking path is not greater than the first preset threshold, use the first target tracking path as the target path;

[0049] A first acquisition module, configured to obtain a dangerous behavior detection result based on a preset safe action path and the target path.

[0050] Fourthly, an embodiment of the present invention provides an HSE management device based on risk items and hazard sources, and the device includes:

[0051] A first acquisition module, configured to obtain a pedestrian video based on multi - modal data acquisition;

[0052] The first transformation module is used to perform mapping transformation on the acquired pedestrian video based on an improved Gaussian model to obtain a sequence of transformed images;

[0053] The first extraction module is used to extract a frame from the pedestrian video as the initial background image;

[0054] The first partitioning module is used to partition the pedestrian video and the sequence of transformed images into multiple segment videos and multiple sequences of segment transformed images respectively based on a preset window length and a preset step size; the preset window length is equal to the preset step size plus one;

[0055] The first analysis module is used to analyze the multiple segment videos and the multiple sequences of segment transformed images respectively based on the initial background image and a target tracking algorithm to obtain multiple segment video paths and multiple segment transformed image paths;

[0056] The first splicing module is used to splice the multiple segment video paths and the multiple segment transformed image paths respectively to obtain a first target tracking path and a second target tracking path;

[0057] The first judgment module is used to judge whether the distance between the first target tracking path and the second target tracking path is greater than a first preset threshold:

[0058] The first update module is used to, if the distance between the first target tracking path and the second target tracking path is greater than the first preset threshold, obtain a new background image corresponding to the multiple segment videos based on the initial background image; obtain new segment video paths based on each segment video and the corresponding new background image, splice the obtained multiple new segment video paths to obtain a new first target tracking path, and re - execute the above judgment step;

[0059] The second update module is used to, if the distance between the first target tracking path and the second target tracking path is not greater than the first preset threshold, take the first target tracking path as the target path;

[0060] The first acquisition module is used to obtain a dangerous behavior detection result based on a preset safe action path and the target path;

[0061] The first implementation module is used to implement a risk warning action according to the dangerous behavior detection result.

[0062] In a fifth aspect, an embodiment of the present invention provides a computer - readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the above - mentioned dangerous behavior detection method based on target tracking, and / or, the HSE management method based on risk items and hazard sources.

[0063] Sixth aspect, an embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the above-mentioned dangerous behavior detection method based on target tracking, and / or the HSE management method based on risk items and hazard sources.

[0064] Seventh aspect, an embodiment of the present invention provides a computer program product containing instructions. When the computer program product runs on a computer device, it causes the computer device to execute the above-mentioned dangerous behavior detection method based on target tracking, and / or the HSE management method based on risk items and hazard sources.

[0065] Eighth aspect, an embodiment of the present invention provides a chip. The chip includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run a computer program or instructions to implement the above-mentioned dangerous behavior detection method based on target tracking, and / or the HSE management method based on risk items and hazard sources.

[0066] The beneficial effects of the above technical solutions provided by the embodiments of the present invention at least include:

[0067] The dangerous behavior detection method based on target tracking provided by the embodiment of the present invention maps and transforms the pedestrian video by using an improved Gaussian model, enhances the features of the image sequence, makes the target easier to track, improves the fidelity of the image after the mapping transformation. At the same time, by processing the pedestrian video and the transformed image sequence in the way of window sliding according to a preset step size, the first target tracking path and the second target tracking path are obtained, and the comparison result of the first target tracking path and the second target tracking path is used to determine and update the background image corresponding to each segment video of the pedestrian video, and the final target path is determined, so that the target path can better adapt to the background change and improve the accuracy of target tracking. Therefore, based on a more accurate target path, the dangerous behavior detection result is obtained, early warning and control of dangerous behaviors are carried out, the accuracy of risk analysis is improved, and the effectiveness of safety management is improved.

[0068] Other features and advantages of the present invention will be described in the following specification, and part of them will become obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained by the structures specifically pointed out in the written specification, claims, and drawings.

[0069] The technical solutions of the present invention will be further described in detail below through the drawings and embodiments. Description of the Drawings

[0070] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the accompanying drawings:

[0071] Figure 1 It is a schematic diagram of the steps of the dangerous behavior detection method based on target tracking provided by an embodiment of the present invention;

[0072] Figure 2 It is a schematic diagram of the steps of the HSE management method based on risk items and hazard sources provided by an embodiment of the present invention;

[0073] Figure 3 It is a schematic diagram of the process of the safety assessment and early warning system provided by an embodiment of the present invention;

[0074] Figure 4 It is a schematic diagram of the safety assessment and early warning system provided by an embodiment of the present invention;

[0075] Figure 5 It is a schematic diagram of the structure of the dangerous behavior detection device based on target tracking provided by an embodiment of the present application;

[0076] Figure 6 It is a schematic diagram of the structure of the HSE management device based on risk items and hazard sources provided by an embodiment of the present application. Detailed implementation manners

[0077] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system structures and technologies are put forward in order to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0078] It should be understood that when used in the specification of the present application and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0079] It should also be understood that the term "and / or" used in the specification of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0080] As used in the specification of this application and the appended claims, the term "if" may be construed, depending on the context, as "when", "once", "in response to determining", or "in response to detecting". Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be construed, depending on the context, to mean "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]".

[0081] In addition, in the description of the specification of this application and the appended claims, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0082] The reference to "one embodiment" or "some embodiments" or the like described in the specification of this application means that a specific feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "comprising", "including", "having", and their variants all mean "including but not limited to", unless otherwise specifically emphasized in another way.

[0083] It should be understood that the magnitudes of the sequence numbers of the steps in the following embodiments do not mean the order of execution is prior or subsequent. The order of execution of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of this application.

[0084] To illustrate the technical solution of this application, the following specific embodiments are used for illustration.

[0085] The inventors found that in the prior art, the generalization ability of the target tracking algorithm is restricted by some background scene changes. Using the same background all the time will result in inaccurate tracking, making it difficult to adapt to all situations, leading to performance degradation and insufficient accuracy, and resulting in inaccurate dangerous behavior detection results. Based on this, through further research and development, the inventors made this invention and provided a dangerous behavior detection method, related methods, and devices based on target tracking.

[0086] Embodiment 1

[0087] The embodiment of the present invention provides a dangerous behavior detection method based on target tracking. Referring to Figure 1 as shown, the method includes:

[0088] S101: Perform a mapping transformation on the acquired pedestrian video based on an improved Gaussian model to obtain a sequence of transformed images.

[0089] In the embodiment of the present application, in the above step S101, performing a mapping transformation on the acquired pedestrian video based on an improved Gaussian model to obtain a sequence of transformed images includes:

[0090] For each frame image of the pedestrian video, calculate the Gaussian kernel of each pixel point;

[0091] Normalize the Gaussian kernel of each pixel point;

[0092] Multiply each value of the Gaussian kernel of each pixel point element - by - element with the pixel value of the corresponding region of the pixel point and then sum to obtain the value after transformation of the pixel point;

[0093] According to the values after transformation of each pixel point in each frame image of the pedestrian video, obtain a sequence of transformed images.

[0094] Among them, for each frame image of the pedestrian video, calculating the Gaussian kernel of each pixel point includes:

[0095] For each frame image of the pedestrian video, based on the following formula 1, calculate each value in the Gaussian kernel of each pixel point in the Gaussian kernel:

[0096]

[0097] In the formula, g(x,y,I) represents the value of the pixel point (x,y) in the Gaussian kernel, σ x,y represents the pixel mapping parameter of the pixel point (x,y), and I represents the pixel value of the pixel point (x,y);

[0098] Among them, the pixel mapping parameter of the pixel point (x,y) is calculated based on formula 2:

[0099]

[0100] In the formula, σ x,y represents the pixel mapping parameter of the pixel point (x,y), and N*M represents the size of the Gaussian kernel.

[0101] In the embodiments of the present application, each frame image of the pedestrian video is traversed, and for each pixel point in each frame image, the Gaussian kernel corresponding to the pixel point is calculated based on the above formulas (1) and (2); the Gaussian kernels calculated for each pixel point are normalized to ensure that the sum of the weights of the Gaussian kernels is 1, maintain the brightness information of the transformed image, and prevent information loss; the Gaussian kernel corresponding to each pixel point is multiplied element by element with the pixel values of the local area centered on the pixel point, and the results of the multiplication are summed to obtain the value of the pixel point after transformation; according to the values of each pixel point in each frame image after transformation, a sequence of transformed images is obtained.

[0102] In the embodiments of the present application, the pedestrian video is subjected to a mapping transformation through an improved Gaussian model, and the obtained sequence of transformed images fuses the position features and pixel features to calculate the pixel mapping parameters of the Gaussian model, which are used as the standard for quality evaluation. Compared with the traditional Gaussian function that only considers the position relationship, the improved Gaussian model proposed in this method simultaneously considers the position information and pixel information, improves the fidelity of the image after the mapping transformation, and at the same time, can also achieve the technical effect of noise reduction for the image. It lays a foundation for the accuracy of subsequent target tracking.

[0103] S102: Extract a frame from the pedestrian video as the initial background image.

[0104] S103: Based on a preset window length and a preset step size, divide the pedestrian video and the sequence of transformed images into a plurality of segment videos and a plurality of segment sequences of transformed images respectively; the preset window length is equal to the preset step size plus one.

[0105] In the embodiments of the present application, a frame is extracted from the pedestrian video as the initial background image. The initial background image can be randomly selected from the pedestrian video, or the first frame of the pedestrian video can be directly used as the initial background image.

[0106] Based on the preset window length and the preset step size, starting from the first frame, the pedestrian video is divided into a plurality of segment videos, and the preset window length is equal to the preset step size plus one, that is, if the preset window length is K, the preset step size is K - 1, and the preset window length should be set as a positive integer greater than 1. The pedestrian video will be divided into segment videos composed of frames from 1 to K, segment videos composed of frames from K to 2K - 1, and so on, dividing the pedestrian video into a plurality of segment videos. The sequence of transformed images is divided in the same way to obtain a plurality of segment sequences of transformed images.

[0107] S104: Analyze the plurality of segment videos and the plurality of segment sequences of transformed images respectively based on the initial background image and a target tracking algorithm to obtain a plurality of segment video paths and a plurality of segment transformed image paths.

[0108] S105: Based on the multiple segment video paths and the multiple segment transformed image paths, respectively splice to obtain a first target tracking path and a second target tracking path.

[0109] In the embodiments of the present application, each frame image in the segment video and the initial background image are input into a pre-trained convolutional neural network to extract image features, which include semantic information and local structures in the image, etc.; based on the extracted image features, a target detection algorithm is used to obtain the initial position of the target, and a target tracking algorithm is used for target tracking to obtain the new position of the target; the above-mentioned steps of inputting the convolutional neural network to extract features and using the target tracking algorithm to update the target position are performed for each frame in a segment video to obtain the segment video path corresponding to the segment video; and so on, all segment videos and segment transformed image sequences are processed in the same way to obtain multiple segment video paths and multiple segment transformed image paths.

[0110] Splice the segment video paths corresponding to every two adjacent segment videos among the multiple segment videos. There are overlapping frames between two adjacent segment videos, so the transition between the two segment video paths is relatively smooth, avoiding unnatural jumps at the splicing points. After splicing all the segment video paths, a first target tracking path is obtained. And so on, splice the segment transformed image paths corresponding to every two adjacent segment transformed image sequences among the multiple segment transformed image sequences to obtain a second target tracking path.

[0111] S106: Determine whether the distance between the first target tracking path and the second target tracking path is greater than a first preset threshold: if so, execute step S107; if not, execute step S108;

[0112] In the embodiments of the present application, in the above step S106, determine whether the distance between the first target tracking path and the second target tracking path is greater than a first preset threshold, and the distance between the first target tracking path and the second target tracking path is calculated in the following manner:

[0113] Take a preset number of feature points from the first target tracking path and the second target tracking path respectively to obtain a preset number of feature point pairs;

[0114] Calculate the mean value of the distances between each feature point pair to obtain the distance between the first target tracking path and the second target tracking path.

[0115] In the embodiments of the present application, calculating the distance between the first target tracking path and the second target tracking path specifically includes the following steps: randomly select a preset number of feature points from the first target tracking path, find the corresponding feature points in the second target tracking path, and form a preset number of feature point pairs. The two feature points in each feature point pair need to have the same features in the first target tracking path and the second target tracking path. For example, the two feature points are respectively the first inflection points of the first target tracking path and the second target tracking path, or the slopes of the two feature points are the same, etc.; calculate the distance of each feature point pair, and the distance calculation can use the Euclidean distance. Average the distances of each feature point pair to obtain the distance between the first target tracking path and the second target tracking path.

[0116] S107: Based on the initial background image, obtain a new background image corresponding to the multiple segment videos; based on each segment video and the corresponding new background image, obtain a new segment video path, splice the obtained multiple new segment video paths to obtain a new first target tracking path, and re-execute the judgment step of S106 above;

[0117] S108: Use the first target tracking path as the target path;

[0118] In the embodiments of the present application, in the above step S107, obtaining a new background image corresponding to the multiple segment videos based on the initial background image includes:

[0119] For every two adjacent segment videos, perform average calculation respectively to obtain the corresponding average image;

[0120] Perform background difference operations on the two average images and the initial background image respectively to obtain two background difference images;

[0121] Judge whether the similarity of the two background difference images is less than a second preset threshold:

[0122] If so, fuse the average image corresponding to the segment video with a later time sequence in the two adjacent segment videos and the initial background image to obtain a new background image corresponding to the segment video;

[0123] If not, use the initial background image as the new background image corresponding to the segment video.

[0124] In the embodiments of the present application, if the distance between the first target tracking path and the second target tracking path is not greater than the first preset threshold, it is necessary to replace the background image of the segment image. The specific steps are as follows: For every two adjacent segment videos, perform video frame averaging on the two segment videos respectively, that is, accumulate and average the pixel values at the same positions in all frames within a segment video to obtain the average images corresponding to the two segment videos; perform background difference operations on the two average images and the initial background image respectively, that is, subtract the average image from the initial background image to obtain the background difference images corresponding to the two average images; calculate the similarity of the two background difference images, and similarity map evaluation algorithms such as the Structural Similarity Index (SSIM) and histogram comparison can be used to obtain the similarity result of the two background difference images; determine whether the similarity result is less than the second preset threshold: if so, fuse the average image corresponding to the segment video with a later time sequence in the two adjacent segment videos with the initial background image to obtain the new background image corresponding to this segment video, if not, use the initial background image as the new background image of the segment video with a later time sequence in the two adjacent segment videos.

[0125] After updating the background images of all adjacent segment videos in the pedestrian video, based on each segment video and the corresponding new background image, obtain new segment video paths, and splice the multiple new segment video paths to obtain a new first target tracking path. The specific implementation method refers to steps S104 and S105. Re-execute step S106 based on the new first target tracking path and the second target tracking path until the distance between the first target tracking path and the second target tracking path is not greater than the first preset threshold, and use the first target tracking path as the target path.

[0126] In the embodiments of the present application, by updating the background image corresponding to each video segment, better adaptation to background changes is achieved, thereby improving the accuracy of target tracking. By continuously updating the background model, this method can capture dynamic changes in the environment, such as lighting, occlusion, or scene changes, making the target tracking algorithm more robust. This real-time adaptation mechanism helps to ensure that target tracking can work stably in complex and dynamic scenarios.

[0127] S109: Obtain a dangerous behavior detection result based on the acquired preset safe action path and the target path.

[0128] In the embodiments of the present application, compare the acquired preset safe action path with the target path. If the comparison result has a large difference and the target path reaches a preset unreachable location, it is determined that the pedestrian target in the pedestrian video has a dangerous behavior, and the dangerous behavior detection result is "exists".

[0129] In the embodiments of the present application, the schematic diagram of the safety evaluation and early warning system implemented based on the present method is as follows Figure 3 shown. The input is HSE-related parameters, including safety conditions and status records, corresponding to the preset safety action paths and pedestrian videos in the above text, etc.; the evaluation and early warning model in the operation part includes HSE data statistical analysis and JSA analysis, that is, corresponding to the dangerous behavior detection method based on target tracking; the output evaluation and early warning report includes early warning prompts and evaluation reports.

[0130] The schematic diagram of the safety evaluation and early warning system is as follows Figure 4 shown, including three parts: data collection, HSE status evaluation and early warning, and data visualization, corresponding to the three parts in Figure 3 respectively. Among them, the functions in data collection include personnel management, vehicle / machinery management, HSE inspection management, HSE examination management, green and civilized construction, work permit management, JSA management, safe man-hours, vehicle / machinery driving mileage, accident events, violation inspections, identification of unsafe behaviors, passing rate of personnel examinations, pre-operation safety analysis form, work permit application; the functions and related data in HSE status evaluation and early warning include million man-hours / vehicle mileage accident event evaluation report, JSA risk library, pre-operation safety analysis form, project area risk level early warning, HSE priority evaluation of subcontractors, subcontractor score sheet, subcontractor management regulations, subcontractor score deduction item early warning; the data visualization includes evaluation reports, one map of project area risks, and 3D models.

[0131] In the embodiments of the present application, the above safety evaluation and early warning system can be simplified into seven parts: data collection, specific function management, HSE status and risk early warning model, HSE behavior intelligent recognition model, HSE problem closed-loop tracking, HSE reward and punishment management, and HSE evaluation report.

[0132] The data collection part mainly includes: automatically counting the total number of on-site personnel, and through the face recognition access control system, realizing one-stop digital safety control of attendance, personnel qualifications, identity review, physical examination, practical training, mobile terminal training, examination, automatic access authorization, and cap label printing; the total number of vehicles, vehicle entry application approval, sorting vehicle information and making two-dimensional codes for printing, and pasting the two-dimensional codes at obvious positions on the vehicle body when handling vehicle entry passes, and the real-time status and information of the vehicle can be viewed by scanning the two-dimensional code label through the mobile phone APP; indicators such as the number of tools, safe man-hours, the number of violations, the number of accidents, violation rate, accident rate, etc., as the basic data statistics for safety status analysis.

[0133] The specific function management includes four parts: examination management, work permit management, inspection management, and emergency management. Examination management means organizing the HSE professional knowledge and operation knowledge in the construction operation process into a question bank and importing it into the system. When personnel enter the site for an exam, HSE managers select the corresponding question bank, allow the on-site training personnel to take the online exam, and automatically calculate the scores to determine whether the exam is passed. Work permit management is the digitization of the work permit management process in the direct operation link. Through integration, managers can issue work permits according to the operation information provided by the system and the on-site situation, bind the electronic work permit with its related operation area and the corresponding manager, integrate the data into the platform, and penetrate layer by layer to display all types of on-site operation information, supporting the online query and supervision of managers' performance. And a single map for high-risk operations is implemented in the model. Inspection management is to implement an electronic point system for contractors for the assessment of violation behaviors. The points are assigned to individuals. When a person reaches the full score, the access control authorization is automatically stopped, and they need to be re-educated and pass the assessment before they can enter the site. For serious violation behaviors, the contractor's personnel will be directly blacklisted and cleared from the site. Develop a grid-based intelligent supervision function for the performance of full-time safety management personnel of the owner, general contractor, supervisor, and construction party to verify the performance of their duties, and conduct statistics on personnel location, trajectory tracking, and online duration, and regularly generate an evaluation report on personnel performance. Emergency management is to enter the names, quantities, usage methods, dosages, and storage locations of first aid equipment and medicines into the system and push them to the mobile terminal. All personnel entering the site can quickly view the corresponding first aid supplies and contact the administrator for use in case of an emergency.

[0134] The HSE status and risk warning model establish an evaluation model corresponding to high, medium, and low risks in the region through HSE data collection, analyze and evaluate the HSE status by determining the risk of the project area based on the risk value of the work permit, and give real-time risk warnings.

[0135] The HSE behavior intelligent recognition model applies a hazard behavior detection method based on object tracking, deploys the HSE behavior intelligent recognition model of construction personnel to an industrial control computer, connects the hardware devices, and conducts debugging. After debugging, deploy the entire system to the site for intelligent recognition of personnel behaviors, record, analyze, and feedback the recognition results, and optimize the behavior intelligent recognition model.

[0136] For the closed-loop tracking of HSE issues, when a violation behavior is encountered, issue a violation notice in the form of video, pictures, text, voice, etc. After being reviewed by the HSE person in charge of the branch, it is received by the contractor's person in charge and the rectification result is feedback to achieve closed-loop management.

[0137] HSE reward and punishment management, as an important means of safety management, in order to establish a management atmosphere of promoting good and suppressing evil, distinguishes the level and severity of violations on site, rewards good safety behaviors, punishes illegal behaviors, and standardizes safe construction behaviors.

[0138] The HSE evaluation report automatically analyzes the on-site safety status and evaluation indicators through business contents such as entry safety education, vehicle entry registration, approval of work tickets, on-site inspections and problem rectification, and compiles the evaluation report according to the corresponding content requirements.

[0139] In the embodiments of this application, the construction is linked all day and all week, and the risk items and hazard sources are analyzed throughout the whole cycle, which truly reflects the safety status of the construction site, predicts the trend, and warns and controls the upcoming risks, which improves the accuracy of risk analysis and the effectiveness of safety management. Combining the hazard source research theory, the intrinsic safety system theory, the accident cause theory, etc., the triggering mechanism of the construction hazard source is analyzed from the source of the accident, and the risk index is established to escort safe construction.

[0140] Embodiment 2

[0141] Based on the same inventive concept, the embodiment of the present invention also provides a HSE management method based on risk items and hazard sources, referring to Figure 2 As shown, the method includes:

[0142] S201: Obtain pedestrian video based on data collection;

[0143] S202: mapping and transforming the acquired pedestrian video based on the improved Gaussian model to obtain a transformed image sequence;

[0144] S203: extracting a frame from the pedestrian video as an initial background image;

[0145] S204: Based on a preset window length and a preset step length, the pedestrian video and the transformed image sequence are divided into a plurality of segment videos and a plurality of segment transformed image sequences respectively; the preset window length is equal to the preset step length plus one;

[0146] S205: Analyze the multiple segment video and multiple segment transformation image sequences respectively based on the initial background image and the target tracking algorithm to obtain multiple segment video paths and multiple segment transformation image paths;

[0147] S206: Based on the multiple segment video paths and the multiple segment transformation image paths, respectively splicing to obtain a first target tracking path and a second target tracking path;

[0148] S207: Determine whether the distance between the first target tracking path and the second target tracking path is greater than a first preset threshold:

[0149] S208: If so, based on the initial background image, obtain a new background image corresponding to the multiple segment videos;

[0150] Based on each segment video and the corresponding new background image, obtain new segment video paths, splice the obtained multiple new segment video paths to obtain a new first target tracking path, and re - execute the above judgment step;

[0151] S209: If not, use the first target tracking path as the target path;

[0152] S210: Based on the obtained preset safe action path and the target path, obtain a dangerous behavior detection result.

[0153] S211: Implement a risk warning action according to the dangerous behavior detection result.

[0154] Embodiment III

[0155] Based on the same inventive concept, an embodiment of the present invention further provides a dangerous behavior detection device based on target tracking. Referring to Figure 5 as shown, the device includes:

[0156] The first transformation module 101 is used to perform a mapping transformation on the obtained pedestrian video based on an improved Gaussian model to obtain a transformed image sequence;

[0157] The first extraction module 102 is used to extract a frame from the pedestrian video as the initial background image;

[0158] The first partitioning module 103 is used to partition the pedestrian video and the transformed image sequence into multiple segment videos and multiple segment transformed image sequences respectively based on a preset window length and a preset step length; the preset window length is equal to the preset step length plus one;

[0159] The first analysis module 104 is used to analyze the multiple segment videos and multiple segment transformed image sequences respectively based on the initial background image and a target tracking algorithm to obtain multiple segment video paths and multiple segment transformed image paths;

[0160] The first splicing module 105 is used to splice and obtain a first target tracking path and a second target tracking path respectively based on the multiple segment video paths and the multiple segment transformed image paths;

[0161] The first judgment module 106 is used to judge whether the distance between the first target tracking path and the second target tracking path is greater than a first preset threshold:

[0162] The first update module 107 is configured to, if the distance between the first target tracking path and the second target tracking path is greater than a first preset threshold, obtain a new background image corresponding to the multiple segment videos based on the initial background image; obtain new segment video paths based on each segment video and the corresponding new background image, splice the obtained multiple new segment video paths to obtain a new first target tracking path, and re - execute the above - mentioned judgment step;

[0163] The second update module 108 is configured to, if the distance between the first target tracking path and the second target tracking path is not greater than the first preset threshold, use the first target tracking path as the target path;

[0164] The first acquisition module 109 is configured to obtain a dangerous behavior detection result based on a preset safe action path and the target path.

[0165] Embodiment 4

[0166] Based on the same inventive concept, an embodiment of the present invention further provides an HSE management device based on risk items and hazard sources. Referring to Figure 6 as shown, the device includes:

[0167] The first acquisition module 201 is configured to obtain a pedestrian video based on multi - modal data acquisition;

[0168] The first transformation module 202 is configured to perform mapping transformation on the obtained pedestrian video based on an improved Gaussian model to obtain a sequence of transformed images;

[0169] The first extraction module 203 is configured to extract one frame from the pedestrian video as the initial background image;

[0170] The first partitioning module 204 is configured to partition the pedestrian video and the sequence of transformed images into multiple segment videos and multiple sequences of segment transformed images respectively based on a preset window length and a preset step size; the preset window length is equal to the preset step size plus one;

[0171] The first analysis module 205 is configured to analyze the multiple segment videos and the multiple sequences of segment transformed images respectively based on the initial background image and a target tracking algorithm to obtain multiple segment video paths and multiple segment transformed image paths;

[0172] The first splicing module 206 is configured to splice the multiple segment video paths and the multiple segment transformed image paths respectively to obtain a first target tracking path and a second target tracking path;

[0173] The first judgment module 207 is configured to judge whether the distance between the first target tracking path and the second target tracking path is greater than a first preset threshold:

[0174] The first update module 208 is configured to, if the distance between the first target tracking path and the second target tracking path is greater than a first preset threshold, obtain a new background image corresponding to the multiple segment videos based on the initial background image; obtain new segment video paths based on each segment video and the corresponding new background image, splice the obtained multiple new segment video paths to obtain a new first target tracking path, and re - execute the above - mentioned judgment step;

[0175] The second update module 209 is configured to, if the distance between the first target tracking path and the second target tracking path is not greater than the first preset threshold, use the first target tracking path as the target path;

[0176] The first acquisition module 210 is configured to obtain a dangerous behavior detection result based on a preset safe action path and the target path;

[0177] The first implementation module 211 is configured to implement a risk warning action according to the dangerous behavior detection result.

[0178] Embodiment Five

[0179] Based on the same inventive concept, an embodiment of the present invention further provides a computer - readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the dangerous behavior detection method based on target tracking described in Embodiment One above, and / or the HSE management method based on risk items and hazard sources described in Embodiment Two above.

[0180] Embodiment Six

[0181] Based on the same inventive concept, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the dangerous behavior detection method based on target tracking described in Embodiment One above, and / or the HSE management method based on risk items and hazard sources described in Embodiment Two above.

[0182] Embodiment Seven

[0183] Based on the same inventive concept, an embodiment of the present invention further provides a computer program product containing instructions. When the computer program product runs on a computer device, it causes the computer device to execute the dangerous behavior detection method based on target tracking described in Embodiment One above, and / or the HSE management method based on risk items and hazard sources described in Embodiment Two above.

[0184] Embodiment Eight

[0185] Based on the same inventive concept, an embodiment of the present invention further provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is configured to run a computer program or instruction to implement the method for detecting dangerous behaviors based on target tracking described in the first embodiment above, and / or the HSE management method based on risk items and hazard sources described in the second embodiment above.

[0186] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0187] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the specified function in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0188] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the specified function in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0189] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the specified function in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0190] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. A method for detecting dangerous behaviors based on object tracking, characterized in that, it includes: Performing mapping transformation on the acquired pedestrian video based on an improved Gaussian model to obtain a sequence of transformed images; Extracting one frame from the pedestrian video as the initial background image; Based on a preset window length and a preset step size, dividing the pedestrian video and the sequence of transformed images into multiple segment videos and multiple sequences of segment transformed images respectively; The preset window length is equal to the preset step size plus one; Analyzing the multiple segment videos and the multiple sequences of segment transformed images respectively based on the initial background image and an object tracking algorithm to obtain multiple segment video paths and multiple segment transformed image paths; Based on the multiple segment video paths and the multiple segment transformed image paths, splicing them respectively to obtain a first object tracking path and a second object tracking path; Judging whether the distance between the first object tracking path and the second object tracking path is greater than a first preset threshold: If so, based on the initial background image, obtaining new background images corresponding to the multiple segment videos; Based on each segment video and the corresponding new background image, obtaining new segment video paths, splicing the obtained multiple new segment video paths to obtain a new first object tracking path, and re - executing the above judgment step; If not, taking the first object tracking path as the target path; Based on the acquired preset safe action path and the target path, obtaining a dangerous behavior detection result.

2. The method according to claim 1, characterized in that, The obtaining new background images corresponding to the multiple segment videos based on the initial background image includes: For every two adjacent segment videos, performing average calculation respectively to obtain corresponding average images; Performing background difference operations on the two average images and the initial background image respectively to obtain two background difference images; Judging whether the similarity of the two background difference images is less than a second preset threshold: If so, fusing the average image corresponding to the segment video with a later time sequence among the two adjacent segment videos and the initial background image to obtain a new background image corresponding to this segment video; If not, taking the initial background image as the new background image corresponding to this segment video.

3. The method according to claim 1, characterized in that, The performing mapping transformation on the acquired pedestrian video based on an improved Gaussian model to obtain a sequence of transformed images includes: For each frame image of the pedestrian video, calculating the Gaussian kernel of each pixel point; Normalizing the Gaussian kernel of each pixel point; Multiplying the Gaussian kernel of each pixel point element - by - element with the pixel values of the region corresponding to the pixel point and then summing to obtain the transformed value of the pixel point; According to the transformed values of each pixel point in each frame image of the pedestrian video, obtaining the sequence of transformed images.

4. The method according to claim 3, characterized in that, The calculating the Gaussian kernel of each pixel point for each frame image of the pedestrian video includes: For each frame image of the pedestrian video, calculate each value in the Gaussian kernel for each pixel point in the Gaussian kernel based on the following formula 1: where g(x, y, I) represents the value of the pixel point (x, y) in the Gaussian kernel, and σ x,y represents the pixel mapping parameter of the pixel point (x, y), and I represents the pixel value of the pixel point (x, y); where the pixel mapping parameter of the pixel point (x, y) is calculated based on formula 2: where σ x,y represents the pixel mapping parameter of the pixel point (x, y), and N*M represents the size of the Gaussian kernel.

5. The method according to claim 1, characterized in that, the distance between the first target tracking path and the second target tracking path is calculated by the following method: Take a preset number of feature points from the first target tracking path and the second target tracking path respectively to obtain a preset number of feature point pairs; Calculate the mean value of the distances between each pair of feature points to obtain the distance between the first target tracking path and the second target tracking path.

6. An HSE management method based on risk items and hazard sources, characterized in that, the method includes: Based on multi-modal data collection, obtain a pedestrian video; According to the pedestrian video, obtain the dangerous behavior detection result based on the dangerous behavior detection method according to any one of claims 1-5; According to the dangerous behavior detection result, implement a risk warning action.

7. A dangerous behavior detection device based on target tracking, characterized in that, it includes: A first transformation module, configured to perform mapping transformation on the acquired pedestrian video based on an improved Gaussian model to obtain a sequence of transformed images; A first extraction module, configured to extract one frame from the pedestrian video as an initial background image; A first partitioning module, configured to partition the pedestrian video and the sequence of transformed images into a plurality of segment videos and a plurality of segment transformed image sequences respectively based on a preset window length and a preset step size; the preset window length is equal to the preset step size plus one; A first analysis module, configured to analyze the plurality of segment videos and the plurality of segment transformed image sequences respectively based on the initial background image and a target tracking algorithm to obtain a plurality of segment video paths and a plurality of segment transformed image paths; A first splicing module, configured to splice the plurality of segment video paths and the plurality of segment transformed image paths respectively to obtain a first target tracking path and a second target tracking path; A first judgment module, configured to judge whether the distance between the first target tracking path and the second target tracking path is greater than a first preset threshold: A first update module, configured to, if the distance between the first target tracking path and the second target tracking path is greater than the first preset threshold, obtain a new background image corresponding to the plurality of segment videos based on the initial background image; Based on each segment video and the corresponding new background image, obtain a new segment video path, splice the obtained plurality of new segment video paths to obtain a new first target tracking path, and re-execute the above judgment step; A second update module, configured to, if the distance between the first target tracking path and the second target tracking path is not greater than the first preset threshold, use the first target tracking path as the target path; A first acquisition module, configured to obtain a dangerous behavior detection result based on a preset safe action path and the target path.

8. An HSE management device based on risk items and hazard sources, characterized in that, it includes: The first acquisition module is used to obtain a pedestrian video based on multi-modal data acquisition; The first transformation module is used to perform mapping transformation on the obtained pedestrian video based on an improved Gaussian model to obtain a sequence of transformed images; The first extraction module is used to extract a frame from the pedestrian video as an initial background image; The first partitioning module is used to partition the pedestrian video and the sequence of transformed images into multiple segment videos and multiple segment transformed image sequences respectively based on a preset window length and a preset step size; the preset window length is equal to the preset step size plus one; The first analysis module is used to analyze the multiple segment videos and the multiple segment transformed image sequences respectively based on the initial background image and a target tracking algorithm to obtain multiple segment video paths and multiple segment transformed image paths; The first splicing module is used to splice the multiple segment video paths and the multiple segment transformed image paths respectively to obtain a first target tracking path and a second target tracking path; The first judgment module is used to judge whether the distance between the first target tracking path and the second target tracking path is greater than a first preset threshold: The first update module is used to, if the distance between the first target tracking path and the second target tracking path is greater than the first preset threshold, obtain a new background image corresponding to the multiple segment videos based on the initial background image; Based on each segment video and the corresponding new background image, obtain new segment video paths, splice the obtained multiple new segment video paths to obtain a new first target tracking path, and re-execute the above judgment step; The second update module is used to, if the distance between the first target tracking path and the second target tracking path is not greater than the first preset threshold, use the first target tracking path as the target path; The first acquisition module is used to obtain a dangerous behavior detection result based on a preset safe action path and the target path; The first implementation module is used to implement a risk warning action according to the dangerous behavior detection result.

9. A computer-readable storage medium, in which instructions are stored. When the instructions are run on a terminal, the terminal is made to execute the dangerous behavior detection method based on target tracking as described in any one of claims 1-5, and / or, the HSE management method based on risk items and hazard sources as described in claim 6.

10. A computer device, characterized in that, it includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the dangerous behavior detection method based on target tracking as described in any one of claims 1-5, and / or, the HSE management method based on risk items and hazard sources as described in claim 6.

11. A computer program product containing instructions. When the computer program product runs on a computer device, the computer device is made to execute the dangerous behavior detection method based on target tracking as described in any one of claims 1-5, and / or, the HSE management method based on risk items and hazard sources as described in claim 6.

12. A chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is configured to run a computer program or instructions to implement the method for detecting dangerous behaviors based on target tracking according to any one of claims 1-5, and / or the HSE management method based on risk items and hazard sources according to claim 6.

Images