A method, system, device and storage medium for video slicing of engineering inspection process

Abstract

The present application relates to the field of video slicing, and discloses a method, system, device and storage medium for video slicing of an engineering inspection process, the method comprising: passing a video through a multi-target detection model to obtain the position information of the equipment, the tester and the number display screen of the inspection object; transforming the position information and cropping the image with the transformed position information to obtain the first information and identify it to obtain the second information; determining the start and end time of the slice according to the appearance and disappearance of the inspection object; selecting the second information of the corresponding time node according to the start and end time to verify the authenticity of the test process, if it is true, slicing the video to be sliced, and binding the second information of the corresponding time node and the authenticity to the slice. The present application determines the start and end time of the slice by the appearance and disappearance of the inspection object, ensures that the dynamic changes of the inspection object are fully recorded, binds the second information to the slice, realizes the traceability of the inspection process, and avoids the problem of false experiments through authenticity verification.

Description

Technical Field

[0001] The present application relates to the field of video slicing, and in particular to a method, system, device and storage medium for video slicing of an engineering inspection process. Background Art

[0002] In modern construction engineering quality inspection, in order to ensure the authenticity and traceability of the engineering inspection process, the inspection agency usually needs to conduct video monitoring of the engineering construction process. However, the use of video image data is relatively limited; the traditional video query method mainly relies on manual browsing or the "average slicing method" to simply process the video. However, these methods have obvious shortcomings. First, the "average slicing method" is based on the average test time for slicing, which cannot ensure that the dynamic or changing process of each detection object can be recorded completely and accurately, and it is easy to have incomplete or mismatched video clips. Secondly, this method cannot extract the detection object and its information in the video, making it difficult to associate the detection object with its unique video slice fragments to effectively verify the authenticity of the test. In addition, there is also a slicing method based on the start and end time points of the test in the prior art. This method intercepts the corresponding video clips from the monitoring video according to the start and end time points of the test machine. However, since the monitoring system and the detection system are usually operated independently, there may be a deviation in the time between the two, resulting in a mismatch between the sliced ​​video and the actual detection process. Secondly, the testing machine may have problems such as recording failure or delay, which further affects the accuracy and integrity of the sliced ​​video. In addition, this method is also unable to effectively extract the test object and its information in the video, and the existing slicing method is unable to extract relevant information in the engineering inspection process video to effectively verify the authenticity of the test, which makes the inspection process limited in traceability and verification. The above problems need to be solved. Summary of the invention

[0003] The main purpose of the present application is to overcome the shortcomings and deficiencies of the prior art, and to provide a method, system, device and storage medium for video slicing of an engineering inspection process, which determines the start and end time points of the video slicing by the appearance and disappearance of the inspection object, can more accurately determine the start and end time of the video slicing, and ensure that the dynamic or changing process of each inspection object can be fully recorded. At the same time, according to the start and end time points of the video slicing, the relevant information of the corresponding time node in the video is selected for authenticity verification. If the verification is authentic, the inspection object is bound and associated with its unique video slice to achieve traceability and reliability of the inspection process.

[0004] In order to achieve the above objectives, this application adopts the following technical solutions:

[0005] In a first aspect, the present application provides a method for slicing a video of an engineering inspection process, comprising the following steps:

[0006] Get the video stream of the engineering inspection process;

[0007] Input the video stream into the pre-established multi-target detection model to obtain the position information of the equipment, tester and detection object number display screen;

[0008] The center point coordinates, width and height of the position information of the device, the tester and the detection object number display screen are respectively transformed, and each frame of the video stream is cropped based on the transformed coordinate position to obtain the first information; the first information includes: the test device image, the tester image and the detection object number display screen image;

[0009] Identify and detect the first information to obtain second information; the second information includes: the location of the detection object, the category of the detection object, the type of equipment, the movement trajectory of the equipment, the detection object number, the identity of the tester, the change of the tester's body movement and the state change of the detection object;

[0010] Acquire the appearance and disappearance of the detection object according to the position of the detection object, and determine the start and end time points of the video slice according to the appearance and disappearance;

[0011] The authenticity of the test process is verified by selecting the second information in the corresponding time node according to the start and end time points of the video slice. If the verification is authentic, the video to be sliced ​​is sliced, and the detection object category, equipment type, tester identity, detection object number and authenticity in the second information at the corresponding time node are bound to the slice.

[0012] As a preferred technical solution, the video stream is input into a pre-established multi-target detection model to obtain the position information of the device, the tester and the detection object number display screen, including:

[0013] The multi-target detection model is based on the improved Yolov4 algorithm, with Hornet as the backbone network, and also includes ASPP module, PANet module, and Decoupled Head module;

[0014] The Hornet backbone network extracts three layers of high-level feature maps of different scales from each frame;

[0015] The ASPP module extracts multi-scale spatial features in the last layer of high-level feature maps at different sampling rates and fuses the multi-scale spatial features to obtain spatial fusion features;

[0016] The PANet module aggregates the high-level feature maps and spatial fusion features of the first two layers at multiple scales to generate three aggregated feature maps at different scales;

[0017] The Decoupled Head module extracts the location information of the device, tester, and test object number display screen from the aggregated feature maps at three scales.

[0018] As a preferred technical solution, the step of identifying and detecting the first information to obtain the second information includes:

[0019] Performing object detection on the device image using a detection object detection model to obtain location information of the detection object;

[0020] Performing device recognition on the device image using a device recognition model to obtain device type information;

[0021] Performing device motion trajectory detection on the device image using a device motion trajectory detection model to obtain device motion trajectory information;

[0022] The detection object number display screen image is numbered and recognized by a text recognition model to obtain the detection object number information;

[0023] Performing identity recognition on the tester image through a facial biometric recognition model to obtain the tester's identity information;

[0024] The state of the detection object image is recognized by using the detection object state recognition model to obtain the state change of the detection object;

[0025] The detection object image is subjected to category recognition by using a detection object category recognition model to obtain detection object category information;

[0026] The experimenter's human motion recognition model is used to perform motion recognition on the experimenter's image to obtain the experimenter's human motion changes.

[0027] As a preferred technical solution, the step of determining the start and end time points of the video slice according to the appearance and disappearance conditions includes:

[0028] Get the image frame where the detected object first appears in the device area, recorded as the t1th frame;

[0029] Get the coordinate position of the detected object in the device area image;

[0030] Starting from the t1th frame, if the coordinate position information of the detection object exists in the subsequent n1 frames, the t1th frame is used as the pre-starting point;

[0031] Obtain the coordinate position of the detection object at the pre-starting point, and calculate the change in the coordinate position of the detection object for a preset number of frames subsequent to the t1 frame. If the change in the coordinate position of the detection object for the subsequent preset number of frames is less than a preset first threshold, the t1 frame is used as the slicing starting point;

[0032] Obtain the image frame where the detected object disappears from the device area for the first time, recorded as the t2th frame;

[0033] Starting from the t2th frame, if there is no coordinate position information of the detection object in the subsequent n3 frames, the t2th frame is used as the end point.

[0034] As a preferred technical solution, the authenticity verification of the test process is to verify the authenticity of the test process by selecting the changes in the tester's body movements, the equipment operating status or the state changes of the detection object in the second information of the corresponding time node through the start and end time points of the video slice.

[0035] As a preferred technical solution, the authenticity verification of the test process is performed by selecting the tester's body movement changes, equipment operation status or detection object status changes in the second information corresponding to the time node through the start and end time points of the video slice, including:

[0036] The device operation state is to obtain the coordinates of the key points of the device moving parts at the starting point of the slice, traverse the coordinates of the key points of the device moving parts of the slice from front to back, and calculate the coordinate distance difference between the key points at each moment and the moment of the starting point of the slice; if the ratio of the coordinate distance of the key points at a certain moment to the width of the device or the height of the device is greater than the preset second threshold or third threshold, the device operation state of the slice is normal operation;

[0037] If the operating state of the sliced ​​equipment is normal, the state change of the detection object is change, or the human motion change of the tester is consistent with the human motion change in the corresponding detection object scene, then the test process is real.

[0038] As a preferred technical solution, if the verification is true, the video to be sliced ​​is sliced, and the detection object category, equipment type, tester identity, detection object number and authenticity in the second information of the corresponding time node are bound to the slice, including:

[0039] Count the number of times the test object category, equipment type, tester identity, and test object number appear at the start and end time nodes respectively;

[0040] Select the test object category, equipment type, tester identity and test object number that appear most frequently in the start and end time nodes as the final test object category, equipment type, tester identity and test object number of the slice;

[0041] If the authenticity verification is not authentic, a non-authenticity warning message will be fed back.

[0042] In a second aspect, the present application provides an engineering inspection process video slicing system, which is applied to the engineering inspection process video slicing method, including a video stream acquisition module, a multi-target detection module, a first information acquisition module, a second information acquisition module, an adaptive cut point module and a slice generation module;

[0043] The video stream acquisition module is used to acquire the video stream of the engineering inspection process;

[0044] The multi-target detection module is used to input the video stream into a pre-established multi-target detection model to obtain the position information of the equipment, the tester and the detection object number display screen;

[0045] The first information acquisition module is used to transform the center point coordinates, width and height of the position information of the equipment, the tester and the detection object number display screen respectively, and crop each frame of the video stream based on the transformed coordinate position to obtain the first information; the first information includes: the test equipment image, the tester image and the detection object number display screen image;

[0046] The second information acquisition module is used to identify and detect the first information to obtain second information; the second information includes: the position of the detection object, the category of the detection object, the type of equipment, the movement trajectory of the equipment, the detection object number and the identity of the tester, the change of the tester's body movement and the state change of the detection object;

[0047] The adaptive cut-point module is used to obtain the appearance and disappearance of the detection object according to the position of the detection object, and determine the start and end time points of the video slice according to the appearance and disappearance;

[0048] The slice generation module is used to select the second information in the corresponding time node according to the start and end time points of the video slice to verify the authenticity of the test process. If the verification is authentic, the video to be sliced ​​is sliced, and the detection object category, equipment type, tester identity, detection object number and authenticity in the second information of the corresponding time node are bound to the slice.

[0049] In a third aspect, the present application provides an electronic device, the electronic device comprising:

[0050] at least one processor; and,

[0051] a memory communicatively connected to the at least one processor; wherein,

[0052] The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can perform the video slicing method of an engineering inspection process.

[0053] In a fourth aspect, the present application provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the method for video slicing of an engineering inspection process.

[0054] In summary, compared with the prior art, the effective effects brought about by the technical solution provided by this application include at least:

[0055] The present application proposes a method for slicing a video of an engineering inspection process, which obtains a video stream of the engineering inspection process, passes the video stream through a multi-target detection model to detect the video stream, and obtains the position information of the equipment, tester, and the number display screen of the inspection object; respectively transforms the center point coordinates, width, and height of the position information of the equipment, tester, and the number display screen of the inspection object, and crops each frame of the video stream based on the transformed coordinate position to obtain the first information, and identifies and detects the first information to obtain the second information, and detects the object position from the second information to obtain the appearance and disappearance of the inspection object, and determines the start and end time points of the video slice according to the appearance and disappearance, so as to more accurately determine the start and end time of the video slice, and ensure that the dynamic or changing process of each inspection object can be completely and accurately recorded, avoiding the situation that the video clip is incomplete or the time does not match; in addition, the authenticity of the test process is verified to obtain the authenticity of the test process, and if the verification is true, the video to be sliced ​​is sliced, The detection object category, equipment type, tester identity, detection object number and authenticity in the second information of the corresponding time node are bound to the slice, so that the detection object is closely associated with its unique video slice segment, so that every link of the engineering construction detection test process can be clearly tracked and verified, and the traceability of the detection process can be achieved; in addition, the present application also verifies the authenticity of the test process through changes in the tester's body movements, equipment operating status or detection object status changes, thereby avoiding the problem of false tests. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0057] Figure 1 A flowchart of a method for video slicing of an engineering inspection process provided by one embodiment of the present application;

[0058] Figure 2 A block diagram of a video slicing system for an engineering inspection process provided for one embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0060] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0061] Example:

[0062] See also Figure 1 In one embodiment of the present application, a method for slicing a video of an engineering inspection process is provided, comprising the following steps:

[0063] S1. Obtain the video stream of the engineering inspection process.

[0064] Furthermore, after obtaining the video stream of the engineering inspection process, the video stream needs to be preprocessed.

[0065] The pre-processing comprises:

[0066] S11, downsampling the video stream to a preset frame rate to obtain a downsampled video stream;

[0067] Furthermore, the original video stream may contain a large number of frame images (for example, the video stream consists of 30 frames per second), so the video stream needs to be downsampled. In this embodiment, the frame rate of the video stream is downsampled to 2 frames per second. This can not only reduce the demand for computing resources, but also improve the processing speed without affecting the detection accuracy.

[0068] S12: Split the downsampled video stream into image frames.

[0069] Furthermore, in order to store and track the information related to each frame, 8 arrays are created. The length of each array is the same as the number of frames in the video stream (if the video stream duration is 1 minute, that is, the number of video stream pictures is 2×60=120, then the array length is 120), which are used to store the detection object position, detection object category, device type, device motion trajectory, detection object number, experimenter identity information, detection object state change and experimenter body key points.

[0070] S13, adjusting each frame of the image to a preset size.

[0071] Furthermore, it is necessary to adjust the size of each frame image to the size required by the multi-target detection model. In this embodiment, the long side of each frame image is scaled to the image size specified in the multi-target detection model, while the short side is scaled proportionally to avoid image deformation, thereby ensuring that the entire image is appropriately resized.

[0072] Furthermore, after adjusting each frame of the image to a preset size, the left and right or upper and lower areas of each frame of the image are filled with 0, so that the width and height of the image meet the specified size of the multi-target detection model.

[0073] S14, converting the format of the frame image adjusted to the preset size and performing standardization processing.

[0074] Furthermore, the frame image adjusted to a preset size is converted into RGB format and standardized, that is, the image parameters such as brightness, contrast and color distribution are adjusted. Finally, the processed image is converted into tensor format. The tensor format is often used to represent multidimensional arrays. Here, it is a format used to store and process image data.

[0075] The detection objects in this embodiment include samples in engineering detection and detection targets at the construction site; for example, samples can be concrete samples, metal samples, etc.; detection targets at the construction site can be various elements that appear at construction sites such as building projects, bridge projects, road projects, etc., for example, structural components such as beams, columns, plates, walls, etc.

[0076] S2. Input the video stream into a pre-established multi-target detection model to obtain the location information of the equipment, the tester, and the display screen of the detection object number.

[0077] Furthermore, the inspection object number is a number used to uniquely identify or track a certain inspection object (sample). During the engineering construction quality inspection process, since multiple inspection objects may be carried out at the same time, in order to ensure that each inspection object can be independently tracked, recorded and managed, a number will be assigned to each inspection object.

[0078] The multi-target detection model in this embodiment is based on the improved Yolov4 algorithm, with Hornet as the backbone network, and also includes an ASPP module (AtrousSpatialPyramidPooling), a PANet module (PathAggregationNetwork), and a DecoupledHead module. The SPP in the Yolov4 algorithm is replaced by ASPP, the Head is changed to a DecoupledHead structure, and the CmBN is replaced by BN (BatchNormalization Layer), so that it can effectively improve the detection accuracy in detecting testers, detection object number display screens, and test equipment; wherein, CmBN refers to Cross mini-Batch Normalization. Specifically:

[0079] The last three stages of the S21 and Hornet backbone networks extract three layers of high-level feature maps of different scales from each frame;

[0080] The S22 and ASPP modules extract multi-scale spatial features from the last layer of high-level feature maps at different sampling rates, and fuse the multi-scale spatial features to obtain spatial fusion features; the ASPP module uses multiple parallel dilated convolutional layers with different sampling rates, including 3x3 convolutional layers with dilated rates of 6, 12, and 18, respectively.

[0081] The S23,PANet module aggregates the high-level feature maps and spatial fusion features of the first two layers at multiple scales to generate three aggregated feature maps of different scales;

[0082] S24, Decoupled Head module extracts the target position information of the device, tester and test object number display screen from the aggregated feature map of the three scales. Its position information includes the center point coordinates (x, y), width and height (w, h).

[0083] Furthermore, the aggregated feature maps of the three scales are input into two 3x3 and one 1x1 convolutional layers to obtain the category probabilities of the test equipment image, the tester image and the detection object number display screen image in the first information; the aggregated feature maps of the three scales are then input into two 3x3 convolutional layers, and then respectively input into two parallel 1x1 convolutional layers to obtain the coordinate values, width, height and confidence of their categories.

[0084] The function of the detection object number display screen in this embodiment is to display the number identification of the detection object currently receiving detection in the video.

[0085] S3. Transform the center point coordinates, width and height of the position information of the equipment, tester and detection object number display screen respectively, and crop each frame image in the video stream based on the transformed coordinate position to obtain first information; the first information includes: the test equipment image, the tester image and the detection object number display screen image.

[0086] Furthermore, the center point, width and height of the position information of the test equipment image, the tester image and the detection object number display screen image are transformed respectively: , , , ;in Indicates the upper left corner of the test equipment, test object or test object number screen in the image. Indicates the lower right corner position of the test equipment, test object or test object number screen in the image; then based on the transformed coordinate position and The rectangular frame formed by the method crops the test equipment, tester and detection object number display screen in each frame image, thereby obtaining the cropped test equipment image, tester image and detection object number display screen image respectively; finally, it also includes the processing operations of scaling, filling, standardizing and converting the format of each frame of the test equipment image, tester image and detection object number display screen image, processing them into the specified size and format of the subsequent detection model, and obtaining the processed test equipment image, tester image and detection object number display screen image.

[0087] S4. Identify and detect the first information to obtain second information; the second information includes: the location of the detection object, the category of the detection object, the type of equipment, the movement trajectory of the equipment, the detection object number, the identity of the tester, the change of the tester's body movement and the state change of the detection object.

[0088] In this embodiment, a plurality of different models are used to identify and detect the first information to obtain the second information.

[0089] S41. Equipment type identification:

[0090] The device identification model in this embodiment is based on the ConvNext algorithm and adds an SE (Squeeze-and-Excitation) attention module to the ConvNext network; its SE attention module first performs global average pooling on the feature map and compresses it into a feature vector, then compresses the weights by 16 times through full connection and then fully connects it back to obtain the weights of each channel, and finally multiplies the channel weights by the original feature map. The device identification model specifically includes:

[0091] S41.1, 7x7 depthwise separable convolutional layer extracts features of processed test equipment images;

[0092] S41.2, LayerNorm layer (normalization layer) normalizes the extracted image features to obtain a normalized feature map;

[0093] S41.3, SE attention module weights the above normalized feature map on the channel to obtain the feature representation of important channels, suppressing the influence of unimportant channels. Further, the steps are: globally average pooling the feature map and compressing it into a feature vector; secondly, compressing the feature vector by 16 times the number of weights through full connection and then fully connecting it back to obtain the weights of each channel, and finally multiplying the channel weights by the original feature map.

[0094] S41.4. Use a 1x1 depthwise separable convolutional layer to further extract features from the reweighted feature map, and use the GELU activation function to learn the fitted nonlinear relationship.

[0095] S41.5. Feature extraction is performed again through the 1x1 depthwise separable convolutional layer. Then, the extracted features are rescaled using the Layer Scale layer, and regularization techniques are used to randomly select certain feature channels in the adaptively adjusted feature map and discard them.

[0096] Finally, the device recognition model performs the final device type prediction through the classification layer, obtains the type of device in each frame, and stores the device type information in the corresponding frame position in the device type array.

[0097] This implementation adds the SE attention module to the ConvNext network, which enables the device recognition model to better focus on important feature channels and improve the accuracy and robustness of recognition.

[0098] S42, device motion trajectory detection: inputting the processed test device image into the device motion trajectory detection model, obtaining the key point coordinates of the device motion trajectory in each frame image, and storing the coordinates in the position of the corresponding frame in the device motion trajectory array.

[0099] Furthermore, the device motion trajectory is composed of a series of key point coordinates of the device motion components in the device motion trajectory array.

[0100] The device motion trajectory detection model in this embodiment is based on the HRNet key point detection algorithm and uses HRNet_w32 as the backbone network. The network includes 4 Stage stages, and a hole convolution group is added in the Stage 3 stage. Specifically: Stage 1 and Stage 2 extract image features step by step; the hole convolution group is added in the Stage 3 stage, and the spatial convolution group uses multiple series hole convolution layers with different sampling rates, and the hole rates are 2, 4, 6 and 8 3x3 convolution layers, respectively, to learn and obtain feature representations of different scales; the Stage 4 stage will fuse the features extracted in the previous stages to form the final high-level feature representation. Finally, according to the high-level feature representation diagram obtained in the Stage 4 stage, the key feature points, that is, the key position points, are obtained, and then the device position points in the identified multi-frame images are used to form the device motion trajectory. In addition, the key point coordinates of the device motion trajectory need to be determined according to the moving parts of the device, and different devices have different key point coordinates.

[0101] This embodiment adds a hole convolution group in Stage 3 to increase the long-distance receptive field of the network, which can more effectively extract spatial features and improve the recognition accuracy of the device motion trajectory.

[0102] S43, detection object: input the processed test equipment image into the detection object detection model. If the detection object exists in the image, obtain the mask map of the detection object in each frame of the image, extract the circumscribed rectangle of the target in the mask map, and use the coordinates of the center point of the rectangle to find the detection object. and width and height As the position information of the detected object, the center point coordinates and width and height are stored in the position of the corresponding frame in the detection object position array.

[0103] In this embodiment, the detection object detection model is The segmentation algorithm is based on the architecture. The hole convolution in En_5Block (specific encoding module) and De_5Block (specific decoding module) is replaced with deformable convolution deform_conv2d, which effectively improves the long-distance receptive field of the model and makes the model segmentation effect better. After En_5Block and De_5Block are replaced, their structure is as follows:

[0104] Conv1: includes 3x3 convolution layer, BN (Batch Normalization Layer) layer and ReLu activation function; the 3x3 convolution layer is used to initially extract image features; and the BN layer is used for normalization;

[0105] Conv2: 3x3 convolution layer, BN layer and ReLu activation function; the 3x3 convolution layer is used to further extract image features;

[0106] Conv3: includes deformable convolution deform_conv2d 3x3, BN layer and ReLu activation function; among them, deformable convolution deform_conv2d 3x3 is used to learn the offset of each position of the feature map extracted by Conv2, so that the convolution kernel can move adaptively on the input feature map, improve the long-distance receptive field of the model, and make the model segmentation effect better.

[0107] Conv4: includes 3x3 convolution layer, BN layer and ReLu activation function; the 3x3 convolution layer in Conv4 will further extract and integrate the information in the output feature map of Conv3 layer;

[0108] The above four Convs are a series structure.

[0109] Finally, the output layer of the object detection model converts the feature map into a final segmentation map, and obtains the precise location information of the detected object in the image based on the segmentation map, including the shape, size, and position of the detected object.

[0110] S44, detection object category recognition: If a detection object is detected, the detection object image is input into the detection object category recognition model, and the detection object category information is stored in the position of the corresponding frame in the detection object category array.

[0111] In this embodiment, the detection object category recognition model is based on the Hornet algorithm, and a trainable scaling factor is added to its core module HorBlock, which greatly improves the model convergence speed and further improves the model recognition accuracy. Specifically, the core module HorBlock extracts the detection object features. During the feature extraction process, the module HorBlock adaptively adjusts the importance of different features through the trainable scaling factor Scale.

[0112] S45, detection object number recognition: input the processed detection object number display screen image into the text recognition model, obtain the text of the detection object number display screen in each frame image, and store the text in the position of the corresponding frame in the detection object number array; wherein, the information of the detection object number display screen is pushed by the data acquisition detection control software, and the detection object number and detection sequence number information are pushed to the LED display screen through the serial port at the beginning of the detection, and the detection equipment, detection object and LED display screen are simultaneously included in the video monitoring system, and the corresponding detection process is associated with the detection object number. The detection object number is an integral part of the video content, and the detection object number can be intuitively seen when viewing the video; the process of detecting the object number is as follows:

[0113] The text recognition model is generally composed of an encoding layer, a decoding layer, and an output layer. This embodiment uses the ConvNext network as the encoding layer to extract features from the image, uses N ViTBlock structures as the decoding layer to perform context modeling on the CNN (Convolutional Neural Network) features, and finally uses CTC Loss as the output layer for decoding output. CTC Loss refers to Connectionist Temporal Classification Loss, which is a loss function used for input and output sequence alignment problems.

[0114] S46, tester identity recognition: input the processed tester image into the facial biometric recognition model to obtain the identity of the tester of the detection object in each frame of the image, and store the identity in the position of the corresponding frame in the tester identity array.

[0115] In this embodiment, the facial biometric recognition model is based on the Arcface algorithm, and the backbone network is replaced by the MobilenetV3-Large network. MobilenetV3-Large has 15 layers of bottleneck modules, and the DW convolution in the 8th to 13th layers of bottleneck modules is replaced by the deformable convolution deform_conv2d, thereby effectively improving the long-distance receptive field of the model, so that the model recognition effect is better; the steps include: 1 to 7 layers of bottleneck modules extract image features step by step; replace the DW convolution in the 8th to 13th layers of bottleneck modules with the deformable convolution deform_conv2d, and the deformable convolution in the 8th to 13th layers of bottleneck modules adaptively changes shape and position according to the features of the input image, so as to capture more complex patterns and structures, and extract more abstract and advanced features; effectively improve the long-distance receptive field of the model, so that the model recognition effect is better. Finally, the model compares and matches the extracted features with the reserved tester information to confirm the identity of the tester.

[0116] S47, detection object state change: input the processed detection object image into the detection object state recognition model for state recognition, obtain the state change of the detection object, and store the state change in the position of the corresponding frame in the detection object state change array.

[0117] The detection object state recognition model in this embodiment is also based on the Hornet algorithm. A trainable scaling factor is added to its core module HorBlock, which greatly improves the model convergence speed and further improves the model recognition accuracy. Specifically, the core module HorBlock extracts the detection object features. During the feature extraction process, the module HorBlock adaptively adjusts the importance of different features through the trainable scaling factor Scale.

[0118] S48, human body motion recognition of the subject: input the processed image of the subject into the human body motion recognition model of the subject to obtain the human body motion changes of the subject; wherein the human body motion recognition model of the subject includes a human body key point detection model and a human body motion recognition model.

[0119] Specifically, the processed test subject image is first input into the human body key point detection model to obtain the coordinates of 15 human body key points of the test subject in each frame of the image [(x1, y1)…(x 15 ,y 15 )], the coordinates of the key points of the human body include: the tip of the nose, the neck, the right shoulder, the right elbow, the right wrist, the left shoulder, the left elbow, the left wrist, the center point of the line connecting the centers of the left and right hip joints, the right hip joint, the right knee, the right ankle, the left hip joint, the left knee and the left ankle, and they are stored in the key point array of the subject's human body.

[0120] The human key point detection model is also based on the HRNet key point detection algorithm and uses HRNet_w32 as the backbone network. The network has a total of 4 stages, and a dilated convolution group is added in the Stage 3 stage; the spatial convolution group uses multiple series-connected dilated convolution layers with different sampling rates, and the 3x3 convolution layers with dilation rates of 2, 4, 6, and 8 respectively; the Stage 3 stage adds a dilated convolution group, and the spatial convolution group uses multiple series-connected dilated convolution layers with different sampling rates to learn feature representations of different scales.

[0121] The coordinates of 15 human key points in each frame obtained by the human key point detection model are spliced ​​into a vector, that is, the vector of each frame consists of 30 values; each vector is preprocessed by standardization and format conversion operations to the specified size and format of the human action recognition model; each processed vector is input into the human action recognition model to obtain the human action changes of the corresponding slice. Among them, the human action recognition model is also based on the Hornet algorithm. A trainable scaling factor is added to its core module HorBlock, which greatly improves the model convergence speed and further improves the model recognition accuracy. Specifically, the core module HorBlock extracts the features of the detection object. During the feature extraction process, the HorBlock module adaptively adjusts the importance of different features through the trainable scaling factor Scale.

[0122] In this embodiment, the data set used to train each model is split into a training set, a validation set, and a test set; the training set accounts for 80% of the total, and the validation set and the test set each account for 10% of the total. The difference between the validation set and the test set is that the validation set is used to evaluate the model effect of each iteration during the training process, while the test set is used to evaluate the final selected model effect. Furthermore, the training set data is enhanced in color such as brightness and saturation, as well as random flipping, scaling, and data enhancement techniques; the detailed parameters are as follows:

[0123] Color enhancement technology: Using LUT (Look-Up Table) technology, the principle is to create a mapping table that associates input pixel values ​​with output color values; the output color range is:

[0124] [0,255]×(1+rand(-0.2,0.2))+(rand(-0.2,0.2)×255).

[0125] Random Flip: Flip the image randomly horizontally or vertically.

[0126] Random Scaling: While keeping the image size unchanged, the content is randomly enlarged or reduced by 0.3 times.

[0127] Mixup data enhancement technology: Mix two random images in random proportions, that is, the two images will be merged and displayed in a semi-transparent form.

[0128] Cutout data enhancement technology: discard some areas in the image and fill them with 0 pixel values.

[0129] CutMix data augmentation technology: discard part of the image area and randomly fill the target area of ​​other images in the training set.

[0130] In addition, for classified data labels, such as device identification models and detection object category identification models, first use the form of one-hot encoding, and then use The label smoothing operation makes the values ​​non-extreme and helps convergence. Represents the unique hot encoding of the true label; N is the total number of categories; is the smoothing coefficient. At the same time, in order to enhance the recognition ability of each model for objects of different scales, the image size is scaled to 448, 648, 800, and 1200 based on the longest side, and then one of the scales is randomly selected for training each time the input image is input. In addition, for the selection of loss function, CIOU Loss (Complete Intersection over Union Loss, a loss function used in target detection tasks) is used to calculate the overlap ratio between the real box and the predicted box. In classification recognition or segmentation, the Focal Loss loss function is used to calculate the loss. In key point detection, MSE Loss (Mean Square Error Loss) is used to calculate the key point loss. In text recognition, CTC Loss is used for recognition and decoding. The SGD optimizer is used for the optimizer. Among them, the learning rate is adopted The momentum is set to 0.92 and the weight_decay is set to 0.0005.

[0131] S5. Acquire the appearance and disappearance of the detection object according to the position of the detection object, and determine the start and end time points of the video slice according to the appearance and disappearance.

[0132] Further, the start and end time points of the video slice are determined according to the appearance and disappearance conditions, and the steps include:

[0133] Slice starting point judgment steps: If the detection object appears in the device area, the detection object position array will record the detection object's position information in the current frame. If the detection object position array always has the detection object's position information, it means that the detection object has appeared in the device area. The starting point judgment is as follows:

[0134] Get the image frame where the detected object first appears in the device area, recorded as the t1th frame;

[0135] Get the position where the detection object appears in the device area image, that is, the coordinates (x, y) of the center point of the detection object;

[0136] Starting from the t1 frame, if the subsequent n1=10 frames all contain the position information of the detected object, the t1 frame is used as the pre-starting point;

[0137] The coordinate position of the detection object in the pre-starting point is obtained, and the change in the coordinate position of the detection object in the preset number of frames subsequent to the t1 frame is calculated. If the change in the coordinate position of the detection object in the subsequent preset number of frames is less than a preset threshold, the t1 frame (i.e., the starting point within a certain time period) is used as the slicing starting point.

[0138] Slice end point judgment steps: If the detection object disappears in the device area, the detection object position array will not have the detection object position information record in the current frame. If the detection object position array has no detection object position information, it means that the detection object has disappeared in the device area. The end point judgment is as follows:

[0139] Obtain the image frame where the detected object disappears from the device area for the first time, recorded as the t2th frame;

[0140] Starting from the t2 frame, if there is no detection object in the subsequent n3=10 frames, the t2 frame is taken as the end point.

[0141] The above steps of determining the start and end time points of the video slice are repeated, and all the slice start points and end points are stored in a preset cut point array.

[0142] S6. Select the second information in the corresponding time node according to the start and end time points of the video slice to verify the authenticity of the test process. If the verification is authentic, slice the video to be sliced, and bind the detection object category, equipment type, tester identity, detection object number and authenticity in the second information at the corresponding time node to the slice.

[0143] In this embodiment, the authenticity of the test process is verified by selecting the changes in the tester's body movements, the equipment operating status or the state changes of the detection object in the second information of the corresponding time node through the start and end time points of the video slice to verify the authenticity of the test process.

[0144] The authenticity verification of the test process is performed by selecting the change of the tester's body movement, the operation state of the equipment or the state change of the detection object in the second information corresponding to the time node through the start and end time points of the video slice, specifically including:

[0145] The device operation state is to obtain the coordinates of the key points of the device moving parts at the starting point of the slice (recorded as time t0), traverse the coordinates of the key points of the device moving parts of the slice from front to back, and calculate the coordinate distance difference between each moment and the key point at the starting point of the slice (time t0); wherein the coordinate distance difference of the x-axis is the horizontal axis movement direction and movement distance of the device relative to time t0; the coordinate distance difference of the y-axis is the vertical axis movement direction and movement distance of the device relative to time t0; the movement distance and direction of the device at all moments in the slice are combined into the motion trajectory of the slice.

[0146] Determine the operating status of the equipment. The operating status of different equipment needs to be determined in different ways. The determination methods are as follows:

[0147] If the ratio of the maximum movement distance of the device in the horizontal axis direction (i.e., the x-coordinate distance difference of the key points) at a certain moment to the width of the device is greater than the preset second threshold, then the device operation status of the slice is normal operation; if the ratio of the maximum movement distance of the device in the vertical axis direction (i.e., the y-coordinate distance difference of the key points) at a certain moment to the height of the device is greater than the preset third threshold, then the device operation status of the slice is normal operation.

[0148] If the device operation status of the slice is normal, the state change of the detection object is changed, or the change of the tester's body movement is consistent with the change of the body movement in the corresponding detection object scene, then the test process is authentic. The authenticity of the test process can be judged by combining the above three rules: the device operation status, the detection object status, and the tester's body movement change, so as to realize the specific detection scene with specific verification rules to verify the authenticity of the test process in the slice, where the detection scene is determined by the device type and detection object category bound to the video slice of the corresponding detection process.

[0149] In this embodiment, by judging the operating state of the equipment, the state changes of the test object and the changes in the human body movements of the tester, the authenticity of the test process can be more comprehensively evaluated; this evaluation method helps to ensure that the results of the test or detection are authentic and avoid the problem of false tests.

[0150] Further, the video stream is cut into multiple slices according to the starting point and ending point of each slice stored in the cut point array, and the detection object category array, device type array, device motion trajectory array, detection object number array, tester identity array and detection object state change array are also cut using the starting point and ending point to obtain multiple array fragments. Finally, if the test process is verified to be true, the video to be sliced ​​is sliced, and the detection object category, device type, tester identity, detection object number and authenticity in the second information of the corresponding time node are bound to the slice, and the steps include:

[0151] Count the number of times the test object category, equipment type, tester identity, and test object number appear at the start and end time nodes respectively;

[0152] Select the test object category, equipment type, tester identity and test object number that appear most frequently in the start and end time nodes as the final test object category, equipment type, tester identity and test object number of the slice;

[0153] In addition, if the authenticity verification is not authentic, a non-authenticity warning message is fed back.

[0154] Specifically, the detection object category array, equipment type array, experimenter identity array and detection object number array contain correct identification, misidentification and non-identification results, but the number of misidentification and non-identification in the video slice is very small. Therefore, the detection object category, equipment type, experimenter identity and detection object number that appear most frequently in the slice are selected as the final detection object category, equipment type, experimenter identity and detection object number.

[0155] In summary, the present application uses the detection object information in the video to slice the video, and the detection object can be a sample, any object or target related to the detection, such as building materials, equipment, personnel, etc.; and by analyzing the behavior, position and other information of these detection objects, the video can be sliced ​​more effectively. The present application determines the start and end time points of the video slice according to the appearance and disappearance, so that the start and end time of the video slice can be determined more accurately. Compared with the traditional "average slicing method" or the slicing method based on the start and end time points of the test, the present application can ensure that the dynamic or change process of each detection object can be recorded completely and accurately, avoiding the situation of incomplete video clips or time mismatch; in addition, the present application also binds the detection object with its unique video slice fragment, realizes the close association between the detection object and its unique video slice fragment, so that the detection object can be clearly tracked and verified in every link of the detection process, realizes the traceability of the detection process, and plays a better role in protecting the effectiveness and authenticity of the quality detection of the construction project; in addition, the authenticity of the test process is verified by the changes in the human body movements of the tester, the operation status of the equipment or the state changes of the detection object, avoiding the problem of false tests.

[0156] It should be noted that, for the sake of convenience of description, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously.

[0157] Based on the same idea as the engineering detection process video slicing method in the above embodiment, the present application also provides an engineering detection process video slicing system, which can be used to execute the above engineering detection process video slicing method. For ease of explanation, the structural schematic diagram of an engineering detection process video slicing system embodiment only shows the parts related to the embodiment of the present application. Those skilled in the art can understand that the illustrated structure does not constitute a limitation on the system, and may include more or fewer components than shown in the diagram, or combine certain components, or arrange the components differently.

[0158] See also Figure 2In another embodiment of the present application, a video slicing system for an engineering inspection process is provided, the system comprising a video stream acquisition module 101, a multi-target detection module 102, a first information acquisition module 103, a second information acquisition module 104, an adaptive cut-point module 105, and a slice generation module 106;

[0159] The video stream acquisition module 101 is used to acquire the video stream of the engineering inspection process;

[0160] The multi-target detection module 102 is used to input the video stream into a pre-established multi-target detection model to obtain the position information of the equipment, the tester and the detection object number display screen;

[0161] The first information acquisition module 103 is used to transform the center point coordinates, width and height of the position information of the device, the tester and the detection object number display screen respectively, and crop each frame of the video stream based on the transformed coordinate position to obtain the first information; the first information includes: the test device image, the tester image and the detection object number display screen image;

[0162] The second information acquisition module 104 is used to identify and detect the first information to obtain second information; the second information includes: the position of the detection object, the detection object category, the device type, the movement trajectory of the device, the detection object number and the identity of the tester, the tester's body movement changes and the detection object state changes;

[0163] The adaptive cut-point module 105 is used to obtain the appearance and disappearance of the detection object according to the position of the detection object, and determine the start and end time points of the video slice according to the appearance and disappearance;

[0164] The slice generation module 106 is used to select the second information in the corresponding time node according to the start and end time points of the video slice to verify the authenticity of the test process. If the verification is authentic, the video to be sliced ​​is sliced, and the detection object category, equipment type, tester identity, detection object number and authenticity in the second information at the corresponding time node are bound to the slice.

[0165] The multi-target detection module 102 is specifically used for:

[0166] The multi-target detection model is based on the improved Yolov4 algorithm, with Hornet as the backbone network, and also includes ASPP module, PANet module, and Decoupled Head module;

[0167] The Hornet backbone network extracts three layers of high-level feature maps of different scales from each frame;

[0168] The ASPP module extracts multi-scale spatial features in the last layer of high-level feature maps at different sampling rates and fuses the multi-scale spatial features to obtain spatial fusion features;

[0169] The PANet module aggregates the high-level feature maps and spatial fusion features of the first two layers at multiple scales to generate three aggregated feature maps at different scales;

[0170] The Decoupled Head module extracts the location information of the device, tester, and test object number display screen from the aggregated feature maps at three scales.

[0171] The second information acquisition module 104 is specifically used for:

[0172] Performing object detection on the device image using a detection object detection model to obtain location information of the detection object;

[0173] Performing device recognition on the device image using a device recognition model to obtain device type information;

[0174] Performing device motion trajectory detection on the device image using a device motion trajectory detection model to obtain device motion trajectory information;

[0175] The detection object number display screen image is numbered and recognized by a text recognition model to obtain the detection object number information;

[0176] Performing identity recognition on the tester image through a facial biometric recognition model to obtain the tester's identity information;

[0177] The state of the detection object image is recognized by the detection object state recognition model to obtain the state change of the detection object;

[0178] The detection object image is subjected to category recognition by using a detection object category recognition model to obtain detection object category information;

[0179] The experimenter's human motion recognition model is used to perform motion recognition on the experimenter's image to obtain the experimenter's human motion changes.

[0180] The adaptive cut-point module 105 is specifically used for:

[0181] Get the image frame where the detected object first appears in the device area, recorded as the t1th frame;

[0182] Get the coordinate position of the detected object in the device area image;

[0183] Starting from the t1th frame, if the coordinate position information of the detection object exists in the subsequent n1 frames, the t1th frame is used as the pre-starting point;

[0184] Obtain the coordinate position of the detection object at the pre-starting point, and calculate the change in the coordinate position of the detection object for a preset number of frames subsequent to the t1 frame. If the change in the coordinate position of the detection object for the subsequent preset number of frames is less than a preset first threshold, the t1 frame is used as the slicing starting point;

[0185] Obtain the image frame where the detected object disappears from the device area for the first time, recorded as the t2th frame;

[0186] Starting from the t2th frame, if there is no coordinate position information of the detection object in the subsequent n3 frames, the t2th frame is used as the end point.

[0187] The slice generation module 106 is specifically used for:

[0188] The authenticity verification of the test process is to select the tester's body movement changes, equipment operation status or detection object status changes in the second information corresponding to the time node through the start and end time points of the video slice to verify the authenticity of the test process, and the steps include:

[0189] The device operation state is to obtain the coordinates of the key points of the device moving parts at the starting point of the slice, traverse the coordinates of the key points of the device moving parts of the slice from front to back, and calculate the coordinate distance difference between the key points at each moment and the moment of the starting point of the slice; if the ratio of the coordinate distance of the key points at a certain moment to the width of the device or the height of the device is greater than the preset second threshold or third threshold, the device operation state of the slice is normal operation;

[0190] If the operating state of the sliced ​​equipment is normal, the state change of the detection object is change, or the human motion change of the tester is consistent with the human motion change in the corresponding detection object scene, then the test process is real.

[0191] The slice generation module 106 is further specifically configured to:

[0192] Count the number of times the test object category, equipment type, tester identity, and test object number appear at the start and end time nodes respectively;

[0193] Select the test object category, equipment type, tester identity and test object number that appear most frequently in the start and end time nodes as the final test object category, equipment type, tester identity and test object number of the slice;

[0194] If the authenticity verification is not authentic, a non-authenticity warning message will be fed back.

[0195] It should be noted that a video slicing system for an engineering inspection process in the present application corresponds one-to-one to a video slicing method for an engineering inspection process in the present application. The technical features and beneficial effects described in the embodiment of the above-mentioned video slicing method for an engineering inspection process are applicable to the embodiment of a video slicing system for an engineering inspection process. For specific contents, please refer to the description in the embodiment of the method of the present application. No further details will be given here. This is hereby declared.

[0196] In addition, in the implementation of an engineering inspection process video slicing system in the above embodiment, the logical division of each program module is only an example. In actual applications, the above functions can be assigned to different program modules as needed, for example, for the configuration requirements of the corresponding hardware or the convenience of software implementation. That is, the internal structure of the engineering inspection process video slicing system is divided into different program modules to complete all or part of the functions described above.

[0197] In another embodiment, an electronic device for implementing a method for video slicing an engineering inspection process is provided, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor; when the processor executes the computer program, a method for video slicing an engineering inspection process according to any embodiment of the present application is implemented.

[0198] Exemplarily, in this embodiment, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present application. The one or more module elements may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the device.

[0199] The device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The device may include, but is not limited to, a processor and a memory.

[0200] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the device, and various interfaces and lines are used to connect various parts of the entire device.

[0201] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; in addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0202] Accordingly, the present application also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a video slicing method for an engineering inspection process as described in any one of the above embodiments.

[0203] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0204] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0205] The above embodiments are preferred implementation modes of the present application, but the implementation modes of the present application are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present application should be equivalent replacement methods and are included in the protection scope of the present application.

Claims

1. A method for slicing video of an engineering inspection process, characterized in that: The steps include: Get the video stream of the engineering inspection process; Input the video stream into the pre-established multi-target detection model to obtain the position information of the equipment, tester and detection object number display screen; The center point coordinates, width and height of the position information of the device, the tester and the detection object number display screen are respectively transformed, and each frame of the video stream is cropped based on the transformed coordinate position to obtain the first information; The first information includes: an image of the test equipment, an image of the tester, and an image of a display screen showing the number of the test object; Identify and detect the first information to obtain second information; the second information includes: the location of the detection object, the category of the detection object, the type of equipment, the movement trajectory of the equipment, the detection object number, the identity of the tester, the change of the tester's body movement and the change of the detection object state; Acquiring the appearance and disappearance of the detection object according to the position of the detection object, and determining the start and end time points of the video slice according to the appearance and disappearance; the determining the start and end time points of the video slice according to the appearance and disappearance includes: Get the image frame where the detected object first appears in the device area, recorded as the t1th frame; Get the coordinate position of the detected object in the device area image; Starting from the t1th frame, if the coordinate position information of the detection object exists in the subsequent n1 frames, the t1th frame is used as the pre-starting point; Obtain the coordinate position of the detection object at the pre-starting point, and calculate the change in the coordinate position of the detection object for a preset number of frames subsequent to the t1 frame. If the change in the coordinate position of the detection object for the subsequent preset number of frames is less than a preset first threshold, the t1 frame is used as the slicing starting point; Obtain the image frame where the detected object disappears from the device area for the first time, recorded as the t2th frame; Starting from the t2th frame, if there is no coordinate position information of the detected object in the subsequent n3 frames, the t2th frame is taken as the end point; The authenticity of the test process is verified by selecting the second information in the corresponding time node according to the start and end time points of the video slice. If the verification is authentic, the video to be sliced ​​is sliced, and the detection object category, equipment type, tester identity, detection object number and authenticity in the second information at the corresponding time node are bound to the slice.

2. According to claim 1, a method for slicing video of an engineering inspection process, characterized in that: The video stream is input into a pre-established multi-target detection model to obtain the position information of the device, the tester and the detection object number display screen, including: The multi-target detection model is based on the improved Yolov4 algorithm, with Hornet as the backbone network, and also includes ASPP module, PANet module, and Decoupled Head module; The Hornet backbone network extracts three layers of high-level feature maps of different scales from each frame; The ASPP module extracts multi-scale spatial features in the last layer of high-level feature maps at different sampling rates and fuses the multi-scale spatial features to obtain spatial fusion features; The PANet module aggregates the high-level feature maps and spatial fusion features of the first two layers at multiple scales to generate three aggregated feature maps at different scales; The Decoupled Head module extracts the location information of the device, tester, and test object number display screen from the aggregated feature maps at three scales.

3. According to claim 1, a method for slicing video of an engineering inspection process, characterized in that: The identifying and detecting the first information to obtain the second information includes: Performing object detection on the device image using a detection object detection model to obtain location information of the detection object; Performing device recognition on the device image using a device recognition model to obtain device type information; Performing device motion trajectory detection on the device image using a device motion trajectory detection model to obtain device motion trajectory information; The detection object number display screen image is numbered and recognized by a text recognition model to obtain the detection object number information; Performing identity recognition on the tester image through a facial biometric recognition model to obtain the tester's identity information; The state of the detection object image is recognized by using the detection object state recognition model to obtain the state change of the detection object; The detection object image is subjected to category recognition by using a detection object category recognition model to obtain detection object category information; The experimenter's human motion recognition model is used to perform motion recognition on the experimenter's image to obtain the experimenter's human motion changes.

4. According to claim 1, a method for slicing video of an engineering inspection process, characterized in that: The authenticity verification of the test process is to verify the authenticity of the test process by selecting the changes in the body movements of the tester, the operating status of the equipment or the status changes of the detection object in the second information of the corresponding time node through the start and end time points of the video slice.

5. According to claim 4, a method for slicing video of an engineering inspection process is characterized in that: The method of selecting the changes in the body movements of the tester, the operating state of the equipment or the state changes of the detection object in the second information corresponding to the time node through the start and end time points of the video slice to verify the authenticity of the test process includes: The device operation state is to obtain the coordinates of the key points of the device moving parts at the starting point of the slice, traverse the coordinates of the key points of the device moving parts of the slice from front to back, and calculate the coordinate distance difference between the key points at each moment and the moment of the starting point of the slice; if the ratio of the coordinate distance of the key points at a certain moment to the width of the device or the height of the device is greater than the preset second threshold or third threshold, the device operation state of the slice is normal operation; If the operating state of the sliced ​​equipment is normal, the state change of the detection object is change, or the human motion change of the tester is consistent with the human motion change in the corresponding detection object scene, then the test process is real.

6. According to claim 1, a method for slicing video of an engineering inspection process, characterized in that: If the verification is true, the video to be sliced ​​is sliced, and the detection object category, equipment type, tester identity, detection object number and authenticity in the second information of the corresponding time node are bound to the slice, including: Count the number of times the test object category, equipment type, tester identity, and test object number appear at the start and end time nodes respectively; Select the test object category, equipment type, tester identity and test object number that appear most frequently in the start and end time nodes as the final test object category, equipment type, tester identity and test object number of the slice; If the authenticity verification is not authentic, a non-authenticity warning message will be fed back.

7. A video slicing system for an engineering inspection process, characterized in that: A method for slicing a video of an engineering inspection process applied to any one of claims 1 to 6, comprising a video stream acquisition module, a multi-target detection module, a first information acquisition module, a second information acquisition module, an adaptive cut-point module, and a slice generation module; The video stream acquisition module is used to acquire the video stream of the engineering inspection process; The multi-target detection module is used to input the video stream into a pre-established multi-target detection model to obtain the position information of the equipment, the tester and the detection object number display screen; The first information acquisition module is used to transform the center point coordinates, width and height of the position information of the device, the tester and the detection object number display screen respectively, and crop each frame of the video stream based on the transformed coordinate position to obtain the first information; The first information includes: an image of the test equipment, an image of the tester, and an image of a display screen showing the number of the test object; The second information acquisition module is used to identify and detect the first information to obtain second information; the second information includes: the position of the detection object, the category of the detection object, the type of equipment, the movement trajectory of the equipment, the detection object number and the identity of the tester, the change of the tester's body movement and the change of the detection object state; The adaptive cut-point module is used to obtain the appearance and disappearance of the detection object according to the position of the detection object, and determine the start and end time points of the video slice according to the appearance and disappearance; the determination of the start and end time points of the video slice according to the appearance and disappearance includes: Get the image frame where the detected object first appears in the device area, recorded as the t1th frame; Get the coordinate position of the detected object in the device area image; Starting from the t1th frame, if the coordinate position information of the detection object exists in the subsequent n1 frames, the t1th frame is used as the pre-starting point; Obtain the coordinate position of the detection object at the pre-starting point, and calculate the change in the coordinate position of the detection object for a preset number of frames subsequent to the t1 frame. If the change in the coordinate position of the detection object for the subsequent preset number of frames is less than a preset first threshold, the t1 frame is used as the slicing starting point; Obtain the image frame where the detected object disappears from the device area for the first time, recorded as the t2th frame; Starting from the t2th frame, if there is no coordinate position information of the detected object in the subsequent n3 frames, the t2th frame is taken as the end point; The slice generation module is used to select the second information in the corresponding time node according to the start and end time points of the video slice to verify the authenticity of the test process. If the verification is authentic, the video to be sliced ​​is sliced, and the detection object category, equipment type, tester identity, detection object number and authenticity in the second information of the corresponding time node are bound to the slice.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute a video slicing method for an engineering inspection process as described in any one of claims 1-6.

9. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, a video slicing method for an engineering inspection process as described in any one of claims 1 to 6 is implemented.

Images