Monitoring offset identification method, device and equipment and storage medium

Abstract

The application relates to the technical field of artificial intelligence, and discloses a monitoring deviation identification method, device, equipment and storage medium, the method comprises the following steps: obtaining monitoring video data to be identified and performing frame extraction processing to obtain a target image data group; the target image data group is input into a monitoring deviation identification model to obtain a deviation identification result; wherein the monitoring deviation identification model is obtained by training a classification model fusing depth information according to sample data. By using the method disclosed by the application, the case that a stable reference object or template cannot be determined due to object movement in a monitoring picture in deviation detection can be avoided, and the problem of automatic detection of monitoring deviation in a logistics site in the prior art is effectively solved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a monitoring offset recognition method, apparatus, device, and storage medium. Background Technology

[0002] Nowadays, logistics sites are commonly equipped with surveillance cameras to ensure cargo safety and prevent loss or damage that could lead to loss or damage. During installation, personnel calibrate the monitoring angle of these cameras. However, with use, factors such as accidental collisions during transport or loosening of fasteners due to natural causes can cause the cameras to shift, making it impossible to continue monitoring the corresponding area. If the site manager fails to notice this shift in time, the surveillance system loses its supervisory function. Therefore, logistics companies have a need for automated detection systems to detect camera shifts in their sites.

[0003] Existing camera offset detection schemes based on image processing technology include: (1) using a pre-set reference object and template matching method to determine whether the reference object has shifted in the image to determine whether the camera has shifted; (2) for specific scenes, such as inside a vertical elevator, semantic segmentation is performed first, and then target detection is combined to detect the position of the elevator door, and the camera is determined by calculating the offset of the door; (3) straight line segments in the image are identified as comparison templates, and if the straight line segments in the current image are offset from the template straight line segments, it is considered that the camera has shifted.

[0004] The above solutions are not sufficiently adaptable to logistics industry scenarios. The problem lies in the fact that when the monitored area is a ground-level area where goods are being transferred, the movement of goods makes it impossible to identify a stable reference point or template in the image, thus hindering effective detection. Therefore, how to automate the detection of offsets in logistics site monitoring is a pressing issue. Summary of the Invention

[0005] This invention provides a monitoring offset identification method, device, equipment, and storage medium to solve the problem in the prior art where the movement of goods in logistics sites makes it impossible to identify whether the monitoring is offset, resulting in the inability to identify whether the monitoring is offset.

[0006] To address the aforementioned technical problems, in a first aspect, the present invention provides a method for monitoring offset identification, the method comprising:

[0007] The surveillance video data to be identified is acquired and frame-sampling is performed to obtain the target image data set.

[0008] The target image data set is input into the monitoring offset recognition model to obtain the offset recognition result; wherein, the monitoring offset recognition model is obtained by training a classification model that fuses depth information based on the sample data.

[0009] Optionally, the step of inputting the target image data set into the monitoring offset recognition model to obtain the offset recognition result includes:

[0010] The target image data set is input into the monitoring offset recognition model to extract the depth information of each target image data in the target image data set, thus obtaining the depth image data set.

[0011] Feature extraction and region division are performed on the depth image data group in both the time and spatial dimensions to obtain several temporal features and several sets of spatial features corresponding to each region.

[0012] Feature fusion is performed on each temporal feature and each set of spatial features corresponding to the same region to obtain several fused features;

[0013] The offset recognition results are obtained by classifying the various fusion features.

[0014] Optionally, the monitoring offset recognition model includes a first encoder and a second encoder; the step of extracting features and dividing regions of the depth image data group in the time and spatial dimensions respectively to obtain several temporal features and several sets of spatial features corresponding to each region includes:

[0015] Using the first encoder, feature information of each depth image data in the depth image data group is extracted to obtain several depth feature data with temporal sequence; each depth feature data is divided into non-overlapping regions, and feature data of the same region in each depth feature data are accumulated to obtain several time features corresponding to each region.

[0016] Using the second encoder, feature information of each depth image data in the depth image data group is extracted to obtain several depth feature data with temporal sequence; each depth feature data is divided into non-overlapping regions, and the feature data of the same region in each depth feature data is taken as a set of spatial features to obtain several sets of spatial features corresponding to each region.

[0017] Optionally, the monitoring offset recognition model includes a depth estimation layer; the step of extracting the depth information of each target image data in the target image data group includes:

[0018] Using the aforementioned depth estimation layer, a monocular depth estimation algorithm is employed to extract the depth information of each target image data in the target image data set.

[0019] Optionally, the monitoring offset recognition model includes a feature fusion layer; the feature fusion layer includes a first multi-head self-attention module, a second multi-head self-attention module, a first multilayer perceptron, and a second multilayer perceptron; the step of fusing the temporal features and spatial features corresponding to the same region to obtain several fused features includes:

[0020] Using the first multi-head self-attention module, the sequence relationship features corresponding to the current region are extracted to obtain the first hidden feature, and the time feature is fused with the first hidden feature to obtain the first intermediate feature;

[0021] Using the second multi-head self-attention module, sequence relationship features are extracted from a set of spatial features corresponding to the current region to obtain the second hidden feature, and the set of spatial features and the second hidden feature are fused to obtain the second intermediate feature;

[0022] Using the first multilayer perceptron, channel mapping is performed on the second intermediate feature to obtain the mapped feature;

[0023] Using the second multilayer perceptron, channel mapping is performed on the mapped features and the first intermediate features to obtain the fused features corresponding to the current region.

[0024] Optionally, the monitoring offset recognition model includes a classification layer; the step of performing classification processing based on each fusion feature to obtain the offset recognition result includes:

[0025] Using the classification layer, each fusion feature is classified separately, and the results of each classification are combined to obtain the offset recognition result; wherein, the offset recognition result includes no offset and offset.

[0026] Optionally, the monitoring offset recognition model is obtained by training a classification model that fuses deep information based on sample data, including:

[0027] Acquire both un-offset and offset surveillance video data and perform frame extraction to obtain several sample image data sets.

[0028] Each sample image data group is labeled with a corresponding category label to obtain sample data; wherein, the category label includes unoffset and offset;

[0029] Based on the sample data, with each sample image data group as input and the labeled category label as the target output, a pre-constructed classification model that integrates depth information is trained to obtain the monitoring offset recognition model.

[0030] In a second aspect, the present invention provides a monitoring offset identification device, comprising:

[0031] The acquisition module is used to acquire the surveillance video data to be identified and perform frame extraction processing to obtain the target image data group;

[0032] The recognition module is used to input the target image data set into the monitoring offset recognition model to obtain the offset recognition result; wherein, the monitoring offset recognition model is obtained by training a classification model that fuses depth information based on the sample data.

[0033] Thirdly, the present invention provides a computer device, including a memory and a processor, wherein:

[0034] The memory is used to store computer programs;

[0035] The processor is used to read the program in the memory and execute the steps of the monitoring offset identification method provided in the first aspect above.

[0036] Fourthly, the present invention provides a computer-readable storage medium having a readable computer program stored thereon, which, when executed by a processor, implements the steps of the monitoring offset identification method provided in the first aspect above.

[0037] Compared with the prior art, the monitoring offset identification method, apparatus, device, and storage medium provided by the present invention have the following beneficial effects:

[0038] By acquiring the surveillance video data to be identified and performing frame extraction processing, a target image data group is obtained; the target image data group is input into the surveillance offset recognition model to obtain the offset recognition result; the present invention combines depth information and classification model to identify surveillance video, which can avoid the situation in the surveillance screen where a stable reference object or template cannot be determined due to the movement of objects in the surveillance screen during offset detection, and effectively solves the problem of automated detection of offset in logistics site monitoring in the prior art. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and not all embodiments. For those skilled in the art, other drawings obtained from these drawings without creative effort are all within the scope of protection of the present invention.

[0040] Figure 1 This is a flowchart of the monitoring offset identification method according to an embodiment of the present invention;

[0041] Figure 2 This is a flowchart of the training method for the monitoring offset recognition model according to an embodiment of the present invention;

[0042] Figure 3This is a schematic diagram of the network structure of the monitoring offset recognition model according to an embodiment of the present invention;

[0043] Figure 4 This is a flowchart illustrating how the target image data set is input into the monitoring offset recognition model to obtain the offset recognition result in the monitoring offset recognition method of this invention.

[0044] Figure 5 This is a schematic diagram of the network structure of the feature fusion layer in an embodiment of the present invention;

[0045] Figure 6 This is a schematic diagram of the monitoring offset identification device according to an embodiment of the present invention;

[0046] Figure 7 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention;

[0047] Figure 8 This is a schematic diagram of the structure of a computer-readable storage medium according to an embodiment of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0049] To make the description of this disclosure more detailed and complete, illustrative descriptions of embodiments and specific examples of the present invention are provided below; however, this is not the only form of implementing or utilizing the specific examples of the present invention. The embodiments cover features of multiple specific examples and the methods, steps, and sequences for constructing and operating these specific examples. However, other specific examples can also be used to achieve the same or equivalent functions and sequence of steps. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present invention.

[0050] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in sequences other than those illustrated or described herein.

[0051] This invention relates to the field of artificial intelligence technology. This invention can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0052] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0053] The embodiments of the present invention can be applied to logistics scenarios to achieve automated identification of monitoring offset. It can solve the problem that when the monitoring area only captures the ground area where goods are transferred, there is movement of goods in each time period, and there is no stable reference object or template in the picture, which makes it impossible to identify whether the monitoring is offset.

[0054] Example 1

[0055] like Figure 1 The flowchart described above is a monitoring offset identification method provided by an embodiment of the present invention, which includes the following steps.

[0056] Step S101: Acquire the surveillance video data to be identified and perform frame extraction processing to obtain the target image data group.

[0057] In step S101, the surveillance video data to be identified is acquired from a monitoring device, such as a surveillance camera. Surveillance cameras are typically installed at a viewing angle that allows for effective monitoring of logistics operations, such as goods stacking and handling. The duration of the acquired surveillance video data can be set as needed; this solution does not limit this. The acquired surveillance video data undergoes further frame extraction processing, and the extracted frames are combined into a target image data group. For example, a 3-second video segment is selected as the surveillance video data to be identified. Existing video frame extraction tools are used to extract image frames, either continuously or intermittently, such as extracting one frame every 0.5 seconds for a total of 6 frames. These 6 frames then form the target image data group. Similarly, this invention does not limit the number of frames extracted as the target image data group. It is understood that a higher number of frames in the target image data group will improve the accuracy of the identification results.

[0058] Step S102: Input the target image data group into the monitoring offset recognition model to obtain the offset recognition result; wherein, the monitoring offset recognition model is obtained by training a classification model that fuses depth information based on the sample data.

[0059] The aforementioned monitoring offset recognition model is obtained by training a classification model that fuses deep information based on sample data. In one feasible embodiment, please refer to... Figure 2 The monitoring offset recognition model can be trained using the following steps:

[0060] Step S201: Obtain the un-offset surveillance video data and the offset surveillance video data, and perform frame extraction processing to obtain several sample image data groups.

[0061] During model training, it is necessary to collect both un-offset and offset surveillance video data. In an optional embodiment, data collection can be performed through manual intervention. This involves actively rotating the camera from its unoffset state by a certain angle, then segmenting the surveillance video data from the camera's transition from the unoffset to the offset state. Each video segment is then processed by frame extraction to obtain multiple sample image data sets. It is understood that the frame extraction process for each segment is the same as the frame extraction process for the surveillance data to be identified, and the number of image frames in the resulting sample image data sets is the same as the number of image frames in the target image data sets.

[0062] Step S202: Label each sample image data group with the corresponding category label to obtain sample data; wherein, the category label includes unoffset and offset.

[0063] Data collection is performed according to step S201. Camera offset rotation can be performed by incorporating various variables such as different time periods, dates, and locations to obtain a rich set of sample image data. Then, each set of sample image data is labeled. Specifically, the set of sample image data that includes the monitored offset process is labeled as offset, and the remaining set is labeled as non-offset, thus obtaining sample data. In one embodiment, there are 10,000 sets of sample data in total, with 5,000 sets each for the non-offset and offset categories, and each set consisting of 6 images. After labeling, 4,000 sets from each category are used as the training set, 500 sets as the validation set, and 500 sets as the test set.

[0064] Step S203: Based on the sample data, using each sample image data group as input and the labeled category label as the target output, train the pre-constructed classification model that integrates depth information to obtain the monitoring offset recognition model.

[0065] The training and validation sets are input into the pre-built classification model that integrates depth information to predict the category label of each sample image data group. The parameters of the classification model are optimized based on the difference between the predicted and true values ​​of the category labels. The test set is input into the classification model for testing until the error of the classification result meets the preset threshold, thus obtaining the trained monitoring offset recognition model.

[0066] In one feasible embodiment, please refer to Figure 3 The network structure of the trained monitoring offset recognition model 30 includes a depth estimation layer 31, a first encoder 32, a second encoder 33, a feature fusion layer 34, and a classification layer 35.

[0067] After the aforementioned surveillance video data to be identified is processed into target image data groups through frame extraction, it is first input to the depth estimation layer 31 to extract the depth information of each target image data in the target image data group, and then output to the first encoder 32 and the second encoder 33 respectively. The first encoder 32 performs temporal feature extraction and region division on the output results of the depth estimation layer 31 to obtain several temporal features corresponding to each region. The second encoder 33 performs spatial feature extraction and region division on the output results of the depth estimation layer 31 to obtain several sets of spatial features corresponding to each region. The feature fusion layer 34 performs feature fusion on the several temporal features and several sets of spatial features corresponding to each region obtained by the first encoder 32 and the second encoder 33 to obtain several fused features. The classification layer 35 performs classification processing based on the results of the feature fusion layer 34 to obtain the offset recognition result. In this embodiment of the invention, the first encoder 32 and the second encoder 33 can be mature encoders in the industry. In an optional embodiment, both the first encoder 32 and the second encoder 33 are Transformer encoders. Transformer encoders have advantages over encoders using convolutional neural networks in sequence modeling and global information feature representation, resulting in higher model recognition accuracy.

[0068] In one feasible embodiment, please refer to Figure 4 The process of inputting the target image data set into the monitoring offset recognition model to obtain the offset recognition result may also include:

[0069] Step S1021: Input the target image data set into the monitoring offset recognition model, extract the depth information of each target image data in the target image data set, and obtain the depth image data set.

[0070] In step S1021, depth estimation techniques are used to extract the depth information of each target image data in the target image data set. Commonly used depth estimation techniques mainly utilize binocular stereo vision or monocular depth estimation algorithms, and this invention does not limit this. In an optional embodiment, a depth estimation layer is used, employing a monocular depth estimation algorithm, to extract the depth information of each target image data in the target image data set. Specifically, the monocular depth estimation network can use NeW CRFs (Conditional Random Fields) to perform depth estimation on each target image data in the target image data set, obtaining a depth image of each target image data. For example, if the target image data set includes 6 target image data, the target image data set is input into the monocular depth estimation network to extract the depth information of the 6 target image data, resulting in a depth image data set.

[0071] Step S1022: Perform feature extraction and region division on the depth image data group in both the time and spatial dimensions to obtain several temporal features and several sets of spatial features corresponding to each region.

[0072] In step S1022, the first encoder is used to extract the feature information of each depth image data in the depth image data group to obtain several depth feature data with temporal sequence; the depth feature data is divided into non-overlapping regions, and the feature data of the same region in each depth feature data is accumulated to obtain several temporal features corresponding to each region; the second encoder is used to extract the feature information of each depth image data in the depth image data group to obtain several depth feature data with temporal sequence; the depth feature data is divided into non-overlapping regions, and the feature data of the same region in each depth feature data is taken as a set of spatial features to obtain several sets of spatial features corresponding to each region.

[0073] In one optional embodiment, for example, the image size of the depth image data set is 224x224. The token region can be divided into non-overlapping 16x16 image blocks. Features are extracted from the image blocks of the same region in each image and added together to obtain several temporal features corresponding to each region. Spatial features are also extracted from the same region tokens in images at different times, but the extracted features are independent of each other. A token refers to a non-overlapping segmented region of the image.

[0074] By fusing temporal and spatial features, since temporal features are obtained by adding the features of corresponding regions after non-overlapping segmentation of the same regions in images from different times, the depth information of the same regions in a set of image data at consecutive times is fused, giving the features temporal sequence.

[0075] Step S1023: Perform feature fusion on each temporal feature and each group of spatial features corresponding to the same region to obtain several fused features.

[0076] In step S1023, in one possible embodiment, please refer to... Figure 5 The feature fusion layer 34 includes a first multi-head self-attention module 341, a second multi-head self-attention module 343, a first multilayer perceptron 342, and a second multilayer perceptron 344. In this embodiment, the first multi-head self-attention module 341 extracts sequence relationship features from the temporal features corresponding to the current region to obtain a first hidden feature, and fuses the temporal features with the first hidden feature to obtain a first intermediate feature; the second multi-head self-attention module 343 extracts sequence relationship features from a set of spatial features corresponding to the current region to obtain a second hidden feature, and fuses the set of spatial features with the second hidden feature to obtain a second intermediate feature; the first multilayer perceptron 342 performs channel mapping on the second intermediate feature to obtain a mapped feature; and the second multilayer perceptron 344 performs channel mapping on the mapped feature and the first intermediate feature to obtain a fused feature corresponding to the current region.

[0077] Step S1024: Perform classification processing based on each fusion feature to obtain the offset recognition result.

[0078] In step S1024, the classification layer is used to classify each fusion feature separately, and the results of each classification process are combined to obtain the offset recognition result; wherein, the offset recognition result includes no offset and offset.

[0079] The monitoring offset recognition method of this invention obtains the monitoring video data to be identified and performs frame extraction processing to obtain a target image data group; the target image data group is input into the monitoring offset recognition model to obtain the offset recognition result; this invention combines depth information and classification model to identify monitoring video, which can avoid the situation in the monitoring screen where a stable reference object or template cannot be determined due to the movement of objects in the monitoring screen during offset detection, and effectively solves the problem of automated detection of offset in logistics site monitoring in the prior art.

[0080] Example 2

[0081] Based on the above-described monitoring offset identification method, embodiments of the present invention provide a monitoring offset identification device, such as... Figure 6 As shown, the monitoring offset identification device 60 includes: an acquisition module 61 and an identification module 62.

[0082] The acquisition module 61 is used to acquire the surveillance video data to be identified and perform frame extraction processing to obtain the target image data group;

[0083] The recognition module 62 is used to input the target image data group into the monitoring offset recognition model to obtain the offset recognition result; wherein, the monitoring offset recognition model is obtained by training a classification model that fuses depth information based on the sample data.

[0084] For other details regarding the implementation of the above technical solution by each module in the above monitoring offset identification device, please refer to the description of the monitoring offset identification method provided in the above invention embodiments, which will not be repeated here.

[0085] Based on the above-mentioned monitoring offset identification method, such as Figure 7 As shown in the diagram, this embodiment of the invention also provides a schematic diagram of a computer device 70, which includes a processor 71 and a memory 72 coupled to the processor 71. The memory 72 stores a computer program, which, when executed by the processor 71, causes the processor 71 to perform the steps of the monitoring offset identification method described in the above embodiment.

[0086] For other details regarding the implementation of the above technical solution by the processor 71 in the computer device 70, please refer to the description in the monitoring offset identification method provided in the above embodiments of the invention, which will not be repeated here.

[0087] The processor 71 can also be called a CPU (Central Processing Unit). The processor 71 may be an integrated circuit chip with signal processing capabilities. The processor 71 can also be a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor, or the processor 71 can be any conventional processor.

[0088] like Figure 8As shown in the diagram, this embodiment of the invention also provides a schematic diagram of a computer-readable storage medium, on which a readable computer program 81 is stored. The computer program 81 can be stored in the storage medium in the form of a software product, including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in various embodiments of the invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks or optical disks, ROM (Read-Only Memory), RAM (Random Access Memory), or terminal devices such as computers, servers, mobile phones, and tablets.

[0089] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, apparatuses, or modules, and may be electrical, mechanical, or other forms.

[0090] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0091] Furthermore, the functional modules in the various embodiments of this invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0092] In the above embodiments, the implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of a computer program product.

[0093] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0094] The technical solution provided by the present invention has been described in detail above. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

[0095] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for monitoring offset identification, characterized in that, include: The surveillance video data to be identified is acquired and frame-sampling is performed to obtain the target image data set. The target image data set is input into the monitoring offset recognition model to extract the depth information of each target image data in the target image data set, thus obtaining the depth image data set. Feature extraction and region division are performed on the depth image data group in both the time and spatial dimensions to obtain several temporal features and several sets of spatial features corresponding to each region. Feature fusion is performed on each temporal feature and each group of spatial features corresponding to the same region to obtain several fused features; classification processing is performed based on each fused feature to obtain the offset recognition result; wherein, the monitoring offset recognition model is obtained by training a classification model that fuses depth information based on sample data.

2. The monitoring offset identification method according to claim 1, characterized in that, The monitoring offset recognition model includes a first encoder and a second encoder; the steps of extracting features and dividing regions of the depth image data group in the time and spatial dimensions respectively to obtain several temporal features and several sets of spatial features corresponding to each region include: Using the first encoder, feature information of each depth image data in the depth image data group is extracted to obtain several depth feature data with temporal sequence; each depth feature data is divided into non-overlapping regions, and the feature data of the same region in each depth feature data are accumulated to obtain several time features corresponding to each region. Using the second encoder, feature information of each depth image data in the depth image data group is extracted to obtain several depth feature data with temporal sequence; each depth feature data is divided into non-overlapping regions, and the feature data of the same region in each depth feature data is taken as a set of spatial features to obtain several sets of spatial features corresponding to each region.

3. The monitoring offset identification method according to claim 1, characterized in that, The monitoring offset recognition model includes a depth estimation layer; the step of extracting the depth information of each target image data in the target image data group includes: Using the aforementioned depth estimation layer, a monocular depth estimation algorithm is employed to extract the depth information of each target image data in the target image data set.

4. The monitoring offset identification method according to claim 1, characterized in that, The monitoring offset recognition model includes a feature fusion layer; the feature fusion layer includes a first multi-head self-attention module, a second multi-head self-attention module, a first multilayer perceptron, and a second multilayer perceptron; the step of fusing features of each temporal feature and each group of spatial features corresponding to the same region to obtain several fused features includes: Using the first multi-head self-attention module, the sequence relationship features corresponding to the current region are extracted to obtain the first hidden feature, and the time feature is fused with the first hidden feature to obtain the first intermediate feature; Using the second multi-head self-attention module, sequence relationship features are extracted from a set of spatial features corresponding to the current region to obtain the second hidden feature, and the set of spatial features and the second hidden feature are fused to obtain the second intermediate feature; Using the first multilayer perceptron, channel mapping is performed on the second intermediate feature to obtain the mapped feature; Using the second multilayer perceptron, channel mapping is performed on the mapped features and the first intermediate features to obtain the fused features corresponding to the current region.

5. The monitoring offset identification method according to claim 1, characterized in that, The monitoring offset recognition model includes a classification layer; the step of performing classification processing based on each fusion feature to obtain the offset recognition result includes: Using the classification layer, each fusion feature is classified separately, and the results of each classification are combined to obtain the offset recognition result; wherein, the offset recognition result includes no offset and offset.

6. The monitoring offset identification method according to any one of claims 1 to 5, characterized in that, The monitoring offset recognition model is obtained by training a classification model that incorporates deep information based on sample data, including: Acquire both un-offset and offset surveillance video data and perform frame extraction to obtain several sample image data sets. Each sample image data group is labeled with a corresponding category label to obtain sample data; wherein, the category label includes unoffset and offset; Based on the sample data, with each sample image data group as input and the labeled category label as the target output, a pre-constructed classification model that integrates depth information is trained to obtain the monitoring offset recognition model.

7. A monitoring offset identification device, characterized in that, include: The acquisition module is used to acquire the surveillance video data to be identified and perform frame extraction processing to obtain the target image data group; The recognition module is used to input the target image data group into the monitoring offset recognition model, extract the depth information of each target image data in the target image data group, and obtain the depth image data group. Feature extraction and region division are performed on the depth image data group in both the time and spatial dimensions to obtain several temporal features and several sets of spatial features corresponding to each region. Feature fusion is performed on each temporal feature and each group of spatial features corresponding to the same region to obtain several fused features; classification processing is performed based on each fused feature to obtain the offset recognition result; wherein, the monitoring offset recognition model is obtained by training a classification model that fuses depth information based on sample data.

8. A computer device, characterized in that, Includes memory and processor, wherein: The memory is used to store computer programs; The processor is used to read the computer program in the memory and execute the steps of any of the monitoring offset identification methods as claimed in claims 1 to 6.

9. A computer-readable storage medium, characterized in that, It stores a readable computer program that, when executed by a processor, implements the steps of any of the monitoring offset identification methods as claimed in claims 1 to 6.

Images