Dual-positioning inspection behavior detection method, equipment and device

Abstract

The invention belongs to the technical field of behavior monitoring, and discloses a dual-positioning inspection behavior detection method, equipment and device. The method comprises the following steps: respectively acquiring first positioning information and second positioning information of an inspector; determining target positioning information according to the first positioning information and the second positioning information; determining a corresponding inspection area based on the target positioning information, and acquiring an area image of the inspection area through monitoring equipment; judging whether the inspector exists in the inspection area or not based on the area image; and if yes, detecting the inspection behavior of the inspector based on the area image. Through the above mode, the inspection behavior of the inspector is detected in combination with dual positioning, and the real-time performance and accuracy of inspection behavior detection are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of behavior monitoring, and in particular to a dual-positioning inspection behavior detection method, equipment and device. Background Art

[0002] In order to ensure the stable operation of the power system, it is necessary to conduct regular inspections of transmission lines and distribution rooms. Currently, most transmission lines and distribution rooms are inspected manually. Inspectors need to follow certain specifications when conducting inspections to ensure the safety and effectiveness of the inspections. Therefore, the detection of inspection behaviors has become an important part of the inspection work.

[0003] Currently, the detection of inspection behavior does not combine positioning with behavior detection. In actual situations, the detection of inspection behavior needs to be combined with the actual inspection area. It only compares with the set behavior without considering the actual situation, which affects the accuracy of the detection.

[0004] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of the present invention is to provide a dual-positioning inspection behavior detection, equipment and device, aiming to solve the technical problem that the current inspection behavior detection does not combine positioning with behavior detection. In actual situations, the inspection behavior detection needs to be combined with the actual inspection area and is only compared according to the set behavior without considering the actual situation, which affects the accuracy of the detection.

[0006] To achieve the above object, the present invention provides a dual-positioning inspection behavior detection method, the dual-positioning inspection behavior method comprising the following steps:

[0007] respectively obtain the first positioning information and the second positioning information of the inspector;

[0008] Determine target positioning information according to the first positioning information and the second positioning information;

[0009] Determine a corresponding inspection area based on the target positioning information, and obtain an area image of the inspection area through a monitoring device;

[0010] Determining whether the inspector exists in the inspection area based on the area image;

[0011] If so, the inspection behavior of the inspector is detected based on the area image.

[0012] In some embodiments, determining the target positioning information according to the first positioning information and the second positioning information includes:

[0013] Determining a first reference area corresponding to the first positioning information;

[0014] Determining a second reference area corresponding to the second positioning information;

[0015] determining a target area based on the first reference area and the second reference area;

[0016] Respectively obtaining time series corresponding to the first positioning information and the second positioning information;

[0017] fusing the first positioning information and the second positioning information based on the time series to obtain fused positioning information;

[0018] Target positioning information is obtained according to the fused positioning information and the target area.

[0019] In some embodiments, the method further comprises:

[0020] Acquire an original regional image of the inspection area through the monitoring device, and perform binarization processing on the original regional image to obtain a first regional image;

[0021] Performing filtering processing on the first region image to obtain a second region image;

[0022] The second image is subjected to corrosion processing to obtain a regional image of the inspection area.

[0023] In some embodiments, the method further comprises:

[0024] Dividing the original region image into two sub-region images less than or equal to the set pixel threshold and greater than the set pixel threshold according to a set pixel threshold, respectively calculating a first grayscale average value and a second grayscale average value of the two sub-region images, and if a difference between the first grayscale average value and the second grayscale average value meets a preset condition, binarizing the original region image according to the set pixel threshold to obtain a first region image;

[0025] Selecting any pixel point from the first regional image, calculating the grayscale average value of all pixel points in a window centered on the pixel point, and using the grayscale average value as a new grayscale value of the selected pixel point to obtain a second regional image;

[0026] The center of the structural element is placed on each pixel of the second area image, and it is determined whether all elements in the structural element are located in the foreground of the second area image. If all elements of the structural element are located in the foreground, the pixel value of the corresponding center point in the second area image is retained; otherwise, the pixel value of the corresponding center point in the second area image is replaced with the background pixel value.

[0027] In some embodiments, judging whether the inspector exists in the inspection area based on the area image includes:

[0028] Determine whether there is anyone in the inspection area based on the area image;

[0029] If a person is detected in the inspection area, the corresponding facial features and appearance features are obtained;

[0030] Comparing the facial features and the appearance features respectively;

[0031] If the facial features and the appearance features are consistent, it is determined that the person in the inspection area is the same inspector.

[0032] In some embodiments, the detecting the inspection behavior of the inspector based on the regional image includes:

[0033] Using a feature extraction model to extract a first feature and a second feature of the sample area image respectively;

[0034] Inputting the first feature and the second feature into an attention module for training, wherein the first feature and the second feature are trained separately;

[0035] In the first stage of training, the attention module and the classification module are frozen, and the conditional variational autoencoder is trained;

[0036] In a first stage of training, the conditional variational autoencoder is frozen, and the frozen attention module and the classification module are trained;

[0037] The classification scores output by the classification module that are higher than the threshold are taken as the corresponding action categories, and a corresponding time interval and action classification confidence are generated for each action category to complete the inspection behavior recognition training, wherein the time interval is the start time and end time of the action;

[0038] The inspection behavior is identified based on the regional image using an inspection behavior detection model, wherein the inspection behavior detection model is composed of the attention module, the classification module and the conditional variational autoencoder.

[0039] In some embodiments, the method further comprises:

[0040] Obtaining the detection result output by the inspection behavior detection model;

[0041] Determining the type of inspection behavior of the inspector based on the detection result;

[0042] Determine the type of standard inspection behavior according to the inspection area;

[0043] Determining whether the inspection behavior type of the inspector matches the standard inspection behavior type;

[0044] If they do not match, an abnormal behavior alarm will be output.

[0045] In addition, to achieve the above-mentioned purpose, the present invention also proposes a dual-positioning inspection behavior detection device, the dual-positioning inspection behavior detection device comprising:

[0046] An acquisition module, used to respectively acquire the first positioning information and the second positioning information of the inspector;

[0047] A fusion module, used to determine target positioning information according to the first positioning information and the second positioning information;

[0048] The acquisition module is used to determine the corresponding inspection area based on the target positioning information, and acquire the regional image of the inspection area through the monitoring device;

[0049] A judgment module, used for judging whether the inspector exists in the inspection area based on the area image;

[0050] A detection module is used to detect the inspection behavior of the inspector based on the area image, if any.

[0051] In some embodiments, the fusion module is used to determine a first reference area corresponding to the first positioning information;

[0052] Determining a second reference area corresponding to the second positioning information;

[0053] determining a target area based on the first reference area and the second reference area;

[0054] Respectively obtaining time series corresponding to the first positioning information and the second positioning information;

[0055] fusing the first positioning information and the second positioning information based on the time series to obtain fused positioning information;

[0056] Target positioning information is obtained according to the fused positioning information and the target area.

[0057] In addition, to achieve the above-mentioned purpose, the present invention also proposes a dual-positioning inspection behavior detection device, which includes: a memory, a processor, and a dual-positioning inspection behavior detection program stored on the memory and executable on the processor, and the dual-positioning inspection behavior detection program is configured to implement the steps of the dual-positioning inspection behavior detection method as described above.

[0058] The present invention obtains the first positioning information and the second positioning information of the inspector respectively; determines the target positioning information according to the first positioning information and the second positioning information; determines the corresponding inspection area based on the target positioning information, and obtains the regional image of the inspection area through the monitoring device; determines whether the inspector exists in the inspection area based on the regional image; if so, detects the inspection behavior of the inspector based on the regional image. In the above manner, the inspection behavior of the inspector is detected in combination with dual positioning, which improves the real-time and accuracy of the inspection behavior detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a flowchart of the first embodiment of the dual positioning inspection behavior detection method of the present invention;

[0060] Figure 2 This is a structural block diagram of the first embodiment of the dual-positioning inspection behavior detection device of the present invention.

[0061] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0062] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0063] The embodiment of the present invention provides a dual positioning inspection behavior detection method, referring to Figure 1 , Figure 1 It is a flowchart diagram of a first embodiment of a dual-positioning inspection behavior detection method of the present invention.

[0064] In this embodiment, the dual positioning inspection behavior detection method includes the following steps:

[0065] Step S10: respectively obtain the first positioning information and the second positioning information of the inspector.

[0066] In this embodiment, the executor of this embodiment is a dual-positioned inspection behavior detection device, wherein the dual-positioned inspection behavior detection device has functions such as data processing, data communication and program running. The dual-positioned inspection behavior detection device can be a computer terminal device or other network device, and of course it can also be other devices with similar functions, and this embodiment does not limit this.

[0067] It should be noted that the current detection of inspection behavior does not combine positioning with behavior detection. In actual situations, the detection of inspection behavior needs to be combined with the actual inspection area. Only comparison is performed according to the set behavior without considering the actual situation, which affects the accuracy of the detection.

[0068] In order to solve the above technical problems, this embodiment obtains the first positioning information and the second positioning information of the inspector respectively; determines the target positioning information according to the first positioning information and the second positioning information; determines the corresponding inspection area based on the target positioning information, and obtains the regional image of the inspection area through the monitoring equipment; determines whether the inspector exists in the inspection area based on the regional image; if so, detects the inspection behavior of the inspector based on the regional image. Through the above method, the inspection behavior of the inspector is detected in combination with dual positioning, which improves the real-time and accuracy of the inspection behavior detection. Specifically, it can be implemented in the following way.

[0069] In the specific implementation, in this embodiment, the first positioning information and the second positioning information of the inspector need to be obtained respectively. The inspector is a person who performs the inspection work. In addition, it should be noted that the inspector can also be replaced by an inspection robot. According to the same method of this solution, the inspection behavior of the inspector robot can also be detected. Among them, the first positioning information can be obtained through the GPS system, and the second positioning information can be obtained through the communication base station.

[0070] Step S20: Determine target positioning information according to the first positioning information and the second positioning information.

[0071] In a specific implementation, after obtaining the first positioning information and the second positioning information, the final target positioning information can be obtained based on the two positioning information in this embodiment. The specific process is to determine the first reference area corresponding to the first positioning information; determine the second reference area corresponding to the second positioning information; determine the target area based on the first reference area and the second reference area; obtain the time series corresponding to the first positioning information and the second positioning information respectively; fuse the first positioning information and the second positioning information based on the time series to obtain the fused positioning information; obtain the target positioning information based on the fused positioning information and the target area.

[0072] It should be noted that the positioning accuracy of the GPS system and the communication base station is different. The GPS system is suitable for positioning in open areas, while the communication base station is more suitable for positioning in indoor environments. The target area can be determined by combining the two. Take the inspection of a power plant as an example. The power plant includes the main plant area, the generator area, the boiler room area, the distribution room area, and the transmission and transformation area. The first reference area in this embodiment is any one of the above areas. It is further assumed that the first reference area is the distribution room area, and the distribution room area includes the switchgear area, the transformer area, and the bus area. The corresponding second reference area is assumed to be the transformer area, and the final target area is the bus area of ​​the power plant distribution room area. Align the first positioning information and the second positioning information at the same timestamp according to the time series, that is, merge the positioning information at the same timestamp to avoid positioning delays. The obtained fused positioning information combined with the above target area can obtain more accurate target positioning information, such as the first busbar cabinet in the busbar area of ​​the power plant distribution room area of ​​the inspector.

[0073] Step S30: determining a corresponding inspection area based on the target positioning information, and acquiring an area image of the inspection area through a monitoring device.

[0074] After obtaining the above target positioning information, the corresponding inspection area can be determined, and then the monitoring equipment in the inspection area is started, and the regional image in the inspection area is collected by the monitoring equipment.

[0075] Furthermore, in this embodiment, the inspection behavior of the inspector is detected based on image recognition. In order to make the detection more accurate, it is necessary to process the original area image in the inspection area. Specifically, the original area image of the inspection area is obtained by the monitoring device, and the original area image is binarized to obtain a first area image; the first area image is filtered to obtain a second area image; the second image is corroded to obtain an area image of the inspection area.

[0076] It should be noted that before performing the binarization process, it is necessary to determine the pixel threshold first. The specific process is to divide the original area image into two sub-area images less than or equal to the set pixel threshold and greater than the set pixel threshold according to the set pixel threshold, and calculate the first grayscale average value and the second grayscale average value of the two sub-area images respectively. If the difference between the first grayscale average value and the second grayscale average value meets the preset conditions, the pixel value of the pixel point greater than the pixel threshold is adjusted to 1, and the pixel value of the pixel point less than or equal to the pixel threshold is adjusted to 0.

[0077] The filtering process is specifically to select any pixel point from the first region image, calculate the grayscale average value of all pixels in a window centered on the pixel point, and use the grayscale average value as the new grayscale value of the selected pixel point to obtain the second region image. The selected window size can be set according to actual needs. Assuming that the grayscale average value of all pixels in the window corresponding to pixel point A is X1, then the pixel value X corresponding to A is A Replaced with X1.

[0078] The corrosion process is specifically to place the center of the structural element on each pixel of the second region image, determine whether all the elements in the structural element are located in the foreground of the second region image, and if all the elements of the structural element are located in the foreground, retain the pixel value of the corresponding center point in the second region image, otherwise, replace the pixel value of the corresponding center point in the second region image with the background pixel value. Before performing the above operation, it is necessary to first determine the foreground pixel points and background pixel points of the second region image. The structural element can be in various forms such as squares, circles, lines, etc., which are used to slide on the second region image. When sliding to any pixel point, based on the above detection, as long as one element in the structural element is in the background, the pixel value of the pixel point is replaced with the background pixel value, and the color of the background pixel point is black.

[0079] Step S40: judging whether the inspector exists in the inspection area based on the area image.

[0080] It should be noted that, in actual situations, in order to avoid false detection, it is necessary to first identify the person after obtaining the regional image.

[0081] If a person is identified, the facial features and appearance features of the person are further obtained through the image of the area, and feature comparison is used to determine whether the detected person is the same inspector as the person corresponding to the acquired positioning information. Feature comparison requires that both facial features and appearance features match. If no person is identified, re-positioning is performed.

[0082] Step S50: If so, detecting the inspection behavior of the inspector based on the regional image.

[0083] In the specific implementation, in this embodiment, the inspection behavior detection model is used to identify the inspection behavior. The specific training process of the inspection behavior detection model is to use the feature extraction model to extract the first feature and the second feature of the sample area image respectively; input the first feature and the second feature into the attention module for training, wherein the first feature and the second feature are trained separately; in the first stage of training, the attention module and the classification module are frozen, and the conditional variational autoencoder is trained; in the first stage of training, the conditional variational autoencoder is frozen, and the frozen attention module and the classification module are trained; the classification score output by the classification module is higher than the threshold as the corresponding action category, and a corresponding time interval and action classification confidence are generated for each action category to complete the inspection behavior recognition training, wherein the time interval is the start time and end time of the action.

[0084] It should be noted that the first feature is the RGB feature, and the second feature is the optical flow feature. For example, the feature extraction part samples a T-frame video as one segment in the complete regional image, cuts this segment into an RGB frame, and converts the RGB frame into an optical flow frame, and then converts the optical flow frame into matrix data. The RGB and optical flow data are divided into several non-overlapping segments, each of which is 16 frames. These segments are input into the network pre-trained with the Kinetics dataset for feature extraction to obtain 1024-dimensional features for each segment. During the training of the inspection behavior detection model, RGB and optical flow features will be trained separately. The attention module is used to extract the attention of the feature frame. The action frame will get a higher attention score, and the relative background frame will have a lower attention score. The classification module uses the foreground and background features to train the network, and the output is the score of each classification corresponding to the feature. During the training of the discriminant attention model, the attention module and the classification module are optimized at the same time.

[0085] Further, after obtaining the inspection behavior detection result, obtain the detection result output by the inspection behavior detection model; determine the inspection behavior type of the inspector based on the detection result; determine the standard inspection behavior type according to the inspection area; determine whether the inspection behavior type of the inspector matches the standard inspection behavior type; if not, output an abnormal behavior alarm prompt.

[0086] It should be noted that different inspection areas have different functions, equipment types, safety risks and other factors, and the corresponding inspection behavior standards will also be different. In this embodiment, different standard inspection behavior types are set for different inspection areas to evaluate the inspection behavior. In this way, the actual on-site situation of the inspection work can be better combined.

[0087] This embodiment obtains the first positioning information and the second positioning information of the inspector respectively; determines the target positioning information according to the first positioning information and the second positioning information; determines the corresponding inspection area based on the target positioning information, and obtains the regional image of the inspection area through the monitoring device; determines whether the inspector exists in the inspection area based on the regional image; if so, detects the inspection behavior of the inspector based on the regional image. In the above manner, the inspection behavior of the inspector is detected in combination with dual positioning, which improves the real-time and accuracy of the inspection behavior detection.

[0088] Reference Figure 2 , Figure 2 This is a structural block diagram of the first embodiment of the dual-positioning inspection behavior detection device of the present invention.

[0089] like Figure 2 As shown, the dual-positioning inspection behavior detection device proposed in the embodiment of the present invention includes:

[0090] The acquisition module 10 is used to respectively acquire the first positioning information and the second positioning information of the inspector.

[0091] The fusion module 20 is used to determine target positioning information according to the first positioning information and the second positioning information.

[0092] The acquisition module 10 is used to determine the corresponding inspection area based on the target positioning information, and acquire the regional image of the inspection area through a monitoring device.

[0093] The judgment module 30 is used to judge whether the inspector exists in the inspection area based on the area image.

[0094] The detection module 40 is used to detect the inspection behavior of the inspector based on the regional image, if any.

[0095] This embodiment obtains the first positioning information and the second positioning information of the inspector respectively; determines the target positioning information according to the first positioning information and the second positioning information; determines the corresponding inspection area based on the target positioning information, and obtains the regional image of the inspection area through the monitoring device; determines whether the inspector exists in the inspection area based on the regional image; if so, detects the inspection behavior of the inspector based on the regional image. In the above manner, the inspection behavior of the inspector is detected in combination with dual positioning, which improves the real-time and accuracy of the inspection behavior detection.

[0096] In some embodiments, the fusion module 20 is used to determine a first reference area corresponding to the first positioning information; determine a second reference area corresponding to the second positioning information; determine a target area based on the first reference area and the second reference area; respectively obtain time series corresponding to the first positioning information and the second positioning information; based on the time series, fuse the first positioning information and the second positioning information to obtain fused positioning information; and obtain target positioning information based on the fused positioning information and the target area.

[0097] In some embodiments, the dual-positioned inspection behavior detection device further includes a processing module;

[0098] The processing module is used to obtain the original area image of the inspection area through the monitoring device, perform binarization processing on the original area image to obtain a first area image; perform filtering processing on the first area image to obtain a second area image; and perform corrosion processing on the second image to obtain an area image of the inspection area.

[0099] In some embodiments, the processing module is used to divide the original region image into two sub-region images less than or equal to the set pixel threshold and greater than the set pixel threshold according to a set pixel threshold, calculate a first grayscale average value and a second grayscale average value of the two sub-region images respectively, and if the difference between the first grayscale average value and the second grayscale average value meets a preset condition, binarize the original region image according to the set pixel threshold to obtain a first region image; select any pixel point from the first region image, calculate the grayscale average value of all pixels in a window centered on the pixel point, and use the grayscale average value as a new grayscale value of the selected pixel point to obtain a second region image; place the center of the structural element on each pixel of the second region image, determine whether all elements in the structural element are located in the foreground of the second region image, if all elements of the structural element are located in the foreground, retain the pixel value of the corresponding center point in the second region image, otherwise, replace the pixel value of the corresponding center point in the second region image with the background pixel value.

[0100] In some embodiments, the judgment module 30 is used to judge whether there is a person in the inspection area based on the area image; if a person is detected in the inspection area, the corresponding facial features and appearance features are obtained; the facial features and the appearance features are compared respectively; if the facial features and the appearance features are consistent, it is determined that the person in the inspection area is the same inspector.

[0101] In some embodiments, the detection module 40 is used to use a feature extraction model to respectively extract the first feature and the second feature of the sample area image; input the first feature and the second feature into the attention module for training, wherein the first feature and the second feature are trained separately; in the first stage of training, freeze the attention module and the classification module, and train the conditional variational autoencoder; in the first stage of training, freeze the conditional variational autoencoder, and train the frozen attention module and the classification module; take the classification score output by the classification module higher than the threshold as the corresponding action category, and generate a corresponding time interval and action classification confidence for each action category to complete the inspection behavior recognition training, wherein the time interval is the start time and end time of the action; use the inspection behavior detection model to identify the inspection behavior based on the area image, and the inspection behavior detection model is composed of the attention module, the classification module and the conditional variational autoencoder.

[0102] In some embodiments, the detection module 40 is used to obtain the detection results output by the inspection behavior detection model; determine the inspection behavior type of the inspector based on the detection results; determine the standard inspection behavior type according to the inspection area; determine whether the inspection behavior type of the inspector matches the standard inspection behavior type; if not, output an abnormal behavior alarm prompt.

[0103] An embodiment of the present application also provides a dual-positioning inspection behavior detection device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus, and the memory is used to store a dual-positioning inspection behavior detection program; the processor is used to implement the above-mentioned dual-positioning inspection behavior detection method when executing the program stored in the memory.

[0104] The communication bus mentioned in the above dual-position inspection behavior detection device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0105] The communication interface is used for communication between the above-mentioned dual-positioning inspection behavior detection device and other devices.

[0106] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0107] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components.

[0108] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0109] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0110] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0111] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0112] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present invention. In specific applications, technicians in this field can make settings as needed, and the present invention does not limit this.

[0113] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0114] In addition, for technical details that are not described in detail in this embodiment, reference can be made to the dual-positioning inspection behavior detection method provided in any embodiment of the present invention, and will not be repeated here.

[0115] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0116] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0117] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0118] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

[0119] It is understandable that the system provided by the embodiment of the present invention corresponds to the method provided by the embodiment of the present invention, and the explanation, examples and beneficial effects of the relevant contents can refer to the corresponding parts in the above method.

Claims

1. A patrol behavior detection method based on dual positioning, characterized in that: The inspection behavior detection method based on dual positioning includes: respectively obtain the first positioning information and the second positioning information of the inspector; Determine target positioning information according to the first positioning information and the second positioning information; Determine a corresponding inspection area based on the target positioning information, and obtain an area image of the inspection area through a monitoring device; Determining whether the inspector exists in the inspection area based on the area image; If so, the inspection behavior of the inspector is detected based on the area image.

2. The inspection behavior detection method based on dual positioning according to claim 1, characterized in that: The determining the target positioning information according to the first positioning information and the second positioning information includes: Determining a first reference area corresponding to the first positioning information; Determining a second reference area corresponding to the second positioning information; determining a target area based on the first reference area and the second reference area; Respectively obtaining time series corresponding to the first positioning information and the second positioning information; fusing the first positioning information and the second positioning information based on the time series to obtain fused positioning information; Target positioning information is obtained according to the fused positioning information and the target area.

3. The inspection behavior detection method based on dual positioning according to claim 1, characterized in that: The method further comprises: Acquire an original regional image of the inspection area through the monitoring device, and perform binarization processing on the original regional image to obtain a first regional image; Performing filtering processing on the first region image to obtain a second region image; The second image is subjected to corrosion processing to obtain a regional image of the inspection area.

4. The inspection behavior detection method based on dual positioning as claimed in claim 3, characterized in that: The method further comprises: Dividing the original region image into two sub-region images less than or equal to the set pixel threshold and greater than the set pixel threshold according to a set pixel threshold, respectively calculating a first grayscale average value and a second grayscale average value of the two sub-region images, and if a difference between the first grayscale average value and the second grayscale average value meets a preset condition, binarizing the original region image according to the set pixel threshold to obtain a first region image; Selecting any pixel point from the first regional image, calculating the grayscale average value of all pixel points in a window centered on the pixel point, and using the grayscale average value as a new grayscale value of the selected pixel point to obtain a second regional image; The center of the structural element is placed on each pixel of the second area image, and it is determined whether all elements in the structural element are located in the foreground of the second area image. If all elements of the structural element are located in the foreground, the pixel value of the corresponding center point in the second area image is retained; otherwise, the pixel value of the corresponding center point in the second area image is replaced with the background pixel value.

5. The inspection behavior detection method based on dual positioning according to claim 1, characterized in that: The determining whether the inspector exists in the inspection area based on the area image includes: Determine whether there is anyone in the inspection area based on the area image; If a person is detected in the inspection area, the corresponding facial features and appearance features are obtained; Comparing the facial features and the appearance features respectively; If the facial features and the appearance features are consistent, it is determined that the person in the inspection area is the same inspector.

6. The inspection behavior detection method based on dual positioning according to any one of claims 1 to 5, characterized in that: The detecting the inspection behavior of the inspector based on the regional image includes: Using a feature extraction model to extract a first feature and a second feature of the sample area image respectively; Inputting the first feature and the second feature into an attention module for training, wherein the first feature and the second feature are trained separately; In the first stage of training, the attention module and the classification module are frozen, and the conditional variational autoencoder is trained; In a first stage of training, the conditional variational autoencoder is frozen, and the frozen attention module and the classification module are trained; The classification scores output by the classification module that are higher than the threshold are taken as the corresponding action categories, and a corresponding time interval and action classification confidence are generated for each action category to complete the inspection behavior recognition training, wherein the time interval is the start time and end time of the action; The inspection behavior is identified based on the regional image using an inspection behavior detection model, wherein the inspection behavior detection model is composed of the attention module, the classification module and the conditional variational autoencoder.

7. The inspection behavior detection method based on dual positioning according to claim 6, characterized in that: The method further comprises: Obtaining the detection result output by the inspection behavior detection model; Determining the type of inspection behavior of the inspector based on the detection result; Determine the type of standard inspection behavior according to the inspection area; Determining whether the inspection behavior type of the inspector matches the standard inspection behavior type; If they do not match, an abnormal behavior alarm will be output.

8. A patrol behavior detection device based on dual positioning, characterized in that: The inspection behavior detection device based on dual positioning includes: An acquisition module, used to respectively acquire the first positioning information and the second positioning information of the inspector; A fusion module, used to determine target positioning information according to the first positioning information and the second positioning information; The acquisition module is used to determine the corresponding inspection area based on the target positioning information, and acquire the regional image of the inspection area through the monitoring device; A judgment module, used for judging whether the inspector exists in the inspection area based on the area image; A detection module is used to detect the inspection behavior of the inspector based on the area image, if any.

9. The inspection behavior detection method based on dual positioning as claimed in claim 8, characterized in that: The fusion module is used to determine a first reference area corresponding to the first positioning information; Determining a second reference area corresponding to the second positioning information; determining a target area based on the first reference area and the second reference area; Respectively obtaining time series corresponding to the first positioning information and the second positioning information; fusing the first positioning information and the second positioning information based on the time series to obtain fused positioning information; Target positioning information is obtained according to the fused positioning information and the target area.

10. A patrol behavior detection device based on dual positioning, characterized in that: The inspection behavior detection device based on dual positioning includes: a memory, a processor, and a inspection behavior detection program based on dual positioning stored on the memory and executable on the processor, wherein the inspection behavior detection program based on dual positioning is configured to implement the steps of the inspection behavior detection method based on dual positioning as described in any one of claims 1 to 7.

Images