A method and device for identifying abnormal intrusion behavior in a restricted area

Abstract

The present application discloses a method and device for identifying abnormal intrusion behavior in a prohibited area, which relates to the field of traffic technology. A video frame sequence to be identified is obtained; the N frames of images are identified, and a target object is identified from M frames of images in the N frames of images; if the target object appears in the prohibited area in at least one frame of the M frames of images, a sampling image sequence is obtained from the M frames of images, a cubic Bezier curve is constructed according to the position information in the sampling image sequence, and according to the cubic Bezier curve and the prohibited area, it is determined whether the target object has intruded into the prohibited area within the time period corresponding to the M frames of images.

Description

Technical Field

[0001] The present application relates to a technical field combining the field of image recognition technology and the field of traffic technology, and in particular to a method and device for identifying abnormal intrusion into a prohibited area. Background Art

[0002] In road traffic management, there are usually areas where pedestrians are prohibited from entering. For example, pedestrians are strictly prohibited from entering at the entrance of a highway; for example, pedestrians are strictly prohibited from entering viaducts on urban roads; for example, some major security activities have some areas that are prohibited from entering, and pedestrians are also strictly prohibited from entering. If pedestrians enter the prohibited area, it is easy to cause traffic accidents, traffic congestion, etc.

[0003] Therefore, how to identify the behavior of pedestrians entering restricted areas is a technical problem that urgently needs to be solved. Summary of the invention

[0004] The embodiments of the present application provide a method and device for identifying abnormal intrusion behavior in a restricted area, so as to identify whether a pedestrian has intruded into a restricted area.

[0005] In a first aspect, a method for identifying abnormal intrusion behavior in a restricted area is provided, comprising:

[0006] A video frame sequence to be identified is obtained, wherein the video frame sequence includes N frames of images, where N is an integer greater than 1; the N frames of images are identified based on a lightweight convolutional neural network, and a target object is identified from M frames of images in the N frames of images, where M is an integer greater than or equal to 1 and less than N; if the target object appears in the restricted area in at least one frame of the M frames of images, a sampling image sequence is obtained from the M frames of images, wherein the images in the sampling image sequence start from the image in which the target object first appears in the restricted area in the M frames of images, until the target object moves out of the image in the restricted area or until the time length corresponding to the sampling image sequence reaches a set time length, a cubic Bezier curve for characterizing the motion trajectory of the target object is constructed according to the position information of the target object in the sampling image sequence, and according to the cubic Bezier curve and the restricted area, whether the target object enters the restricted area within the time period corresponding to the M frames of images is determined.

[0007] Optionally, the input information of the lightweight neural network includes a first channel image, a second channel image, a third channel image and a fourth channel image corresponding to each frame of the N frames of images, the first channel image to the third channel image are images of each color channel of the three primary colors, respectively, the fourth channel image is a grayscale image, and the grayscale image is a proportional gradient image.

[0008] Optionally, the following method is used to determine whether the target object in the first image of the M frames of images appears in the prohibited area:

[0009] Obtain the number of intersection points of a ray that takes the center point of the area where the target object in the first image is located and passes through the no-entry area and the boundary of the no-entry area, where the first image is any frame of the M frames; if the number of intersection points is an odd number, it is determined that the target object in the first image appears in the no-entry area.

[0010] Optionally, determining whether the target object enters the prohibited area within a time period corresponding to the M frames of images according to the cubic Bezier curve and the prohibited area includes:

[0011] The length of the cubic Bezier curve is compared with the square root of the area of ​​the forbidden area; if the length of the cubic Bezier curve is greater than the square root of the area of ​​the forbidden area, and the time length of the target object in the forbidden area is greater than the set time length, it is determined that the target object has entered the forbidden area; if the length of the cubic Bezier curve is less than or equal to the square root of the area of ​​the forbidden area, and the target object moves out of the forbidden area within the set time length, it is determined that the target object has not entered the forbidden area.

[0012] Optionally, acquiring a sampling image sequence from the M frames of images includes:

[0013] The M frames of images are sampled according to a first image frame interval to obtain the sampled image sequence; the sampled image sequence is divided into K subsequences according to a second image frame interval; a cubic Bezier curve corresponding to each subsequence is constructed according to the images in each subsequence of the K subsequences; and a cubic Bezier curve corresponding to the sampled image sequence is obtained according to the cubic Bezier curve corresponding to the K subsequences.

[0014] In a second aspect, a mobile edge computing device is provided, including:

[0015] The acquisition module is configured to acquire a video frame sequence to be identified, wherein the video frame sequence includes N frames of images, where N is an integer greater than 1; the identification module is configured to identify the N frames of images based on a lightweight convolutional neural network, and identify the target object from M frames of images in the N frames of images, where M is an integer greater than or equal to 1 and less than N; the judgment module is configured to acquire a sampling image sequence from the M frames of images if the target object appears in the prohibited area in at least one frame of the M frames of images, wherein the images in the sampling image sequence start from the image in which the target object first appears in the prohibited area in the M frames of images, until the target object moves out of the image in the prohibited area or until the time length corresponding to the sampling image sequence reaches a set time length, construct a cubic Bezier curve for characterizing the motion trajectory of the target object according to the position information of the target object in the sampling image sequence, and determine whether the target object enters the prohibited area within the time period corresponding to the M frames of images according to the cubic Bezier curve and the prohibited area.

[0016] Optionally, the input information of the lightweight neural network includes a first channel image, a second channel image, a third channel image and a fourth channel image corresponding to each frame of the N frames of images, the first channel image to the third channel image are images of each color channel of the three primary colors, respectively, the fourth channel image is a grayscale image, and the grayscale image is a proportional gradient image.

[0017] Optionally, the judgment module is specifically configured to:

[0018] Obtain the number of intersection points of a ray that takes the center point of the area where the target object is located in the first image as the starting point and passes through the no-entry area and the boundary of the no-entry area, where the first image is any frame of the M frames; if the number of intersection points is an odd number, it is determined that the target object in the first image appears in the no-entry area.

[0019] Optionally, the judgment module is specifically configured to:

[0020] The length of the cubic Bezier curve is compared with the square root of the area of ​​the forbidden area; if the length of the cubic Bezier curve is greater than the square root of the area of ​​the forbidden area, and the time length of the target object in the forbidden area is greater than the set time length, it is determined that the target object has entered the forbidden area; if the length of the cubic Bezier curve is less than or equal to the square root of the area of ​​the forbidden area, and the target object moves out of the forbidden area within the set time length, it is determined that the target object has not entered the forbidden area.

[0021] Optionally, the judgment module is specifically configured to:

[0022] The M frames of images are sampled according to a first image frame interval to obtain the sampled image sequence; the sampled image sequence is divided into K subsequences according to a second image frame interval; a cubic Bezier curve corresponding to each subsequence is constructed according to the images in each subsequence of the K subsequences; and a cubic Bezier curve corresponding to the sampled image sequence is obtained according to the cubic Bezier curve corresponding to the K subsequences.

[0023] In the embodiment of the present application, N frames of images are recognized based on a lightweight convolutional neural network (such as a mobilenet network), so that the speed of image recognition can be improved; and because the target object is judged to appear in the prohibited area based on the position information of the target object in each frame of the M frames and the geographical scope of the prohibited area, a cubic Bezier curve for characterizing the motion trajectory of the target object and the prohibited area are constructed based on the position information of the target object in the sampled image sequence, and it is further determined whether the target object enters the prohibited area within the time period corresponding to the M frames, thereby reducing the judgment error and improving the accuracy of the judgment.

[0024] On the basis of conforming to the common sense in the art, the above-mentioned preferred conditions can be arbitrarily combined to obtain the preferred embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0026] Figure 1 A schematic diagram of the structure of the YOLO v4 algorithm applicable to the embodiments of the present application;

[0027] Figure 2 A schematic diagram of the structure of a lightweight convolutional neural network applicable to an embodiment of the present application;

[0028] Figure 3 A flow chart of a method for identifying abnormal intrusion behavior in a restricted area provided in an embodiment of the present application;

[0029] Figure 4 A schematic diagram of determining that a target object is in a prohibited area provided in an embodiment of the present application;

[0030] Figure 5 A schematic diagram of the structure of a mobile edge computing device provided in an embodiment of the present application;

[0031] Figure 6 A schematic diagram of the structure of a communication device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0033] In the following, the terms "first", "second" are used for descriptive purposes only and should not be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" is two or more.

[0034] In the prior art, Mobile Edge Computing (MEC) devices generally use the cspdarknet network in the target recognition algorithm (such as the yolo v4 algorithm, which is a convolutional neural network) as the backbone feature extraction network to detect target objects (such as pedestrians in images). The computational cost of using the cspdarknet network for feature extraction is high. Since the computing power of MEC devices is lower than that of servers (for example, the computing power of MEC devices is only 21TOPS, which is 1 / 6 of that of servers), the efficiency of MEC devices in identifying target objects is low.

[0035] Therefore, in order to improve the recognition efficiency of target objects (such as pedestrians in images) in traffic scenarios, a lightweight neural network (such as a mobilenet network) is used as the backbone feature extraction network in the yolo v4 algorithm in the embodiment of the present application to realize the recognition of target objects. Taking the mobilenet network as an example, the overall architecture of the yolo v4 algorithm is as follows: Figure 1 As shown in the figure, it mainly includes a mobilenet network and a binary cross entropy loss function. The mobilenet network is used to extract features from the image to be recognized, and the binary cross entropy loss function is used to classify the features extracted by the mobilenet network (for example, distinguish whether the extracted features are pedestrians or backgrounds).

[0036] Optionally, in the embodiment of the present application, while improving the image recognition speed based on a lightweight convolutional neural network (such as a mobilenet network), in order to enhance the feature extraction capability of the target object, an equal-proportional gradient image may be added to the input layer based on the channel-by-channel operation characteristics of the depth-separable convolution in the lightweight convolutional neural network to further enhance the feature extraction capability of the target object. For example, the input information of the lightweight neural network may include a first channel image, a second channel image, a third channel image, and a fourth channel image corresponding to each frame of the N frames of images, the first channel image to the third channel image are images of each color channel in the three primary colors, respectively, and the fourth channel image is a grayscale image (for example, an equal-proportional gradient image).

[0037] Figure 2 The convolutional layer of a mobilenet network and the input information of each layer provided in the embodiment of the present application. Figure 2 As shown in Figure 1, the network consists of 20 convolutional layers connected in sequence and 1 maximum classifier (Softmax). Figure 2 Each row in the table corresponds to a convolution layer. For example, the first layer is a standard convolution layer with a stride of S2, an input image size of 224*224*4, and 32 filters of 3*3*3 are used, where 224 represents the length and width of the image, and 4 is the number of channels of the convolution; the second layer is a depthwise convolution layer with a stride of S1, an input image size of 112*112*32, and 1 filter of 3*3*32 is used. And so on. The last two layers are the average pooling layer and the fully connected layer.

[0038] Optionally, in some embodiments, since pedestrians (target objects) may be blocked by large vehicles (such as buses, coaches, trucks, etc.) when walking on traffic roads, in order to reduce the possibility of missed detection, a binary cross entropy loss function can be added to YOLO v4 to further classify the features extracted by the lightweight convolutional neural network (for example, the distinction between pedestrians and background), thereby improving the accuracy of the detection results.

[0039] Taking the distinction between pedestrians and background as an example, in the case of binary classification, the final detection results of the pedestrian detection algorithm model (such as the binary classification cross entropy loss function algorithm) generally only have these two cases, and the probability of each category is p and 1-p. Among them, the formula of the binary classification cross entropy loss function is:

[0040]

[0041] Among them, yi is the label of sample i, the positive class (which can represent pedestrians) is 1, and the negative class (which can represent the background) is 0; pi represents the probability that sample i is predicted to be a positive class.

[0042] In an embodiment of the present application, based on a lightweight convolutional neural network (such as a mobilenet network), the information input to the lightweight convolutional neural network is increased, and feature extraction is performed on the input image, thereby improving the recognition efficiency of the target object. The extracted features are then passed through an algorithm of a binary cross entropy loss function to expand the distinction between pedestrians and backgrounds, thereby improving the recognition accuracy of the target object by limiting the output.

[0043] Figure 3 A flow chart of a method for identifying abnormal intrusion behavior in a prohibited area provided in an embodiment of the present application. The method can be executed by a mobile edge computing device. The device can be implemented by software, hardware, or a combination of software and hardware.

[0044] In the embodiment of the present application, before obtaining the video frame sequence to be identified, multiple pedestrian prohibited areas can be configured for each surveillance video based on the cameras configured on the traffic road. For example, in the viaduct scene, pedestrian prohibited areas are configured on both sides of the road. Among them, the prohibited area can be any polygonal area, and the embodiment of the present application does not limit it here.

[0045] like Figure 3 As shown, the process includes the following steps:

[0046] 301: Obtain a video frame sequence to be identified, wherein the video frame sequence includes N frames of images, where N is an integer greater than 1.

[0047] In this step, the Real Time Streaming Protocol (RTSP) video stream captured by the camera in real time can be obtained, and based on the hardware decoding chip (such as NVDEC) provided by the hardware device, a self-written hardware decoding library (such as nvhw) is called to decode the RTSP code stream data into a video frame sequence in the required format.

[0048] Optionally, the hardware device can be Jetson NX. Using the hardware decoding library on Jetson NX has better decoding performance than using CPU decoding.

[0049] 302: Recognize the N frames of images acquired above based on a lightweight convolutional neural network, and recognize the target object from M frames of images among the N frames of images, where M is an integer greater than or equal to 1 and less than N.

[0050] After recognizing a frame of an image based on a lightweight convolutional neural network (such as a mobilenet network), at least one candidate target object bounding box may be obtained, and each bounding box corresponds to a classification category and a confidence level of the corresponding category. According to the confidence level corresponding to each candidate target object bounding box, a candidate target object bounding box that meets the confidence level requirement is selected from them, thereby recognizing the target object.

[0051] 303: According to the position information of the target object in each of the M frames and the geographical range of the prohibited area, determine whether the target object appears in the prohibited area. If yes, proceed to step 304; if no, end this process. In some embodiments, it can be determined whether the target object appears in the prohibited area based on the position information of the prohibited area and the position information of the target object.

[0052] Optionally, taking the first image as any one of the M frames as an example, whether the target object appears in the prohibited area in the first image can be determined in the following manner: starting from the center point of the area where the target object is located in the first image, draw a ray that passes through the prohibited area, and determine the number of intersections between the ray and the boundary of the prohibited area; if the number of intersections is an odd number, it is determined that the target object in the first image appears in the prohibited area, otherwise, the target object is outside the prohibited area. Figure 4 As shown, a ray L is drawn from the center point C of the area where the target object is located to pass through the forbidden area, and then the number of intersection points (Q) between the ray L and the boundary of the forbidden area is counted to be 1, then it is determined that the target object is in the forbidden area.

[0053] Optionally, if the target object (such as a pedestrian) is not recognized from N frames of images, this process can be terminated.

[0054] 304: Obtain a sampling image sequence from the M frame images, construct a cubic Bezier curve according to the position information of the target object in the sampling image sequence, and determine whether the target object enters the prohibited area within the time period corresponding to the M frame images according to the cubic Bezier curve and the prohibited area. If so, proceed to 305; if not, end this process.

[0055] Among them, the cubic Bezier curve is used to characterize the motion trajectory of the target object.

[0056] In some embodiments, the images in the sampled image sequence may start from the image in which the target object first appears in the prohibited area in the M frames of images, until the image in which the target object moves out of the prohibited area.

[0057] In some other real-time examples, the images in the sampled image sequence may start from the image in which the target object first appears in the prohibited area in the M frames of images, until the duration corresponding to the sampled image sequence reaches a set duration.

[0058] Optionally, the sampled image sequence can be obtained in the following manner: sampling M frames of images according to the first image frame interval to obtain the sampled image sequence, then dividing the sampled image sequence into K subsequences according to the second image frame interval, respectively constructing the cubic Bezier curve corresponding to each subsequence according to the images in each subsequence of the K subsequences; and obtaining the cubic Bezier curve corresponding to the sampled image sequence according to the cubic Bezier curves corresponding to the K subsequences. For example, starting from the image (the first frame of image) in which the target object first appears in the prohibited area, sampling subsequent image frames every 5 frames to obtain a sampled image sequence (for example, 20 frames of image), and then dividing the sampled image sequence into 5 subsequences every 4 frames starting from the first frame of image, constructing the cubic Bezier curves corresponding to the 5 subsequences according to the position information of the target object in each frame of image in each subsequence of the 5 subsequences, and finally obtaining the cubic Bezier curve corresponding to the sampled image sequence according to the cubic Bezier curves corresponding to the 5 subsequences.

[0059] Optionally, the cubic Bezier curve construction formula is:

[0060] B(t)=(1-t) 3 P0 + 3(1-t) 2 tP1+3(1-t)t 2 P2+t 3 P3……………………(2)

[0061] Among them, P0 is the starting control point of the target object, p1, p2, and p3 are control points, and t is any value in the interval [0, 1].

[0062] Optionally, whether the target object has entered a prohibited area may be determined based on the following conditions:

[0063] The length of the cubic Bezier curve constructed above is compared with the square root of the area of ​​the forbidden area; if the length of the cubic Bezier curve is greater than the square root of the area of ​​the forbidden area, and the time length of the target object in the forbidden area is greater than the set time length, it is determined that the target object has entered the forbidden area.

[0064] In some real-time examples, if the length of the cubic Bezier curve is less than or equal to the square root of the area of ​​the restricted area, and the target object moves out of the restricted area within a set time, it is determined that the target object has not entered the restricted area.

[0065] 305: When it is determined that the target object has entered the prohibited area, the behavior of the target object entering the prohibited area is reported.

[0066] By reporting the behavior of the above-mentioned target objects entering the restricted area, timely query and control can be facilitated, thereby improving traffic safety.

[0067] In the embodiment of the present application, N frames of images are recognized based on a lightweight convolutional neural network (such as a mobilenet network), so that the speed of image recognition can be improved; and because the target object is judged to appear in the prohibited area based on the position information of the target object in each frame of the M frames and the geographical scope of the prohibited area, a cubic Bezier curve for characterizing the motion trajectory of the target object and the prohibited area are constructed based on the position information of the target object in the sampled image sequence, and it is further determined whether the target object enters the prohibited area within the time period corresponding to the M frames, thereby reducing the judgment error and improving the accuracy of the judgment.

[0068] Based on the same technical concept, an embodiment of the present application also provides a device, which can implement the method flow of identifying abnormal intrusion behavior in the prohibited area in the embodiment of the present application.

[0069] Figure 5 The present invention provides a schematic diagram of the structure of a mobile edge computing device. As shown in the figure, the device includes: an acquisition module 501, an identification module 502, and a judgment module 503.

[0070] The acquisition module 501 is configured to acquire a video frame sequence to be identified, where the video frame sequence includes N frames of images, where N is an integer greater than 1.

[0071] The recognition module 502 is configured to recognize the N frames of images based on a lightweight convolutional neural network, and obtain the target object from M frames of images among the N frames of images, where M is an integer greater than or equal to 1 and less than N.

[0072] The judgment module 503 is configured to obtain a sampling image sequence from the M frame images if the target object appears in the prohibited area in at least one frame of the M frame images, the images in the sampling image sequence starting from the image in which the target object first appears in the prohibited area in the M frame images, until the target object moves out of the prohibited area or until the time length corresponding to the sampling image sequence reaches a set time length, construct a cubic Bezier curve for characterizing the motion trajectory of the target object according to the position information of the target object in the sampling image sequence, and determine whether the target object enters the prohibited area within the time period corresponding to the M frame images according to the cubic Bezier curve and the prohibited area.

[0073] It should be noted here that the above-mentioned device provided in the embodiment of the present application can implement all the method steps in the above-mentioned method embodiment for identifying abnormal intrusion into a restricted area, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those of the method embodiment will not be described in detail here.

[0074] Figure 6 The structural diagram of the communication device provided in the embodiment of the present application is exemplified.

[0075] As shown in the figure, the device may include: a processor 601 , a memory 602 , and a bus interface 603 .

[0076] The processor 601 is responsible for managing the bus architecture and general processing, and the memory 602 can store data used by the processor 601 when performing operations.

[0077] The bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 601 and memory represented by memory 602. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The processor 601 is responsible for managing the bus architecture and general processing, and the memory 602 may store data used by the processor 601 when performing operations.

[0078] The process disclosed in the embodiment of the present application can be applied to the processor 601, or implemented by the processor 601. In the implementation process, each step of the signal processing process can be completed by the hardware integrated logic circuit or software instructions in the processor 601. The processor 601 can be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method applied in conjunction with the embodiment of the present application can be directly embodied as a hardware processor to be executed, or the hardware and software modules in the processor are combined and executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 602, and the processor 601 reads the information in the memory 602 and completes the steps of the information processing process in conjunction with its hardware.

[0079] Specifically, the processor 601 is used to read the computer instructions in the memory 602 and execute a method for identifying abnormal intrusion behavior in a prohibited area in an embodiment of the present application.

[0080] It should be noted here that the above-mentioned communication device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0081] An embodiment of the present application further provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a method for identifying abnormal intrusion behavior in a prohibited area in the above embodiment.

[0082] An embodiment of the present application further provides a computer program product. When the computer program product is called by a computer, the computer executes a method for identifying abnormal intrusion behavior in a prohibited area in the above embodiment.

[0083] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0084] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0085] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0086] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0087] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for identifying abnormal intrusion behavior in a restricted area, characterized in that: include: Acquire a video frame sequence to be identified, the video frame sequence includes N frames of images, where N is an integer greater than 1; identify the N frames of images based on a lightweight convolutional neural network, and identify a target object from M frames of images among the N frames of images, where M is an integer greater than or equal to 1 and less than N; If it is determined that the target object appears in the forbidden area according to the position information of the target object in each frame of the M frames and the geographical scope of the forbidden area, a sampling image sequence is obtained from the M frames, and the images in the sampling image sequence start from the image in which the target object first appears in the forbidden area in the M frames, until the image in which the target object moves out of the forbidden area or until the time length corresponding to the sampling image sequence reaches a set time length, a cubic Bezier curve for characterizing the motion trajectory of the target object is constructed according to the position information of the target object in the sampling image sequence, and the length of the cubic Bezier curve is compared with the square root of the area of ​​the forbidden area; if the length of the cubic Bezier curve is greater than the square root of the area of ​​the forbidden area, and the time length of the target object in the forbidden area is greater than the set time length, it is determined that the target object has intruded into the forbidden area; if the length of the cubic Bezier curve is less than or equal to the square root of the area of ​​the forbidden area, and the target object moves out of the forbidden area within the set time length, it is determined that the target object has not intruded into the forbidden area.

2. The method according to claim 1, characterized in that The input information of the lightweight neural network includes a first channel image, a second channel image, a third channel image and a fourth channel image corresponding to each frame of the N frames of images, the first channel image to the third channel image are respectively images of each color channel of the three primary colors, the fourth channel image is a grayscale image, and the grayscale image is a proportional gradient image.

3. The method according to claim 1, characterized in that The following method is used to determine whether the target object in the first image of the M frames of images appears in the prohibited area: Obtaining the number of intersection points between a ray that takes the center point of the region where the target object is located in the first image and passes through the restricted region and the boundary of the restricted region, wherein the first image is any one of the M frames of images; If the number of the intersection points is an odd number, it is determined that the target object in the first image appears in the prohibited area.

4. The method according to any one of claims 1 to 3, characterized in that: The acquiring a sampling image sequence from the M frames of images comprises: Sampling the M frames of images according to a first image frame interval to obtain the sampled image sequence; Dividing the sampled image sequence into K subsequences according to a second image frame interval; According to the images in each subsequence of the K subsequences, a cubic Bezier curve corresponding to each subsequence is constructed; According to the cubic Bezier curves corresponding to the K subsequences, a cubic Bezier curve corresponding to the sampled image sequence is obtained.

5. A mobile edge computing device, characterized in that: include: An acquisition module is configured to acquire a video frame sequence to be identified, wherein the video frame sequence includes N frames of images, where N is an integer greater than 1; A recognition module is configured to recognize the N frames of images based on a lightweight convolutional neural network, and obtain a target object from M frames of images among the N frames of images, where M is an integer greater than or equal to 1 and less than N; The judgment module is configured to, if it is determined that the target object appears in the prohibited area according to the position information of the target object in each frame of the M frames of images and the geographical scope of the prohibited area, obtain a sampling image sequence from the M frames of images, the images in the sampling image sequence starting from the image in which the target object first appears in the prohibited area in the M frames of images, until the image in which the target object moves out of the prohibited area or until the time corresponding to the sampling image sequence reaches a set time, construct a cubic Bezier curve for representing the motion trajectory of the target object according to the position information of the target object in the sampling image sequence, and compare the length of the cubic Bezier curve with the square root of the area of ​​the prohibited area; if the length of the cubic Bezier curve is greater than the square root of the area of ​​the prohibited area, and the time length of the target object in the prohibited area is greater than the set time length, determine that the target object has entered the prohibited area; if the length of the cubic Bezier curve is less than or equal to the square root of the area of ​​the prohibited area, and the target object moves out of the prohibited area within the set time length, determine that the target object has not entered the prohibited area.

6. The device according to claim 5, characterized in that The input information of the lightweight neural network includes a first channel image, a second channel image, a third channel image and a fourth channel image corresponding to each frame of the N frames of images, the first channel image to the third channel image are respectively images of each color channel of the three primary colors, the fourth channel image is a grayscale image, and the grayscale image is a proportional gradient image.

7. The device according to claim 5, characterized in that The determination module is specifically configured as follows: Obtaining the number of intersection points between a ray starting from the center point of the region where the target object is located in the first image and passing through the restricted region and the boundary of the restricted region, wherein the first image is any one of the M frames of images; If the number of the intersection points is an odd number, it is determined that the target object in the first image appears in the prohibited area.

8. The device according to any one of claims 5 to 7, characterized in that: The determination module is specifically configured as follows: Sampling the M frames of images according to a first image frame interval to obtain the sampled image sequence; Dividing the sampled image sequence into K subsequences according to a second image frame interval; According to the images in each subsequence of the K subsequences, a cubic Bezier curve corresponding to each subsequence is constructed; According to the cubic Bezier curves corresponding to the K subsequences, a cubic Bezier curve corresponding to the sampled image sequence is obtained.

Images