Method, device and equipment for identifying stray target and medium
By performing object recognition and movement judgment on the surveillance video frame images, combining the depth separation convolution layer and the selective nuclear convolution layer, and combining the human posture classification model, the accuracy and efficiency of homeless personnel recognition are solved, and the city is achieved Accurate and timely identification of homeless people in the streets.
Patent Information
- Application Number
- CN202410028334.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-08
- Publication Date
- 2024-11-08
AI Technical Summary
It is difficult to accurately and timely identify homeless people in urban streets in the existing technology, especially because the homeless people are mobility and the drone shooting height does not meet the identification requirements, so existing personnel patrol time and laborious.
By extracting each frame of the surveillance video, inputting it to the pre-trained target recognition model, obtaining the position information and type of the target to be identified, and determining whether the target to be identified is moving. If it is still and within the preset distance, it is determined to be a wandering Target. Feature extraction is performed using depth-separable convolutional layer and selective nuclear convolutional layer, combined with human posture classification model to improve recognition accuracy.
It realizes accurate and timely identification of homeless people in urban streets, improves identification accuracy and efficiency, and reduces the time and cost of manpower inspections.
Smart Images

Figure CN118918525A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of intelligent analysis technology, and in particular to a method, device, equipment and medium for identifying a stray target. Background Art
[0002] Some people wander the streets or even beg for money due to family reasons, economic reasons, personal illness, etc., which not only makes their daily lives unprotected, but also has a certain impact on the city's appearance. Due to the high mobility of homeless people, the existing personnel inspection scheme is time-consuming and laborious. The drone scheme usually does not meet the requirements of homeless identification due to the high shooting height, and the shooting angle is limited in monitoring time.
[0003] Therefore, how to accurately and timely identify homeless people on city streets is a technical problem that needs to be solved urgently. Summary of the invention
[0004] The present application provides a method, device, equipment and medium for accurately and timely identifying homeless persons on city streets.
[0005] In a first aspect, the present application provides a method for identifying a stray target, the method comprising:
[0006] Extract each frame of the surveillance video; input each frame of the image into a pre-trained target recognition model in turn, and obtain the location information of the target to be identified and the type of the target to be identified contained in each frame of the image output by the target detection model;
[0007] If the type of the target to be identified is a preset type, determining whether the target to be identified of the preset type moves according to the position information of the target to be identified contained in each frame of image;
[0008] If not, it is determined that there is a wandering target in the area where the surveillance video is located.
[0009] In a possible implementation manner, if the preset type of target to be identified moves, the method further includes:
[0010] Determine the moving distance of the preset type of target to be identified according to the position information of the preset type of target to be identified contained in each frame of image;
[0011] If the distance is less than the first preset distance, it is determined that there is a wandering target in the area where the surveillance video is located.
[0012] In a possible implementation manner, if the preset type is a human body, before determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image, the method further includes:
[0013] Inputting the image of the preset type of target to be identified into a pre-trained human posture classification model, and obtaining the target posture of the human body output by the human posture classification model;
[0014] If the target posture is not standing, the subsequent step of determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image is performed.
[0015] In a possible implementation manner, if the preset types are begging signs and human bodies, before determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image, the method further includes:
[0016] Determine the distance between the target to be identified whose preset type is a human body and the target to be identified whose preset type is a begging sign;
[0017] If the distance is less than the second preset distance, the subsequent step of determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image is performed;
[0018] Wherein, determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image includes:
[0019] According to the position information of the target to be identified contained in each frame of image, it is determined whether the distance between the target to be identified of the preset type being a human body and the target to be identified of the preset type being a begging sign has changed.
[0020] In a possible implementation, inputting each frame of the image into a pre-trained target recognition model in sequence, and obtaining the position information of the target to be recognized contained in each frame of the image output by the target detection model and the type of the target to be recognized includes:
[0021] The target recognition model uses a depthwise separable convolutional layer to perform convolution on the input image, and uses a common convolutional layer to adjust the number of channels so that the feature map output by the depthwise separable convolutional layer is equal to the number of channels of the input image;
[0022] Convolving the feature map using a selective kernel convolution layer; wherein the selective kernel convolution layer includes a conventional convolution layer and a hole convolution layer; obtaining a first feature matrix according to the conventional convolution layer, and obtaining a second feature matrix according to the hole convolution layer;
[0023] Obtaining an output feature matrix corresponding to the input image according to the first feature matrix and the second feature matrix and their corresponding weights;
[0024] According to the output feature matrix, the position information of the target to be identified contained in the image and the type of the target to be identified are determined and output.
[0025] In a second aspect, an embodiment of the present application provides a wandering target identification device, the device comprising:
[0026] An extraction module is used to extract each frame of the surveillance video;
[0027] A target recognition module is used to input each frame of the image into a pre-trained target recognition model in sequence, and obtain the position information of the target to be recognized contained in each frame of the image output by the target detection model and the type of the target to be recognized;
[0028] A target tracking module, for determining whether the target of the preset type moves according to the position information of the target contained in each frame of image if the type of the target to be identified is a preset type;
[0029] The judgment module is used to determine that there is a wandering target in the area where the surveillance video is located if the preset type of target to be identified has not moved.
[0030] In a possible implementation, the target tracking module is further configured to determine, if the preset type of target to be identified moves, a distance moved by the preset type of target to be identified according to position information of the preset type of target to be identified contained in each frame of image;
[0031] The judgment module is further configured to determine that there is a wandering target in the area where the surveillance video is located if the distance is less than a first preset distance.
[0032] In a possible implementation, the target recognition module is specifically configured to, if the preset type is a human body, input an image of a target to be recognized of the preset type into a pre-trained human posture classification model, and obtain a target posture of the human body output by the human posture classification model;
[0033] The target tracking module is specifically used to determine whether the target to be identified of the preset type moves according to the position information of the target to be identified contained in each frame image if the target posture is not standing.
[0034] In a possible implementation, the target tracking module is specifically used to determine the distance between the target to be identified whose preset type is a human body and the target to be identified whose preset type is a begging sign if the preset types are a begging sign and a human body; if the distance is less than a second preset distance, then determine whether the distance between the target to be identified whose preset type is a human body and the target to be identified whose preset type is a begging sign has changed based on the position information of the target to be identified contained in each frame image.
[0035] In a possible implementation, the target recognition module is specifically used to enable the target recognition model to perform convolution on the input image using a depthwise separable convolution layer, and to adjust the number of channels using a common convolution layer so that the feature map output by the depthwise separable convolution layer is equal to the number of channels of the input image; to perform convolution on the feature map using a selective kernel convolution layer; wherein the selective kernel convolution layer includes a conventional convolution layer and a hole convolution layer; to obtain a first feature matrix according to the conventional convolution layer, and to obtain a second feature matrix according to the hole convolution layer; to obtain an output feature matrix corresponding to the input image according to the first feature matrix and the second feature matrix and their respective corresponding weights; and to determine and output the position information of the target to be identified contained in the image and the type of the target to be identified according to the output feature matrix.
[0036] In a third aspect, an embodiment of the present application provides an electronic device, which includes at least a processor and a memory, and the processor is used to implement the steps of any of the above methods when executing a computer program stored in the memory.
[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program executable by an electronic device. When the program runs on the electronic device, the electronic device executes the steps of any of the methods described above.
[0038] In an embodiment of the present application, each frame of the surveillance video is extracted, and each frame of the image is sequentially input into a pre-trained target recognition model to obtain the position information and type of the target to be identified contained in each frame of the image output by the target detection model, so as to accurately determine whether there is a stray target; since stray targets and their related items generally stay in a certain place for a short period of time, if there is a target to be identified of a preset type related to the stray target, then based on the position information of the target to be identified contained in each frame of the image, it is determined whether the target to be identified of the preset type has moved; if not, it is determined that there is a stray target in the area where the surveillance video is located, so as to accurately and timely determine whether there are homeless people in the city streets. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the implementation methods in the embodiments of the present application or the related technologies, the following is a brief introduction to the drawings required for use in the embodiments or the related technology descriptions. Obviously, the drawings described below are some embodiments of the present application, and a person skilled in the art can also obtain other drawings based on these drawings.
[0040] Figure 1 A schematic diagram of a wandering target identification process provided for some embodiments of the present application;
[0041] Figure 2 A schematic diagram of rescuing a stray target provided for some embodiments of the present application;
[0042] Figure 3 A schematic diagram of the structure of a backbone feature extraction network provided for some embodiments of the present application;
[0043] Figure 4 A schematic diagram of a structure of a selective kernel convolution provided for some embodiments of the present application;
[0044] Figure 5 Another schematic diagram of a wandering target identification process provided for some embodiments of the present application;
[0045] Figure 6 A schematic diagram of the structure of a wandering target identification device provided for some embodiments of the present application;
[0046] Figure 7 A schematic diagram of the structure of an electronic device provided for some embodiments of the present application. DETAILED DESCRIPTION
[0047] In order to make the purpose, technical solutions and advantages of this application clearer, this application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the embodiments described in this application are only part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0048] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.
[0049] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.
[0050] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.
[0051] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and / or software code that is capable of performing the functions associated with that element.
[0052] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
[0053] Some people wander the streets or even beg for money due to family reasons, economic reasons, personal illness, etc., which not only makes their daily lives unprotected, but also has a certain impact on the city's appearance. Due to the high mobility of homeless people, the existing personnel inspection scheme is time-consuming and laborious. The drone scheme usually does not meet the requirements of homeless identification due to the high shooting height, and the shooting angle is limited in monitoring time.
[0054] Based on this, in order to accurately and timely identify homeless people on urban streets, the present application provides a method, device, equipment and medium for identifying homeless targets. In the method, each frame of the surveillance video is extracted; each frame of the image is sequentially input into a pre-trained target recognition model to obtain the location information of the target to be identified and the type of the target to be identified contained in each frame of the image output by the target detection model; if the type of the target to be identified is a preset type, then according to the location information of the target to be identified contained in each frame of the image, it is determined whether the target to be identified of the preset type has moved; if not, it is determined that there is a homeless target in the area where the surveillance video is located.
[0055] Embodiment 1:
[0056] Figure 1A schematic diagram of a stray target identification process provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the process includes the following steps:
[0057] S101: extract each frame of the surveillance video.
[0058] The wandering target identification method provided in the embodiment of the present application is applied to an electronic device, which may be a smart camera, a PC, a server, or other device.
[0059] In order to accurately and timely identify homeless people, i.e., homeless targets, in the embodiments of the present application, smart cameras can be set up on urban roads and streets to collect surveillance videos, or surveillance videos collected by existing smart cameras on urban sidewalks can be obtained, and the present application does not limit this. In order to accurately identify homeless targets, each frame of the surveillance video can be extracted and analyzed in real time.
[0060] S102: Input each frame of the image into a pre-trained target recognition model in turn, and obtain the position information of the target to be recognized and the type of the target to be recognized contained in each frame of the image output by the target detection model.
[0061] In order to accurately determine whether there are stray targets in the surveillance video, a target detection model can be pre-trained, wherein the target detection model can be an existing target feature extraction network, such as a YOLOv8s network. Each frame of the image can be sequentially input into the pre-trained target recognition model to obtain the location information and type of the target to be identified contained in each frame of the image output by the target recognition model.
[0062] Specifically, the target detection model can identify the targets to be identified contained in the image in real time, and output the location information (Rect) of each target to be identified and the type (TypeOD) with the highest confidence (CONFod), where the type of the target to be identified can be the type of items related to stray targets, including but not limited to human bodies, begging signs, and bedding such as quilts and mats.
[0063] S103: If the type of the target to be identified is a preset type, determining whether the target to be identified of the preset type moves according to the position information of the target to be identified contained in each frame of image.
[0064] Since the embodiment of the present application is mainly used to identify stray targets, and the notable features of stray targets are sitting or lying on the ground, usually remaining still or moving slightly, in order to accurately determine whether there are stray targets in the surveillance video, the types of items with a high degree of correlation with stray targets can be saved as preset types. If there is a target to be identified of a preset type, the target to be identified of the preset type can be tracked, and based on the position information of the target to be identified contained in each frame image, it is determined whether the target to be identified of the preset type has moved.
[0065] Specifically, for example, if the preset type is bedding such as quilts and mats, if there is a target to be identified in the image and the type is bedding, it can be determined whether the target to be identified of the preset type, which is bedding, in the surveillance video has moved based on the position information of the target to be identified contained in each frame of the image.
[0066] S104: If not, it is determined that there is a wandering target in the area where the surveillance video is located.
[0067] If the preset type of target to be identified moves, it means that the target to be identified may be a passerby or an item related to a passerby, that is, there is probably no stray target in the surveillance video, and the target to be identified can be no longer processed. If the preset type of target to be identified does not move, it indicates that the target to be identified is likely to be a stray target, that is, a stray person or an item related to a stray person, and it can be determined that there is a stray target in the area where the surveillance video is located. An early warning can also be issued to prompt relevant personnel to provide timely rescue. Figure 2 A schematic diagram of rescuing a stray target is provided for some embodiments of the present application. Figure 2 As shown, it is determined that there is a stray target in the area where the surveillance video is located, and relevant personnel can be prompted to provide timely assistance to the stray person.
[0068] Specifically, in an embodiment of the present application, the frame rate of the smart camera can be appropriately lowered because the focus is on relatively static or slow-moving targets such as homeless people, begging signs, mats, and bedding. The idea of multi-target tracking can be adopted, and the location information of the preset type of target to be identified contained in the current frame image and the historical location information of the target to be identified contained in the previous frame image can be matched through the positioning area matching algorithm. For example, the common area ratio (IOU) of the target frame of each preset type of target to be identified can be compared. When the IOU is greater than the preset threshold, it is matched as the same target to be identified. In particular, when the location information of multiple preset types of targets to be identified is matched at the same time, the target to be identified with the largest IOU is selected as the only matched target to be identified. According to the location information of the same target to be identified in different frame images, the historical location trajectory of the target to be identified is determined and updated in real time.
[0069] In a possible implementation manner, four status flags may be set for preset types of to-be-identified targets: a new flag (Create), an update flag (Update), an overwrite flag (Cover), and a delete flag (Delete). If the preset type of target to be identified in the current frame image is not successfully matched with the preset type of target to be identified in the previous frame, it means that the target to be identified appears for the first time, and its status flag in the current frame image can be recorded as Create, and identification information such as an ID number can be set for the target to be identified and saved; if the preset type of target to be identified in the current frame image matches the position information of the preset type of target to be identified in the previous frame image, the position information of the preset type of target to be identified is updated, and its status flag in the current frame image is recorded as Update; if a preset type of target to be identified in the previous frame image has no matching target to be identified in the current frame, it means that the preset type of target to be identified is lost in the current frame image, and its status flag at the corresponding position in the current frame image can be recorded as Cover; when the number of frames of the target to be identified corresponding to the position where the status flag in the current frame image is recorded as Cover exceeds a preset number of frames, for example, 8 frames, it means that the target to be identified may have left the area where the surveillance video is located, and the status flag of the target to be identified in the current frame can be recorded as Updated to Delete, and the historical position track of the target to be identified is deleted. The historical position trajectory and ID of the target to be identified with the status flag marked as Update in the current frame image can be transmitted to the alarm logic judgment module to perform stray target identification and judgment, and finally output the result of whether there is a stray target in the area where the monitoring video is located.
[0070] In an embodiment of the present application, each frame of the surveillance video is extracted, and each frame of the image is sequentially input into a pre-trained target recognition model to obtain the position information and type of the target to be identified contained in each frame of the image output by the target detection model, so as to accurately determine whether there is a stray target; since stray targets and their related items generally stay in a certain place for a short period of time, if there is a target to be identified of a preset type related to the stray target, then based on the position information of the target to be identified contained in each frame of the image, it is determined whether the target to be identified of the preset type has moved; if not, it is determined that there is a stray target in the area where the surveillance video is located, so as to accurately and timely determine whether there are homeless people in the city streets.
[0071] Embodiment 2:
[0072] In order to further improve the accuracy of stray target recognition, based on the above embodiment, in the embodiment of the present application, if the preset type of target to be recognized moves, the method further includes:
[0073] Determine the moving distance of the preset type of target to be identified according to the position information of the preset type of target to be identified contained in each frame of image;
[0074] If the distance is less than the first preset distance, it is determined that there is a wandering target in the area where the surveillance video is located.
[0075] Since homeless persons and their related items generally stay in one place or move slightly, in order to further improve the accuracy of stray target recognition, a smaller distance, namely the first preset distance, can be preset. For example, the first preset distance can be 1 meter. According to the position information of the preset type of target to be identified contained in each frame image, the distance moved by the preset type of target to be identified is determined. If the distance is less than the first preset distance, it means that the target to be identified has moved slightly, that is, the target to be identified is likely not a passerby or a passerby's item, and it can be determined that there is a stray target in the area where the surveillance video is located.
[0076] In the embodiment of the present application, if the preset type of target to be identified moves, the distance moved by the preset type of target to be identified can be determined based on the position information of the preset type of target to be identified contained in each frame of image. If the distance is less than the first preset distance, it can also be determined that there is a wandering target in the area where the surveillance video is located.
[0077] Embodiment 3:
[0078] In order to further improve the accuracy of stray target recognition, based on the above embodiments, in the embodiment of the present application, if the preset type is a human body, before determining whether the preset type of target to be recognized has moved according to the position information of the target to be recognized contained in each frame image, the method further includes:
[0079] Inputting the image of the preset type of target to be identified into a pre-trained human posture classification model, and obtaining the target posture of the human body output by the human posture classification model;
[0080] If the target posture is not standing, the subsequent step of determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image is performed.
[0081] Since homeless people will maintain a non-standing posture most of the time, such as lying on the ground, sitting on the ground or kneeling on the ground. In order to further improve the accuracy of stray target recognition, a human posture classification model can be pre-trained, and the pre-trained human posture classification model can be used to determine the posture of the human body contained in the image. Among them, the training process of the human posture classification model can refer to the existing technology, and will not be repeated here. If the preset type is human body, that is, the image contains a target to be identified of the preset type of human body, before determining whether the target to be identified of the preset type of human body moves according to the position information of the target to be identified contained in each frame of the image, the image where the target to be identified of the preset type of human body is located can be input into the pre-trained human posture classification model to obtain the target posture of the human body output by the human posture classification model. If the target posture is not standing, for example, the target posture is sitting, kneeling or lying, etc., it means that the target to be identified of the preset type of human body is likely to be a homeless person, and the subsequent step of determining whether the target to be identified of the preset type of human body moves according to the position information of the target to be identified contained in each frame of the image can be performed.
[0082] In an embodiment of the present application, if the preset type is human body, the image of the target to be identified whose preset type is human body can be input into a pre-trained human posture classification model to determine the target posture of the target to be identified whose preset type is human body; if the target posture is not standing, it means that the target to be identified whose preset type is human body is likely to be a wandering target, and further judgment can be performed, that is, a subsequent step of determining whether the target to be identified of the preset type has moved according to the position information of the target to be identified contained in each frame image is performed, thereby further improving the accuracy of wandering target identification.
[0083] Embodiment 4:
[0084] In order to further improve the accuracy of stray target recognition, based on the above embodiments, in the embodiment of the present application, if the preset type is a begging sign and a human body, before determining whether the preset type of target to be recognized has moved according to the position information of the target to be recognized contained in each frame image, the method further includes:
[0085] Determine the distance between the target to be identified whose preset type is a human body and the target to be identified whose preset type is a begging sign;
[0086] If the distance is less than the second preset distance, the subsequent step of determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image is performed;
[0087] Wherein, determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image includes:
[0088] According to the position information of the target to be identified contained in each frame of image, it is determined whether the distance between the target to be identified of the preset type being a human body and the target to be identified of the preset type being a begging sign has changed.
[0089] Since the vagrant is likely to carry a begging sign, the begging sign and the human body can also be saved as preset types. Because the distance between the vagrant and the begging sign is very short, a very short distance can also be set, i.e., the second preset distance. For example, the second preset distance is 0.5 meters. If the preset types are the begging sign and the human body, before determining whether the target to be identified of the preset type moves according to the position information of the target to be identified contained in each frame image, the distance between the target to be identified of the preset type being the human body and the target to be identified of the preset type being the begging sign can be determined. If the distance is less than the second preset distance, the target to be identified of the preset type indicates that it is likely to be a vagrant carrying a begging sign. Then a subsequent step of determining whether the target to be identified of the preset type moves according to the position information of the target to be identified contained in each frame image can be performed.
[0090] Since homeless people may carry begging signs while begging on the road, it is necessary to determine whether the target to be identified with the preset type of begging sign and the target to be identified with the preset type of human body move, that is, to determine whether there is relative movement between the target to be identified with the preset type of human body and the target to be identified with the preset type of begging sign, that is, to determine whether the distance between the target to be identified with the preset type of human body and the target to be identified with the preset type of begging sign changes.
[0091] Specifically, it is possible to determine whether the distance between the target to be identified of the preset type of human body and the target to be identified of the preset type of begging sign has changed based on the position information of the target to be identified contained in each frame of image. If the distance changes, it means that the target to be identified of the preset type of human body has moved, while the target to be identified of the preset type of begging sign has not moved, that is, the two targets to be identified may not be homeless people carrying begging signs, and no subsequent processing is required. If the distance does not change, it means that the target to be identified of the preset type of begging sign and the target to be identified of the preset type of human body have moved or stopped together, that is, the two targets to be identified are likely to be homeless people carrying begging signs, and it can be determined that there are homeless targets in the area where the surveillance video is located. An early warning can also be issued to prompt relevant personnel to provide timely assistance.
[0092] In the embodiment of the present application, if the preset types are a begging sign and a human body, the distance between the target to be identified of the preset type of the begging sign and the preset type of the human body can be determined. If the distance is less than the second preset distance, the two preset types of targets to be identified are likely to be homeless persons carrying begging signs. Further judgment can be performed, that is, a subsequent step of determining whether the two preset types of targets to be identified have moved relative to each other based on the position information of the target to be identified contained in each frame image is performed, so as to further determine whether the two preset types of targets to be identified are homeless persons carrying begging signs, thereby further improving the accuracy of identifying homeless targets.
[0093] Since homeless persons may also move, based on the above embodiments, in a possible implementation, in order to further improve the accuracy of homeless target recognition, a preset time length may be saved in advance, for example, the preset time length is 10 minutes. If there is a target to be recognized of a preset type, it may be determined whether the target to be recognized of the preset type moves within the preset time length according to the position information of the target to be recognized contained in each frame of image.
[0094] In another possible implementation, the ratio of the number of frames of the image where the target to be identified is located in the surveillance video to the total number of frames of the surveillance video may be counted. If the ratio is greater than a preset threshold, such as 50%, it may be determined that there is a stray target in the surveillance video.
[0095] Specifically, for example, if the preset type is bedding, and the target to be identified of the preset type of bedding does not move within the preset time, then the target to be identified can be determined to be the bedding of a vagrant. For example, if the preset type is human body, and the target posture of the target to be identified of the preset type of human body is not standing, and does not move within the preset time, then the target to be identified can be determined to be a vagrant. For another example, if the preset type is a begging sign and a human body, and the distance between the preset type of begging sign and the target to be identified of the preset type of human body is less than the second preset distance, and does not move within the preset time, then the target to be identified can be determined to be a vagrant carrying a begging sign.
[0096] Embodiment 5:
[0097] In order to further improve the accuracy of stray target recognition, based on the above embodiments, in the embodiment of the present application, the step of sequentially inputting each frame of the image into a pre-trained target recognition model, and obtaining the position information of the target to be recognized contained in each frame of the image output by the target detection model and the type of the target to be recognized includes:
[0098] The target recognition model uses a depthwise separable convolutional layer to perform convolution on the input image, and uses a common convolutional layer to adjust the number of channels so that the feature map output by the depthwise separable convolutional layer is equal to the number of channels of the input image;
[0099] Convolving the feature map using a selective kernel convolution layer; wherein the selective kernel convolution layer includes a conventional convolution layer and a hole convolution layer; obtaining a first feature matrix according to the conventional convolution layer, and obtaining a second feature matrix according to the hole convolution layer;
[0100] Obtaining an output feature matrix corresponding to the input image according to the first feature matrix and the second feature matrix and their corresponding weights;
[0101] According to the output feature matrix, the position information of the target to be identified contained in the image and the type of the target to be identified are determined and output.
[0102] Since urban homeless people usually appear at the corners of the streets, they are farther from the camera than pedestrians on the streets, and they usually sit or lie down when they are resting, so they are usually only one-fourth or smaller than the normal pedestrian size in the surveillance video. In addition, since the quality of life of homeless people is usually not guaranteed, their faces and clothes in the surveillance video are relatively blurred. In order to further improve the accuracy of homeless target recognition, a target recognition model that can detect blurred small targets can be pre-trained. Among them, the target recognition model can improve the existing YOLOv8s network to adapt to the detection of blurred small targets.
[0103] First, in order to meet the real-time requirements, the existing YOLOv8s network can be lightweight improved. The backbone feature extraction network C2fDarknet in the existing YOLOv8s network includes an inverted residual structure, in which the input image needs to be dimensionally upgraded so that the number of channels of the output feature map is higher than the number of channels of the input image, thereby extracting more features, but it will increase the time consumption.
[0104] In order to meet the real-time requirements, in an embodiment of the present application, based on the principle of minimizing memory usage when the channels of the convolutional layer input feature matrix and the output feature matrix are equal, a depthwise separable convolutional layer can be used to perform convolution on the input image, and an ordinary convolutional layer can be used to adjust the number of channels so that the feature map output by the depthwise separable convolutional layer is equal to the number of channels of the input image, without the need for complex dimensionality increase operations, thereby further reducing time consumption.
[0105] Specifically, the original backbone feature extraction network (C2fDarknet) is improved, that is, in the process of dimensionality increase in the inverted residual structure of C2fDarknet, a 3x3 depthwise separable convolution layer (DSC) is used instead to perform preliminary feature extraction without changing the number of channels. A 1x1 ordinary convolution layer can be set before and after the depthwise separable convolution layer to adjust the number of channels, that is, a 1x1 ordinary convolution layer is set immediately after each C2fDarknet of YOLOv8s to adjust the number of channels, so that the number of channels remains consistent and the time consumption is further reduced. Figure 3 A schematic diagram of the structure of a backbone feature extraction network provided in some embodiments of the present application. Figure 3 As shown, according to the input image, a 1x1 ordinary convolution layer Conv(1,1) can be used to adjust the number of channels, a depth-separable convolution layer DWConv(3,1) can be used for convolution, and then a 1x1 ordinary convolution layer Conv(1,1) can be used to adjust the number of channels to obtain a preliminary feature vector. At the same time, the features extracted last time, that is, the historical feature vector saved in the ordinary convolution layer Conv(1,1), can be inherited, and the preliminary feature vector and the historical feature vector are fused, that is, concatenated (Concat), and then a 1x1 ordinary convolution layer Conv(1,1) is used to adjust the number of channels to obtain the output feature map. Under the condition of basically maintaining the detection effect, an additional Conv(1,1) convolution is added to align the number of channels, and the network structure of the target recognition model is optimized to reduce the network forward calculation time, further reduce the model time consumption and resource occupation, and facilitate its deployment in the front end of electronic equipment.
[0106] In order to further improve the recognition rate of fuzzy small targets, it is necessary to combine the local attention in the image context while considering the global attention. Therefore, in an embodiment of the present application, a selective kernel convolution layer SKnet (Selective Kernel Convolution) that supports multi-scale feature extraction is introduced into the existing YOLOv8s, so that different neurons in the target recognition model can obtain different receptive fields. Among them, the selective kernel convolution layer includes a conventional convolution layer and a hole convolution layer. The hole convolution is also called an expanded convolution, which can provide a larger receptive field than a conventional convolution layer of the same size. The feature map is convolved according to the conventional convolution layer to obtain a first feature matrix, and the feature map is convolved according to the hole convolution layer to obtain a second feature matrix; according to the first feature matrix and the second feature matrix and their corresponding weights, the output feature matrix corresponding to the input image is obtained. Among them, the process of convolution using a conventional convolution layer and a hole convolution layer can refer to the prior art and will not be repeated here.
[0107] Figure 4A schematic diagram of a selective kernel convolution structure provided for some embodiments of the present application. Figure 4 As shown, for the input feature map, selective kernel convolution is used for convolution, that is, conventional convolution layer and hole convolution layer can be used for convolution respectively, and the first feature matrix (R1) is obtained according to the conventional convolution layer, and the second feature matrix (R2) is obtained according to the hole convolution layer. According to the first feature matrix and the second feature matrix and the weight a corresponding to the first feature matrix and the weight b corresponding to the second feature matrix, the output feature matrix corresponding to the input image is obtained.
[0108] Specifically, the output feature matrix can be determined using the following formula:
[0109] O=aR1+bR2,a+b=1
[0110] Among them, O represents the output feature matrix, a represents the weight corresponding to the first feature matrix, R1 represents the first feature matrix, R2 represents the second feature matrix, and b represents the weight corresponding to the second feature matrix.
[0111] In the embodiment of the present application, the introduction of SKNet includes three stages: splitting, fusion, and selection: in the splitting stage, two different convolutional layer operations, a 3x3 conventional convolutional layer and a 3x3 hole convolutional layer, can be used to perform convolution operations to obtain different convolution results R1 and R2; in the fusion and selection stages, the input weights are processed by the Softmax function to obtain the weight a corresponding to the first feature matrix and the weight b corresponding to the second feature matrix, so that the neurons of the target recognition model obtain different inputs, and finally the output feature matrix is calculated. According to the output feature matrix, the position information and type of each target to be identified contained in the image can be determined. Among them, the type can be the type with the highest confidence corresponding to the target to be identified.
[0112] In order to improve the detection of blurred small targets by the target recognition model, the idea of SKnet was introduced. While introducing the hole convolution layer to increase the receptive fields of different scales, different neurons were assigned to obtain different receptive field combinations, thereby improving the detection of distant blurred small targets.
[0113] In an embodiment of the present application, the target recognition model uses a deep separable convolutional layer to perform convolution on the input image, and uses a normal convolutional layer to adjust the number of channels so that the feature map output by the deep separable convolutional layer is equal to the number of channels of the input image, further reducing the time consumption and resource usage of the target recognition model, facilitating the deployment of front-ends such as sub-electronic devices, and introducing the idea of SKnet. While introducing dilated convolution to increase the receptive fields of different scales, different neurons can be assigned to obtain different receptive field combinations, thereby improving the detection of fuzzy small targets.
[0114] The whole process of stray target identification in the embodiment of the present application is further explained below with a specific example.
[0115] Figure 5 Another schematic diagram of a wandering target identification process is provided for some embodiments of the present application. Figure 5 As shown, the process includes the following steps:
[0116] S501: Extract each frame of the surveillance video; input each frame of the image into a pre-trained target recognition model in turn, and obtain the location information of the target to be identified and the type of the target to be identified contained in each frame of the image output by the target detection model.
[0117] S502: Determine whether the type of the target to be identified is a preset type; if it is bedding in the preset type, execute S504; if it is a human body in the preset type, execute S503a; if it is a begging sign and a human body in the preset type, execute S503b; if there is no preset type, execute S507b.
[0118] S503a: Input the image of the preset type of target to be identified into a pre-trained human posture classification model, obtain the target posture of the human body output by the human posture classification model, and if the target posture is not standing, execute S504; if the target posture is standing, execute S507b.
[0119] S503b: Determine the distance between the target to be identified of the preset type being a human body and the target to be identified of the preset type being a begging sign; if the distance is less than a second preset distance, execute S504; if the distance is not less than the second preset distance, execute S507b.
[0120] S504: Determine whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image. If so, execute S505; if not, execute S506.
[0121] S505: Determine the moving distance of the preset type of target to be identified according to the position information of the preset type of target to be identified contained in each frame image; if the distance is less than the first preset distance, execute S506; if the distance is not less than the first preset distance, execute S507b.
[0122] S506: Determine whether the existence of a preset type of target to be identified that has not moved or has moved a distance less than a first preset distance in the current frame image exceeds a preset time length, or whether the proportion of the number of frames of the image in which the preset type of target to be identified that has not moved or has moved a distance less than the first preset distance accounts for the total number of frames of the surveillance video exceeds a preset threshold; if any of the judgment results is yes, execute S507a; if none of the judgment results is negative, execute S507b.
[0123] S507a: Determine whether there is a wandering target in the area where the surveillance video is located.
[0124] S507b: Determine that there is no stray target in the area where the surveillance video is located.
[0125] In a possible implementation, for the target to be identified and the duration of existence of the target to be identified in S508a, if the preset type is a human body, the target posture of the human body needs to be uploaded as a warning result to the upper platform, and the relevant departments are notified to determine whether to provide immediate assistance. In the embodiment of the present application, a comprehensive judgment is made based on the target posture of the human body and the category of environmental items. Different priorities can be set for each preset type identified. The existence time of the target to be identified, the target posture of the human body, and the possible temporary residence place, i.e., bedding, can also be recorded to improve the recognition recall rate of homeless beggars.
[0126] In the embodiment of the present application, in the urban street environment, the urban management cameras on the road and at the street entrance are combined with intelligent video analysis technology and alarm logic design to realize the identification and alarm of the homeless population in the city, and notify the relevant department personnel to provide timely assistance. In particular, it is difficult to identify the human body when the homeless person uses bedding, mats or sleeping bags to cover the human body. The present application can also provide auxiliary warning for homeless people by identifying bedding, mats, sleeping bags and other bedding. Through the target posture of the human body and the interactive connection between environmental objects, the inevitable relationship between specific human behavior and environmental objects, not only can the behavior recognition rate of homeless beggars be improved, but also the time of their homeless begging, the state of the human body, i.e., the target posture, and the possible temporary residence place, i.e., bedding, can be identified. Through the judgment logic such as the proportion of historical position trajectories and the association of environmental objects, it can be avoided that the false detection results of a few frames affect the accuracy of homeless target recognition.
[0127] Embodiment 6:
[0128] Based on the same technical concept, in the above-mentioned embodiments, a wandering target identification device is provided in the embodiment of the present application. Figure 6 A schematic diagram of a stray target identification device provided in some embodiments of the present application. Figure 6 As shown, the device comprises:
[0129] Extraction module 601, used to extract each frame of image in the surveillance video;
[0130] The target recognition module 602 is used to input each frame of the image into a pre-trained target recognition model in sequence, and obtain the position information of the target to be recognized contained in each frame of the image output by the target detection model and the type of the target to be recognized;
[0131] The target tracking module 603 is used to determine whether the target of the preset type moves according to the position information of the target contained in each frame of image if the type of the target to be identified is a preset type;
[0132] The judgment module 604 is used to determine whether there is a wandering target in the area where the surveillance video is located if the preset type of target to be identified has not moved.
[0133] In a possible implementation, the target tracking module 603 is further configured to determine, if the preset type of target to be identified moves, a moving distance of the preset type of target to be identified according to the position information of the preset type of target to be identified contained in each frame of image;
[0134] The judgment module 604 is further configured to determine that a wandering target exists in the area where the surveillance video is located if the distance is less than a first preset distance.
[0135] In a possible implementation, the target recognition module 602 is specifically configured to, if the preset type is a human body, input an image of a target to be recognized of the preset type into a pre-trained human posture classification model, and obtain a target posture of the human body output by the human posture classification model;
[0136] The target tracking module 603 is specifically used to determine whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame image if the target posture is not standing.
[0137] In one possible implementation, the target tracking module 603 is specifically used to determine the distance between the target to be identified of the preset type being a human body and the target to be identified of the preset type being a begging sign if the preset types are a begging sign and a human body; if the distance is less than a second preset distance, then determine whether the distance between the target to be identified of the preset type being a human body and the target to be identified of the preset type being a begging sign has changed based on the position information of the target to be identified contained in each frame image.
[0138] In a possible implementation, the target recognition module 602 is specifically used to enable the target recognition model to perform convolution on the input image using a depthwise separable convolution layer, and to adjust the number of channels using a common convolution layer so that the feature map output by the depthwise separable convolution layer is equal to the number of channels of the input image; to perform convolution on the feature map using a selective kernel convolution layer; wherein the selective kernel convolution layer includes a conventional convolution layer and a hole convolution layer; to obtain a first feature matrix according to the conventional convolution layer, and to obtain a second feature matrix according to the hole convolution layer; to obtain an output feature matrix corresponding to the input image according to the first feature matrix and the second feature matrix and their respective corresponding weights; and to determine and output the position information of the target to be identified contained in the image and the type of the target to be identified according to the output feature matrix.
[0139] The specific manner in which each module performs operations in the device of the embodiment of the present application has been described in detail in the embodiment of the wandering target identification method, and will not be elaborated here.
[0140] In an embodiment of the present application, each frame of the surveillance video is extracted, and each frame of the image is sequentially input into a pre-trained target recognition model to obtain the position information and type of the target to be identified contained in each frame of the image output by the target detection model, so as to accurately determine whether there is a stray target; since stray targets and their related items generally stay in a certain place for a short period of time, if there is a target to be identified of a preset type related to the stray target, then based on the position information of the target to be identified contained in each frame of the image, it is determined whether the target to be identified of the preset type has moved; if not, it is determined that there is a stray target in the area where the surveillance video is located, so as to accurately and timely determine whether there are homeless people in the city streets.
[0141] Embodiment 7:
[0142] Based on the same technical concept and on the basis of the above embodiments, an electronic device is provided in an embodiment of the present application.
[0143] Figure 7 A schematic diagram of the structure of an electronic device provided in some embodiments of the present application. Figure 7 As shown, the electronic device includes: a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704;
[0144] The memory 703 stores a computer program. When the program is executed by the processor 701, the processor 701 performs the following steps:
[0145] Extract each frame of the surveillance video; input each frame of the image into a pre-trained target recognition model in turn, and obtain the location information of the target to be identified and the type of the target to be identified contained in each frame of the image output by the target detection model;
[0146] If the type of the target to be identified is a preset type, determining whether the target to be identified of the preset type moves according to the position information of the target to be identified contained in each frame of image;
[0147] If not, it is determined that there is a wandering target in the area where the surveillance video is located.
[0148] In a possible implementation manner, the processor 701 is further configured to:
[0149] If the preset type of target to be identified moves, determine the moving distance of the preset type of target to be identified according to the position information of the preset type of target to be identified contained in each frame of image;
[0150] If the distance is less than the first preset distance, it is determined that there is a wandering target in the area where the surveillance video is located.
[0151] In a possible implementation manner, the processor 701 is further configured to:
[0152] If the preset type is a human body, inputting an image of a target to be identified of the preset type into a pre-trained human posture classification model, and obtaining a target posture of the human body output by the human posture classification model;
[0153] If the target posture is not standing, the subsequent step of determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image is performed.
[0154] In a possible implementation manner, the processor 701 is further configured to:
[0155] The preset types are a begging sign and a human body, and determining the distance between the target to be identified whose preset type is a human body and the target to be identified whose preset type is a begging sign;
[0156] If the distance is less than the second preset distance, the subsequent step of determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image is performed;
[0157] Wherein, determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image includes:
[0158] According to the position information of the target to be identified contained in each frame of image, it is determined whether the distance between the target to be identified of the preset type being a human body and the target to be identified of the preset type being a begging sign has changed.
[0159] In a possible implementation manner, the processor 701 is further configured to:
[0160] The object recognition model uses a depthwise separable convolutional layer to perform convolution on the input image, and uses a common convolutional layer to adjust the number of channels so that the feature map output by the depthwise separable convolutional layer has the same number of channels as the input image;
[0161] Convolving the feature map using a selective kernel convolution layer; wherein the selective kernel convolution layer includes a conventional convolution layer and a hole convolution layer; obtaining a first feature matrix according to the conventional convolution layer, and obtaining a second feature matrix according to the hole convolution layer;
[0162] Obtaining an output feature matrix corresponding to the input image according to the first feature matrix and the second feature matrix and their corresponding weights;
[0163] According to the output feature matrix, the position information of the target to be identified contained in the image and the type of the target to be identified are determined and output.
[0164] Since the principle of solving the problem by the above electronic device is similar to that of the wandering target identification method, the implementation of the above electronic device can refer to the implementation of the method, and the repeated parts will not be repeated.
[0165] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0166] The communication interface 702 is used for communication between the electronic device and other devices.
[0167] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0168] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (Network Processor, NP), etc.; it can also be a digital signal processing processor (Digital Signal Processing, DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.
[0169] In an embodiment of the present application, each frame of the surveillance video is extracted, and each frame of the image is sequentially input into a pre-trained target recognition model to obtain the position information and type of the target to be identified contained in each frame of the image output by the target detection model, so as to accurately determine whether there is a stray target; since stray targets and their related items generally stay in a certain place for a short period of time, if there is a target to be identified of a preset type related to the stray target, then based on the position information of the target to be identified contained in each frame of the image, it is determined whether the target to be identified of the preset type has moved; if not, it is determined that there is a stray target in the area where the surveillance video is located, so as to accurately and timely determine whether there are homeless people in the city streets.
[0170] Embodiment 8:
[0171] Based on the same technical concept, on the basis of the above embodiments, in an embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program executable by an electronic device, and when the program is run on the electronic device, the electronic device implements the following steps when executing:
[0172] Extract each frame of the surveillance video; input each frame of the image into a pre-trained target recognition model in turn, and obtain the location information of the target to be identified and the type of the target to be identified contained in each frame of the image output by the target detection model;
[0173] If the type of the target to be identified is a preset type, determining whether the target to be identified of the preset type moves according to the position information of the target to be identified contained in each frame of image;
[0174] If not, it is determined that there is a wandering target in the area where the surveillance video is located.
[0175] In a possible implementation manner, if the preset type of target to be identified moves, the method further includes:
[0176] Determine the moving distance of the preset type of target to be identified according to the position information of the preset type of target to be identified contained in each frame of image;
[0177] If the distance is less than the first preset distance, it is determined that there is a wandering target in the area where the surveillance video is located.
[0178] In a possible implementation manner, if the preset type is a human body, before determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image, the method further includes:
[0179] Inputting the image of the preset type of target to be identified into a pre-trained human posture classification model, and obtaining the target posture of the human body output by the human posture classification model;
[0180] If the target posture is not standing, the subsequent step of determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image is performed.
[0181] In a possible implementation manner, if the preset types are begging signs and human bodies, before determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image, the method further includes:
[0182] Determine the distance between the target to be identified whose preset type is a human body and the target to be identified whose preset type is a begging sign;
[0183] If the distance is less than the second preset distance, the subsequent step of determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image is performed;
[0184] Wherein, determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image includes:
[0185] According to the position information of the target to be identified contained in each frame of image, it is determined whether the distance between the target to be identified of the preset type being a human body and the target to be identified of the preset type being a begging sign has changed.
[0186] In a possible implementation, inputting each frame of the image into a pre-trained target recognition model in sequence, and obtaining the position information of the target to be recognized contained in each frame of the image output by the target detection model and the type of the target to be recognized includes:
[0187] The target recognition model uses a depthwise separable convolutional layer to perform convolution on the input image, and uses a common convolutional layer to adjust the number of channels so that the feature map output by the depthwise separable convolutional layer is equal to the number of channels of the input image;
[0188] Convolving the feature map using a selective kernel convolution layer; wherein the selective kernel convolution layer includes a conventional convolution layer and a hole convolution layer; obtaining a first feature matrix according to the conventional convolution layer, and obtaining a second feature matrix according to the hole convolution layer;
[0189] Obtaining an output feature matrix corresponding to the input image according to the first feature matrix and the second feature matrix and their corresponding weights;
[0190] According to the output feature matrix, the position information of the target to be identified contained in the image and the type of the target to be identified are determined and output.
[0191] The above-mentioned computer-readable storage medium can be any available medium or data storage device that can be accessed by the processor in the electronic device, including but not limited to magnetic storage such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc., optical storage such as CD, DVD, BD, HVD, etc., and semiconductor storage such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD), etc.
[0192] In an embodiment of the present application, each frame of the surveillance video is extracted, and each frame of the image is sequentially input into a pre-trained target recognition model to obtain the position information and type of the target to be identified contained in each frame of the image output by the target detection model, so as to accurately determine whether there is a stray target; since stray targets and their related items generally stay in a certain place for a short period of time, if there is a target to be identified of a preset type related to the stray target, then based on the position information of the target to be identified contained in each frame of the image, it is determined whether the target to be identified of the preset type has moved; if not, it is determined that there is a stray target in the area where the surveillance video is located, so as to accurately and timely determine whether there are homeless people in the city streets.
[0193] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0194] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0195] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0196] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0197] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A method for identifying a stray target, characterized in that: The method comprises: Extract each frame of the surveillance video; input each frame of the image into a pre-trained target recognition model in turn, and obtain the location information of the target to be identified and the type of the target to be identified contained in each frame of the image output by the target detection model; If the type of the target to be identified is a preset type, determining whether the target to be identified of the preset type moves according to the position information of the target to be identified contained in each frame of image; If not, it is determined that there is a wandering target in the area where the surveillance video is located.
2. The method according to claim 1, characterized in that: If the preset type of target to be identified moves, the method further includes: Determine the moving distance of the preset type of target to be identified according to the position information of the preset type of target to be identified contained in each frame of image; If the distance is less than the first preset distance, it is determined that there is a wandering target in the area where the surveillance video is located.
3. The method according to claim 1 or 2, characterized in that: If the preset type is a human body, before determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image, the method further includes: Inputting the image of the preset type of target to be identified into a pre-trained human posture classification model, and obtaining the target posture of the human body output by the human posture classification model; If the target posture is not standing, the subsequent step of determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image is performed.
4. The method according to claim 1 or 2, characterized in that: If the preset type is a begging sign and a human body, before determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image, the method further includes: Determine the distance between the target to be identified whose preset type is a human body and the target to be identified whose preset type is a begging sign; If the distance is less than the second preset distance, the subsequent step of determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image is performed; Wherein, determining whether the preset type of target to be identified moves according to the position information of the target to be identified contained in each frame of image includes: According to the position information of the target to be identified contained in each frame of image, it is determined whether the distance between the target to be identified of the preset type being a human body and the target to be identified of the preset type being a begging sign has changed.
5. The method according to claim 1, characterized in that The step of sequentially inputting each frame of the image into a pre-trained target recognition model, and obtaining the position information of the target to be recognized contained in each frame of the image output by the target detection model and the type of the target to be recognized comprises: The target recognition model uses a depthwise separable convolutional layer to perform convolution on the input image, and uses a common convolutional layer to adjust the number of channels so that the feature map output by the depthwise separable convolutional layer is equal to the number of channels of the input image; Convolving the feature map using a selective kernel convolution layer; wherein the selective kernel convolution layer includes a conventional convolution layer and a hole convolution layer; obtaining a first feature matrix according to the conventional convolution layer, and obtaining a second feature matrix according to the hole convolution layer; Obtaining an output feature matrix corresponding to the input image according to the first feature matrix and the second feature matrix and their corresponding weights; According to the output feature matrix, the position information of the target to be identified contained in the image and the type of the target to be identified are determined and output.
6. A wandering target identification device, characterized in that: The device comprises: An extraction module is used to extract each frame of the surveillance video; A target recognition module is used to input each frame of the image into a pre-trained target recognition model in sequence, and obtain the position information of the target to be recognized contained in each frame of the image output by the target detection model and the type of the target to be recognized; A target tracking module, for determining whether the target of the preset type moves according to the position information of the target contained in each frame of image if the type of the target to be identified is a preset type; The judgment module is used to determine that there is a wandering target in the area where the surveillance video is located if the preset type of target to be identified has not moved.
7. The device according to claim 6, characterized in that The target tracking module is further configured to determine, if the preset type of target to be identified moves, a moving distance of the preset type of target to be identified based on the position information of the preset type of target to be identified contained in each frame of image; The judgment module is further configured to determine that there is a wandering target in the area where the surveillance video is located if the distance is less than a first preset distance.
8. The device according to claim 6, characterized in that The target recognition module is specifically used to input the image of the preset type of target to be recognized into a pre-trained human posture classification model if the preset type is a human body, and obtain the target posture of the human body output by the human posture classification model; The target tracking module is specifically used to determine whether the target to be identified of the preset type moves according to the position information of the target to be identified contained in each frame image if the target posture is not standing.
9. An electronic device, characterized in that: The electronic device comprises at least a processor and a memory, and the processor is configured to implement the steps of any one of the methods described in claims 1 to 5 when executing a computer program stored in the memory.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program executable by an electronic device. When the program runs on the electronic device, the electronic device executes the steps of any one of the methods described in claims 1-5.
Citation Information
Patent Citations
Event detection method and device, electronic equipment and storage medium
CN112507813A
Monitoring method and device
CN112989893A
Orchard complex road segmentation method based on lightweight semantic segmentation algorithm
CN114463542A
Image processing method and device, model training method and device, equipment and storage medium
CN115908982A
Intelligent cabin-oriented driver distraction behavior identification method and device
CN116883975A
