Behavior information identification method, system and device, storage medium and program product

Abstract

The invention provides a behavior information identification method, system and device, a storage medium and a program product. The method comprises the following steps: acquiring a monitoring video frame in a specified area; identifying target bag information existing in a human body object area in the monitoring video frame; the bagging posture and the article holding behavior of the human body object are detected; and if the bagging posture is a preset posture corresponding to the target bag information and the human body object has an article holding behavior before the bagging posture occurs, determining that the human body object has a bagging behavior in the designated area, and sending prompt information. According to the method and the device, whether the human body object is subjected to the bagging behavior or not can be accurately judged by combining the bagging posture, the article holding behavior and the preset posture corresponding to the target bag information, the possibility of misjudgment is reduced, and the identification efficiency of the bagging behavior is improved.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and in particular to a method, system, device, storage medium and program product for identifying behavior information. Background Art

[0002] With the development of retail business, supermarkets no longer require customers to deposit bags. Customers pack goods into their own bags (hereinafter referred to as "packing") and take them out of the supermarket, resulting in loss of goods. In order to promptly detect and stop packing behavior, supermarkets are generally equipped with loss prevention personnel to manually patrol and detect possible packing behavior in supermarkets.

[0003] However, the entire packaging process usually takes a short time, and supermarkets are large and crowded, so it is difficult for loss prevention officers to detect and stop the packaging behavior in a timely and accurate manner, resulting in low inspection efficiency. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to provide a method, system, device, storage medium and program product for identifying behavior information, which can accurately determine whether a human object has performed a packaging behavior by combining the packaging posture, the behavior of holding an item and the preset posture corresponding to the target bag information, thereby reducing the possibility of misjudgment and improving the recognition efficiency of the packaging behavior.

[0005] In a first aspect, an embodiment of the present application provides a method for identifying behavioral information, including: acquiring surveillance video frames within a specified area; identifying target bag information existing in a human object area in the surveillance video frames; detecting a bagging posture and a hand-holding behavior of the human object; if the bagging posture is a preset posture corresponding to the target bag information, and the human object has hand-holding behavior before the bagging posture occurs, it is determined that the human object has performed a bagging behavior in the specified area, and a prompt message is issued.

[0006] In one embodiment, the identifying target bag information existing in the human object area in the surveillance video frame includes: identifying human trajectory information in the surveillance video frame; capturing a human image from the surveillance video frame according to the human trajectory information; and inputting the human image into a preset bag detection model so that the preset bag detection model outputs target bag information existing in the human image, the target bag information including bag position and / or bag type.

[0007] In one embodiment, before inputting the human body image into the preset bag detection model, it also includes: obtaining a bag sample image set, in which the positions and types of different bag objects are marked; using the bag sample image set to train a preset detector to obtain the preset bag detection model.

[0008] In one embodiment, the detecting of the packing posture of the human object includes: identifying human trajectory information in the surveillance video frame; capturing a human image from the surveillance video frame according to the human trajectory information; and inputting the human image into a preset posture detection model, so that the preset posture detection model outputs the packing posture existing in the human image, the type of bag in which the packing posture occurs, and the occurrence time of the packing posture.

[0009] In one embodiment, before inputting the human body image into a preset posture detection model, it also includes: obtaining a set of packaging sample images, in which a variety of packaging postures and non-packaging postures corresponding to different target bag information are marked; and using the packaging sample image set to train a preset detector to obtain the preset posture detection model.

[0010] In one embodiment, detecting the hand-holding item behavior of the human object includes: identifying human trajectory information in the monitoring video frame; capturing a human image from the monitoring video frame according to the human trajectory information; inputting the human image into a preset behavior detection model, so that the preset behavior detection model outputs the hand of the human object and the target item held by the hand in the human image; if the distance between the position of the target item and the position of the hand of the human object is less than a preset threshold, determining that the human object has performed the hand-holding item behavior, and determining the occurrence time of the hand-holding item behavior.

[0011] In one embodiment, before inputting the human body image into the preset behavior detection model, it also includes: obtaining a set of sample images of handheld items, in which objects with different appearance features are marked; using the handheld item sample image set to train a preset detector to obtain the preset behavior detection model.

[0012] In one embodiment, the bag types include one or more of a backpack, a shoulder bag with the bag located above the waist, a shoulder bag below the waist, a forearm bag, and a handbag.

[0013] In a second aspect, an embodiment of the present application provides a supermarket anti-theft detection method, comprising: acquiring surveillance video frames in a supermarket; identifying target bag information existing in a human object area in the surveillance video frames; detecting the bagging posture and commodity holding behavior of the human object; if the bagging posture is a preset posture corresponding to the target bag information, and the human object has held the commodity before the bagging posture occurs, it is determined that the human object has bagged the commodity in the supermarket, and an anti-theft prompt message is issued.

[0014] In a third aspect, an embodiment of the present application provides a device for identifying behavior information, including: An acquisition module is used to acquire surveillance video frames within a specified area; An identification module, used to identify target bag information existing in the human object area in the monitoring video frame; A detection module, used to detect the bagging posture and the object holding behavior of the human subject; The determination module is used to determine that the human subject has performed a packing behavior in the specified area and issue a prompt message if the packing posture is a preset posture corresponding to the target bag information and the human subject has held an item before the packing posture occurs.

[0015] In one embodiment, the recognition module is used to identify human body trajectory information in the surveillance video frame; capture a human body image from the surveillance video frame according to the human body trajectory information; and input the human body image into a preset bag detection model so that the preset bag detection model outputs target bag information present in the human body image, wherein the target bag information includes bag location and / or bag type.

[0016] In one embodiment, it also includes: a first training module, used to obtain a set of bag sample images before inputting the human body image into the preset bag detection model, the bag sample images marking the positions and types of different bag objects; use the bag sample image set to train a preset detector to obtain the preset bag detection model.

[0017] In one embodiment, the detection module is used to identify human body trajectory information in the surveillance video frame; capture a human body image from the surveillance video frame according to the human body trajectory information; and input the human body image into a preset posture detection model so that the preset posture detection model outputs the packing posture existing in the human body image, the type of bag in which the packing posture occurs, and the occurrence time of the packing posture.

[0018] In one embodiment, it also includes: a second training module, which is used to obtain a set of packaging sample images before inputting the human body image into the preset posture detection model, and the packaging sample images are marked with a variety of packaging postures and non-packaging postures corresponding to different target bag information; use the packaging sample image set to train the preset detector to obtain the preset posture detection model.

[0019] In one embodiment, the detection module is used to identify human trajectory information in the monitoring video frame; capture a human image from the monitoring video frame according to the human trajectory information; input the human image into a preset behavior detection model, so that the preset behavior detection model outputs the hand of the human object and the target object held by the hand in the human image; if the distance between the position of the target object and the hand position of the human object is less than a preset threshold, it is determined that the human object has performed the hand-holding object behavior, and the occurrence time of the hand-holding object behavior is determined.

[0020] In one embodiment, it also includes: a third training module, which is used to obtain a set of sample images of handheld items before inputting the human body image into the preset behavior detection model, and the handheld item sample images are marked with objects with different appearance features; use the handheld item sample image set to train a preset detector to obtain the preset behavior detection model.

[0021] In one embodiment, the bag types include one or more of a backpack, a shoulder bag with the bag located above the waist, a shoulder bag below the waist, a forearm bag, and a handbag.

[0022] In a fourth aspect, an embodiment of the present application provides a behavior information recognition system, including: A bag detection module is used to obtain surveillance video frames in a specified area and identify target bag information existing in the human object area in the surveillance video frames; A hand-held object detection module, used to detect hand-held object behavior of the human subject; A packing posture detection module, used to detect the packing posture of the human object; The packing behavior decision module is respectively connected to the bag detection module, the handheld item detection module and the packing posture detection module, and is used to determine that the human object has a packing behavior in the specified area, and issue a prompt message if the packing posture is a preset posture corresponding to the target bag information and the human object has a handheld item behavior before the packing posture occurs.

[0023] In a fifth aspect, an embodiment of the present application provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the electronic device to execute the method described in any one of the above aspects.

[0024] In a sixth aspect, an embodiment of the present application provides a cloud device, including: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the cloud device to execute the method described in any one of the above aspects.

[0025] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the method described in any one of the above aspects is implemented.

[0026] In an eighth aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements the method described in any of the above aspects when executed by a processor.

[0027] The behavior information recognition method, system, device, storage medium and program product provided in the embodiments of the present application, by identifying the target bag information of the human object area in the monitoring video frame and the bagging posture and hand-held item behavior of the human object, and matching the bagging posture of the human object with the target bag information, if the bagging posture is the preset posture corresponding to the target bag information, and the human object has hand-held item behavior before the bagging posture occurs, it is determined that the human object has bagging behavior in the specified area, and a prompt message can be issued at this time to stop the bagging behavior in time. In this way, by combining the bagging posture, hand-held item behavior and the preset posture corresponding to the target bag information, it is possible to accurately determine whether the human object has bagging behavior, reduce the possibility of misjudgment, and improve the recognition efficiency of bagging behavior. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. Obviously, the drawings described below are some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0029] Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application; Figure 2 A schematic diagram of an application scenario of a behavior information recognition system provided in an embodiment of the present application; Figure 3 A schematic diagram of a framework of a behavior information recognition system provided in an embodiment of the present application; Figure 4 A flowchart of a method for identifying behavior information provided in an embodiment of the present application; Figure 5 A schematic diagram of a packing posture of a shoulder bag provided in an embodiment of the present application; Figure 6 A flowchart of a method for identifying behavior information provided in an embodiment of the present application; Figure 7 A schematic diagram of a supermarket anti-theft detection method flow chart provided in an embodiment of the present application; Figure 8 A schematic diagram of the structure of a device for identifying behavior information provided in an embodiment of the present application; Fig. 9 A schematic diagram of the structure of a cloud device provided in an embodiment of the present application.

[0030] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0031] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application.

[0032] The terms "first", "second", "third", "fourth", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances. For example, without departing from the scope of this document, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information.

[0033] The word "if" as used herein may be interpreted as "when" or "when" or "in response to determining," depending on the context.

[0034] Furthermore, as used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context indicates otherwise.

[0035] It should be further understood that the terms “comprises” and “includes” indicate the existence of features, steps, operations, elements, components, items, types, and / or groups, but do not exclude the existence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, types, and / or groups.

[0036] The term "and / or" in this article is used to describe the association relationship of associated objects, specifically indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0037] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0038] In order to clearly describe the technical solution of the embodiment of the present application, the terms involved in the present application are first defined: Bags: Any bag or bag that can hold items, including but not limited to backpacks, shoulder bags, messenger bags, briefcases, canvas bags, plastic bags, paper bags, etc.

[0039] Packing behavior: the act of putting items into one's own bags, such as putting supermarket items into one's own bag.

[0040] POS: Point of Sales, generally refers to a cash register terminal device. In the embodiment of the present application, it may refer to a self-service QR code scanning cash register device used by supermarkets for customers.

[0041] ID: Identification, unique identification.

[0042] Yolov5: is a deep learning model for object detection, belonging to the "You Only Look Once" (YOLO) family.

[0043] ResNet: Residual Network, the full name is residual network.

[0044] Resnet-18: is a deep convolutional neural network architecture and a member of the ResNet family.

[0045] The watermark information embedding method of the embodiment of the present application can be applied to any field involving watermark processing, such as AIGC model distribution tracing, AIGC synthetic image marking protection and other scenarios.

[0046] With the development of retail business, supermarkets no longer require customers to deposit bags. Customers pack goods into their own bags (hereinafter referred to as "packing") and take them out of the supermarket, resulting in loss of goods. In order to promptly detect and stop packing behavior, supermarkets are generally equipped with loss prevention personnel to manually patrol and detect possible packing behavior in supermarkets.

[0047] However, the entire packaging process usually takes a short time, and supermarkets are large and crowded, so it is difficult for loss prevention officers to detect and stop the packaging behavior in a timely and accurate manner, resulting in low inspection efficiency.

[0048] In the related art, many manufacturers have developed products in the fields of self-service checkout machines and unmanned supermarkets to address the problem of supermarket loss prevention. For example, in response to the demand for supermarket loss prevention, there are generally POS-side loss prevention and loss prevention based on pick-up and placement identification. Among them, POS-side loss prevention targets self-service checkout equipment. However, people who pack in supermarkets may not pass through the checkout area, but leave directly from the non-shopping channel, resulting in the packaged goods not being taken out in front of the self-service checkout machine. In this case, the self-service checkout machine cannot detect risks. Therefore, it can complement the solution of the embodiment of the present application. The loss prevention solution based on pick-up and placement identification requires a longer link, such as confirming that the customer takes the goods away from the shelf, and comparing it with the taken away goods at checkout. It involves multiple links such as pick-up and placement identification, product identification, personnel tracking, and connecting with bill data, which is difficult to implement or has high costs.

[0049] In an optional embodiment, for the packaging recognition method, the packaging behavior can be identified based on the video action recognition method, but this method requires a larger computing device to support the calculation of inter-frame information. In addition, there are complexities in obtaining training samples and the generalization ability of the model.

[0050] In order to solve at least one of the above problems, the embodiment of the present application provides a behavior information recognition scheme, which recognizes the target bag information of the human object area in the monitoring video frame and the bagging posture and hand-held item behavior of the human object, and matches the bagging posture of the human object with the target bag information. If the bagging posture is the preset posture corresponding to the target bag information, and the human object has hand-held item behavior before the bagging posture occurs, it is determined that the human object has bagging behavior in the specified area. At this time, a prompt message can be issued to stop the bagging behavior in time. In this way, by combining the bagging posture, hand-held item behavior and the preset posture corresponding to the target bag information, it is possible to accurately judge whether the human object has bagging behavior, reduce the possibility of misjudgment, and improve the recognition efficiency of bagging behavior.

[0051] Some embodiments of the present application are described in detail below in conjunction with the accompanying drawings. In the case where there is no conflict between the embodiments, the following embodiments and the features in the embodiments can be combined with each other. In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.

[0052] like Figure 1 As shown, this embodiment provides an electronic device 1, including: at least one processor 11 and a memory 12, Figure 1 A processor is taken as an example. The processor 11 and the memory 12 are connected via a bus 10. The memory 12 stores instructions that can be executed by the processor 11, and the instructions are executed by the processor 11 so that the electronic device 1 can execute all or part of the process of the method in the following embodiment, so as to realize that by combining the packing posture, the handheld object behavior and the preset posture corresponding to the target bag information, it can accurately determine whether the human object has a packing behavior, reduce the possibility of misjudgment, and improve the recognition efficiency of the packing behavior.

[0053] In one embodiment, the electronic device 1 may be a mobile phone, a tablet computer, a laptop computer, a desktop computer, or a large computing system composed of multiple computers.

[0054] Figure 2 A schematic diagram of an application scenario 200 of a behavior information recognition system provided in an embodiment of the present application. Figure 2 As shown, the system includes: a server 210 and a terminal 220, wherein: Server 210 may be a data platform that provides behavioral information recognition services, such as a supermarket monitoring service platform. In actual scenarios, a monitoring service platform may have multiple servers 210. Figure 2 In the figure, one server 210 is taken as an example.

[0055] The terminal 220 may be a computer, mobile phone, tablet or other device used by the user to log in to the monitoring service platform. There may be multiple terminals 220. Figure 2 Two terminals 220 are taken as an example for illustration.

[0056] The terminal 220 and the server 210 can transmit information via the Internet, so that the terminal 220 can access the data on the server 210. The terminal 220 and / or the server 210 can be implemented by the electronic device 1.

[0057] The behavior information identification scheme of the embodiment of the present application can be deployed on the server 210, can also be deployed on the terminal 220, or can be deployed partially on the server 210 and partially on the terminal 220. In actual scenarios, it can be selected based on actual needs, and this embodiment does not limit it.

[0058] When the behavior information recognition solution is fully or partially deployed on the server 210 , a calling interface may be opened to the terminal 220 to provide algorithm support to the terminal 220 .

[0059] The method provided in the embodiment of the present application can be implemented by executing the corresponding software code by the electronic device 1, and can be implemented by exchanging data with the server. The electronic device 1 can be a local terminal device. When the method is run on the server, the method can be implemented and executed based on the cloud interaction system, wherein the cloud interaction system includes a server and a client device.

[0060] In a possible implementation, the method provided in the embodiment of the present application provides a graphical user interface through a terminal device, wherein the terminal device can be the local terminal device mentioned above, or can be the client device in the cloud interaction system mentioned above.

[0061] like Figure 3 FIG. 1 is a schematic diagram of a framework of a behavior information recognition system according to an embodiment of the present application. The system includes a bag detection module, a handheld object detection module, a packing posture detection module, and a packing behavior decision module, wherein: The bag detection module is used to obtain surveillance video frames in a specified area and identify target bag information existing in the human object area in the surveillance video frames.

[0062] The handheld object detection module is used to detect the handheld object behavior of the human subject.

[0063] The packing posture detection module is used to detect the packing posture of the human body object.

[0064] The packing behavior decision module is respectively connected to the bag detection module, the handheld object detection module and the packing posture detection module, and is used to determine that the human object has a packing behavior in the specified area and issue a prompt message if the packing posture is a preset posture corresponding to the target bag information and the human object has a handheld object behavior before the packing posture occurs.

[0065] The system of the embodiment of the present application, by identifying the human body trajectory in the monitoring video frame, determines the target bag information of the human body object area according to the human body trajectory through the bag detection module, identifies the bagging posture of the human body object according to the monitoring video frame through the bagging posture detection module, identifies the hand-held object behavior of the human body object according to the monitoring video frame through the hand-held object detection module, and then matches the bagging posture of the human body object with the target bag information through the bagging behavior decision module. If the bagging posture is the preset posture corresponding to the target bag information, and the human body object has hand-held object behavior before the bagging posture occurs, it is determined that the human body object has bagging behavior in the specified area. At this time, prompt information can be issued, such as outputting the bagging behavior decision conclusion, the bagging personnel information or the video clip of the bagging behavior, so as to stop the bagging behavior in time. In this way, by combining the bagging posture, the hand-held object behavior and the preset posture corresponding to the target bag information, it can accurately determine whether the human body object has bagging behavior, reduce the possibility of misjudgment, and improve the recognition efficiency of bagging behavior.

[0066] Optionally, the system may also include a human trajectory extraction module, connected to the bag detection module, for extracting human trajectory information from the surveillance video frame to form a structured description of the human object. Specifically, for an acquired frame of image, a human detection algorithm is used to detect the positions of all human objects from the image, and then a human tracking algorithm is used to associate the detected human objects with the same human objects in the previous frame, and finally the trajectory information of each human object in the surveillance video frame is formed. The human trajectory information includes but is not limited to the identity ID of the human object and the position coordinates of the human object in each frame. The human trajectory information can be used as the prerequisite information for packaging behavior recognition and sent to the subsequent bag detection module together with the corresponding surveillance video frame.

[0067] Please see Figure 4 , which is a method for identifying behavior information in an embodiment of the present application, the method can be Figure 1 The electronic device 1 shown is used to perform and can be applied to Figure 2-Figure 3 In the supermarket monitoring application scenario shown in , by combining the bagging posture, the hand-held object behavior and the preset posture corresponding to the target bag information, it is possible to accurately determine whether the human object has bagging behavior, thereby reducing the possibility of misjudgment and improving the recognition efficiency of bagging behavior. This embodiment takes the terminal 220 as an example of the execution end, and the method includes the following steps: Step 401: Acquire surveillance video frames within a specified area.

[0068] In this step, the designated area may refer to an area covered by surveillance, such as a surveillance coverage area of ​​a supermarket or a surveillance coverage area of ​​a shopping mall. The surveillance video frame may be acquired in real time from a surveillance camera, or may be a surveillance video stored locally or remotely. There is no limitation on the method of acquiring the surveillance video frame.

[0069] Step 402: Identify target bag information existing in the human object area in the surveillance video frame.

[0070] In this step, in order to accurately identify whether bagging behavior occurs in the specified area, the target bag information in the human subject area in the surveillance video frame can be first identified, and then it can be determined whether the human subject carries a bag and what kind of bag it is. The target bag information includes but is not limited to the type, size, bag appearance and other information of the target bag carried by the human subject. The bag types here include but are not limited to backpacks, shoulder bags, handbags, messenger bags and other types. By identifying the target bag information, a reliable data basis can be provided for subsequent bagging behavior decisions.

[0071] In one embodiment, identifying target bag information existing in a human object area in a surveillance video frame includes: identifying human trajectory information in the surveillance video frame. Capturing a human image from the surveillance video frame according to the human trajectory information. Inputting the human image into a preset bag detection model so that the preset bag detection model outputs target bag information existing in the human image, the target bag information including bag position and / or bag type.

[0072] In this embodiment, by identifying human body trajectory information and intercepting human body images, the human body object area in the video frame can be effectively located. The identification method of human body trajectory information can refer to the description of the human body trajectory extraction module in the previous embodiment. By inputting the human body image into the preset bag detection model, the target bag information in the human body image can be accurately identified. The target bag information includes but is not limited to the location and type of the target bag. In this way, "bag detection" and "bag classification" are implemented by the same model. The model takes the human body image as input and gives two outputs of the bag location and bag type after reasoning, thereby improving the efficiency and accuracy of bag information recognition. The preset model is trained and optimized, and can process a large number of video frames in a short time, thereby improving the processing efficiency of the system. On the other hand, using human body trajectory information, human objects in the video can be dynamically tracked, and the target bag can be continuously identified and monitored even in complex scenes. This dynamic tracking capability ensures the continuity and stability of the recognition process.

[0073] In one embodiment, before inputting the human body image into the preset bag detection model, the method further includes: obtaining a bag sample image set, in which the locations and types of different bag objects are marked, and using the bag sample image set to train a preset detector to obtain a preset bag detection model.

[0074] In this embodiment, by pre-acquiring and marking a set of bag sample images, and then using these marked sample image sets to train a preset detector, the accuracy and robustness of the bag detection model can be significantly improved. In order to balance speed and accuracy, the preset detector can be implemented by a Yolov5 detector, and other detectors that can perform similar functions can be applicable. The locations and types of different bag objects are marked in the sample images, so that the training process can better capture the characteristics and variability of the bags, thereby improving the detection performance of the model in practical applications. This enables the bag detection model to adapt to bags of different types and styles, improves the generalization ability of the model, and enables it to maintain efficient detection effects in a variety of scenarios. Through this preprocessing and training step, the final preset bag detection model can more accurately identify and classify bag objects in human images, and improve the recognition efficiency and reliability of the overall system.

[0075] In one embodiment, the bag types include one or more of a backpack, a shoulder bag with the bag located above the waist, a shoulder bag with the bag located below the waist, a forearm bag, and a handbag.

[0076] In this embodiment, bag types can be classified according to people's "bag holding posture", and bag types include but are not limited to: bags carried on shoulders, bags slung over one shoulder (bags above the waist), bags slung over one shoulder (bags near and below the waist), bags slung over forearms, and bags carried in hands. The above five bag types correspond to different "bag holding postures", which are subsequently matched with the conclusions given by the posture detection model to achieve higher alarm accuracy. Other bag types can also be configured according to actual scene requirements. In the training process of the bag detection model, the bag sample image set can include as many bag types as possible, so that the trained bag detection model can recognize a variety of bags.

[0077] Step 403: Detecting the bag-packing posture and the object-holding behavior of the human object.

[0078] In this step, the packing posture refers to the posture of a human hand or an item in contact with a bag, and at least includes the posture of a human subject putting an item into a bag brought by the human subject, such as the posture of a user putting goods on the shelf into a bag brought by the user in a supermarket scene. In actual scenes, if a packing behavior occurs, there must be a packing posture, so the packing posture can be used as reference information for subsequent packing behavior decisions. The behavior of holding items refers to the behavior of a human subject holding an item in his hand, such as the behavior of a user holding goods in a supermarket scene. Generally, a human subject uses his hands to put items into a bag, so the behavior of holding items can be used as reference information for subsequent packing behavior decisions. By combining the packing posture and the behavior of holding items, the accuracy of the packing behavior decision can be improved.

[0079] In one embodiment, the detection of the hand-held object behavior of the human subject in step 403 includes: identifying the human body trajectory information in the monitoring video frame. Capturing a human body image from the monitoring video frame according to the human body trajectory information. Inputting the human body image into a preset behavior detection model so that the preset behavior detection model outputs the hand of the human body object and the target object held by the hand in the human body image. If the distance between the position of the target object and the position of the hand of the human body object is less than a preset threshold, it is determined that the human body object has a hand-held object behavior, and the time when the hand-held object behavior occurs is determined.

[0080] In this embodiment, the behavior of holding an object in a human image can be identified by a pre-trained behavior detection model. By identifying the human trajectory information in the video frame, the relevant human image can be accurately captured from the video to ensure the accuracy of subsequent detection. After the captured human image is input into the preset behavior detection model, the hand of the human object in the human image and the target object held by the hand can be identified. By calculating the distance between the position of the target object and the position of the hand of the human object, and comparing it with the preset threshold, it can be accurately determined whether the behavior of holding an object has occurred, reducing misjudgment, and accurately recording the time of occurrence of the behavior. The workload of manual monitoring is reduced, the efficiency and accuracy of detection are improved, and it can be applied to security monitoring, retail analysis and other fields, and has significant application value. The preset threshold can be set according to the actual scene requirements, for example, the preset threshold can be the length of the target object. The target object position can be represented by the center point coordinates of the target object, and the hand position can be represented by the key point coordinates of the hand.

[0081] Taking the supermarket scene as an example, the target item is a target product held by a person. The behavior detection model takes the human image as input and gives the position of the target product held by the person in the human image. In order to avoid misdetection of products in the background, the detected product position is matched with the hand position. If the distance between the coordinates of the center point of the product and the coordinates of the key points of the hand is less than a threshold (such as the length of the product), the detected product is determined to be the target product held by the person.

[0082] In one embodiment, before inputting the human body image into the preset behavior detection model, the method further includes: obtaining a sample image set of handheld items, wherein the handheld item sample images are marked with items with different appearance features, and using the handheld item sample image set to train a preset detector to obtain a preset behavior detection model.

[0083] In this embodiment, the sample image of a handheld item refers to an image of a person holding an item. Different types of items generally have different appearance features, such as different commodities have different outer packaging features. In order to improve the recognition accuracy of the model, as many sample images of people holding different types of items as possible can be collected to form a sample image set of handheld items, and the items with different appearance features held by people are marked in these sample images, so as to provide rich training data for the model. Since the sample images cover the appearance features of a variety of items, the model can learn the recognition features of different items during the training process. By using these sample image sets to train the preset detector, the preset behavior detection model finally obtained has strong generalization ability and recognition accuracy, ensuring that the model can more accurately identify the items held by the human subject in the surveillance video frame. The detection efficiency and accuracy of the model in practical applications are improved, and the cases of false detection and missed detection are reduced. It can be applied to various scenarios that require accurate recognition of the behavior of handheld items, such as security monitoring, retail management and other fields.

[0084] The preset detector may be implemented by the Yolov5 detector, and other detectors may also be applicable.

[0085] Taking the supermarket scenario as an example, in order to improve the accuracy of commodity detection and distinguish it from non-commodity items, the output of the behavior detection model can be configured to include at least five categories: boxed goods, bottled goods, bagged goods, mobile phones and other items, among which mobile phones and other items are non-commodity items.

[0086] In one embodiment, detecting the packing posture of the human object in step 403 includes: identifying human trajectory information in the monitoring video frame. Capturing a human image from the monitoring video frame according to the human trajectory information. Inputting the human image into a preset posture detection model so that the preset posture detection model outputs the packing posture existing in the human image, the type of bag in which the packing posture occurs, and the occurrence time of the packing posture.

[0087] In this embodiment, by identifying the human body trajectory information in the video frame, a clear human body image can be effectively captured from the video to ensure the accuracy of subsequent posture detection. Subsequently, by inputting the captured human body image into a preset posture detection model, the packing posture in the image can be automatically identified. Not only can the packing posture be identified, but the type of bag involved in the packing posture and the specific time of occurrence can also be further determined. The time of occurrence of the packing posture can be a time point or a continuous period of time. In this way, the efficiency and accuracy of posture recognition are improved, the need for manual intervention is reduced, and it can be applied to multiple fields such as security monitoring and behavior analysis, and has broad application prospects.

[0088] In one embodiment, before inputting the human body image into the preset posture detection model, the method further includes: obtaining a set of packaging sample images, in which a plurality of packaging postures and non-packaging postures corresponding to different target bag information are marked, and using the packaging sample image set to train a preset detector to obtain a preset posture detection model.

[0089] In this embodiment, the packing sample image refers to an image containing a packing posture, such as an image of a human subject putting an item into a bag. Different types of bags and various packing postures and non-packing postures related thereto may be marked in the packing sample image. The packing sample image set usually contains a variety of different types of bags, such as backpacks, handbags, suitcases, etc. These images show the various postures of people interacting with these bags in different scenes. The sample image set contains not only packing postures, but also non-packing postures (such as human subjects standing and walking without involving bags) to help the model distinguish between packing postures and other non-packing postures. Each sample image may be accompanied by marking information, indicating the type of bag appearing in the image and the corresponding packing posture and non-packing posture. These marking information provides supervision signals, so that the model can learn the characteristics of various packing actions and non-packing actions during the training process.

[0090] Optionally, the preset detector here may be a Resnet-18 classifier, and other types of classifiers may be applicable.

[0091] In actual scenarios, bags of the same type have similar "packing postures", while different "bag types" have different "packing postures", and are separable from unpacked postures. Taking the supermarket scenario as an example, assuming that the bag types include five types: bags carried on shoulders, bags above the waist, bags below the waist, bags carried on the forearm, and bags carried in the hands. The posture detection model takes human body images as input and can give 6 categories, including a "non-packing posture" and the packing postures corresponding to the 5 bag types.

[0092] like Figure 5 As shown, it is a schematic diagram of the packing posture of a single-shoulder bag provided in an embodiment of the present application. In the figure, the human subject carries the bag on one shoulder, the bag is located at and above the waist of the human body, and the human subject holds the goods in contact with the bag, and the packing posture occurs.

[0093] Step 404: If the packing posture is the preset posture corresponding to the target bag information, and the human subject has the behavior of holding an item before the packing posture occurs, it is determined that the human subject has the behavior of packing in the designated area, and a prompt message is issued.

[0094] In this step, the packing posture of the human object detected in step 403 is matched with the target bag information. If the packing posture is the preset posture corresponding to the target bag information, and the human object has a hand-held object behavior before the packing posture occurs, it is determined that the human object has a packing behavior in the specified area. At this time, a prompt message can be issued to stop the packing behavior in time. In this way, by combining the packing posture, the hand-held object behavior and the preset posture corresponding to the target bag information, it is possible to accurately determine whether the human object has a packing behavior, reduce the possibility of misjudgment, and improve the recognition efficiency of the packing behavior.

[0095] Optionally, in order to improve the execution speed of the system, when no bag is detected in step 402, the bagging posture detection and handheld item behavior detection in step 403 may not be performed, thereby saving calculation amount and improving resource utilization.

[0096] Taking the supermarket scenario as an example, the packing behavior decision is based on the "bag" detection results, the handheld product results, and the packing posture detection results to give a conclusion on whether to pack. The decision conditions may include the following: 1. The packing posture detection module gives the packing posture, the bag type involved and the time when the packing posture occurs.

[0097] 2. The bag detection module detects the target bag information, and the type of the target bag is consistent with the bag type involved in the packing posture.

[0098] 3. Before the packaging gesture occurs, the handheld item module detects the product, and the distance between the product and the key points of the hand of the human object is less than a preset threshold (such as the length of the long side of the circumscribed rectangle of the product).

[0099] When the above three decision conditions are met at the same time, it can be determined that a customer has packed goods in a supermarket. At this time, the packing behavior decision module can issue a prompt message, which can include the body trajectory of the customer who is packing, and provide a video clip of the packing behavior according to the time when the packing gesture occurs, so that relevant personnel can view and handle it.

[0100] Optionally, the system can immediately push the risks of packaging behavior to the supermarket's loss prevention personnel in a variety of ways, so that the loss prevention personnel can judge whether the system's decision results are accurate by watching the video, and then track the customer's trajectory in the supermarket, determine whether the checkout behavior is completed, and take further measures to reduce the risk of theft and loss of goods in the supermarket.

[0101] The above-mentioned behavior information recognition method structures the surveillance video frames obtained by the camera to obtain the position of the human object in each frame of the video. The "bag" detection algorithm is used to obtain the position of the bag in the human image and distinguish the types of bags. The commodity detection algorithm is used to identify whether the human object has goods in his hand. Different packing postures are configured for different bag types, and then the packing behavior decision is made by combining the bag type, handheld commodity behavior detection and packing posture detection to achieve accurate packing behavior recognition.

[0102] like Figure 6 As shown, it is a flow chart of a method for identifying behavior information according to an embodiment of the present application. The method can be performed by Figure 1 The electronic device 1 shown is used to perform and can be applied to Figure 2-Figure 3 In the supermarket monitoring application scenario shown in , by combining the bagging posture, the hand-held object behavior and the preset posture corresponding to the target bag information, it is possible to accurately determine whether the human object has bagging behavior, thereby reducing the possibility of misjudgment and improving the recognition efficiency of bagging behavior. This embodiment takes the terminal 220 as an example of the execution end, and the method includes the following steps: Step 601: Acquire surveillance video frames within a specified area.

[0103] Step 602: Identify human body trajectory information in the surveillance video frame.

[0104] Step 603: intercepting a human body image from the monitoring video frame according to the human body trajectory information. Then proceeding to step 604.

[0105] Step 604: Input the human image into the preset bag detection model, and determine whether the preset bag detection model outputs target bag information existing in the human image, the target bag information including bag position and / or bag type. If yes, proceed to step 605 and step 606 respectively; otherwise, it means that the bag is not detected and the subsequent steps may not be executed, and the process may return to step 601.

[0106] Step 605: Input the human body image into the preset posture detection model to determine whether the preset posture detection model outputs the packing posture existing in the human body image, the bag type where the packing posture occurs, and the occurrence time T1 of the packing posture. If yes, proceed to step 609; otherwise, it means that the packing posture is not detected, and the process can return to step 601.

[0107] Step 606: Input the human image into the preset behavior detection model to determine whether the preset behavior detection model outputs the hands of the human object in the human image and the target object held by the hands. If yes, proceed to step 607; otherwise, return to step 601.

[0108] Step 607: Determine whether the distance between the position of the target object and the hand position of the human object is less than a preset threshold. If so, proceed to step 608. Otherwise, it means that the behavior detection model has a misjudgment, and you can return to step 601.

[0109] Step 608: Determine that the human subject has performed a behavior of holding an object, and determine the time T2 when the behavior of holding an object occurs. Then proceed to step 610.

[0110] Step 609: Determine whether the packing posture is the preset posture corresponding to the target bag information. If so, proceed to step 610. Otherwise, return to step 601 and re-test.

[0111] Step 610: Determine whether the occurrence time T2 of the holding item behavior is before the occurrence time T1 of the packing gesture. If so, proceed to step 611; otherwise, return to step 601 for re-testing.

[0112] Step 611: Determine that a human object has performed a packing behavior in a designated area, and issue a prompt message.

[0113] It should be noted that the execution order of the above step 609 and step 610 is only an example. In an actual scenario, step 610 may be executed first, and then step 609 may be executed if the judgment result of step 610 is yes. Alternatively, step 609 and step 610 may be executed simultaneously, and then step 611 may be entered if the judgment results of both are yes. This embodiment does not limit this.

[0114] For details of each step of the above method, please refer to the relevant description of the above embodiment, which will not be repeated here.

[0115] Please see Figure 7 , which is a flow chart of a supermarket anti-theft detection method according to an embodiment of the present application, the method can be Figure 1 The electronic device 1 shown is used to perform and can be applied to Figure 2-Figure 3 In the supermarket monitoring application scenario shown in , by combining the bagging posture, the behavior of holding items, and the preset posture corresponding to the target bag information, it is possible to accurately determine whether the human object has bagging behavior, thereby reducing the possibility of misjudgment and improving the recognition efficiency of bagging behavior. This embodiment takes the behavior detection scenario of bagging of prize-winning goods in a supermarket as an example, and the method includes the following steps: Step 701: Obtain surveillance video frames in a supermarket, where the supermarket may be a large shopping mall, a retail store, or other offline physical stores.

[0116] Step 702: Identify target bag information existing in the human object area in the surveillance video frame.

[0117] Step 703: Detecting the bagging posture and commodity holding behavior of the human object.

[0118] Step 704: If the packing posture is the preset posture corresponding to the target bag information, and the human subject has held the commodity before the packing posture occurs, it is determined that the human subject has packed the commodity in the supermarket, and an anti-theft prompt message is issued.

[0119] The solution of the embodiment of the present application uses the existing surveillance cameras in the supermarket and adds computing power to detect packaging behavior and discover suspicious packaging behavior. The implementation is simple with small changes to the site. For the whole supermarket loss prevention field, it can realize the autonomous discovery of packaging risks. By combining package detection, package type recognition, packaging posture recognition and handheld product recognition, comprehensive decision-making on packaging behavior is achieved, with the characteristic of high recognition accuracy. By matching different package types with different packaging postures, training a multi-category packaging posture recognition model, and matching package types and packaging postures, the alarm accuracy is improved.

[0120] Although in the related art, packaging detection can be achieved in unmanned stores by identifying the goods taken by customers and combining them with the checked-out goods. However, such solutions often rely on a large number of hardware devices, including densely arranged cameras, deep-reading sensors, weighing shelves, etc., and require huge computing power, high deployment costs, and cannot be applied on a large scale. In comparison, the solution of the embodiment of the present application aims to discover the loss of goods through packaging behavior, and then combine it with manual confirmation and take measures to further improve the efficiency of loss prevention. It can be achieved by using existing surveillance cameras, and the cost is relatively lower.

[0121] Compared with the method based on picking and placing goods identification, the packaging behavior identification risk link of the present application is short, and can rely solely on the problem of human behavior identification under a single camera, which is relatively simple to implement.

[0122] Compared with the acoustic magnetic anti-theft door, in actual scenarios, only a small number of commodities are affixed with magnetic labels, and they can be removed. The solution of the embodiment of the present application can detect the packaging behavior of all commodities through packaging behavior recognition, which is more efficient.

[0123] For details of each step of the above method, please refer to the relevant description of the above embodiment, which will not be repeated here.

[0124] Please see Figure 8 , which is a behavior information recognition device 800 of an embodiment of the present application, and the device can be applied to Figure 1 The electronic device 1 shown can be applied to Figure 2-Figure 3 In the supermarket monitoring application scenario shown in , by combining the bagging posture, the behavior of holding items and the preset posture corresponding to the target bag information, it is possible to accurately determine whether the human object has bagging behavior, reduce the possibility of misjudgment, and improve the recognition efficiency of bagging behavior. The device includes: an acquisition module 801, an identification module 802, a detection module 803 and a determination module 804, and the functional principles of each module are as follows: The acquisition module 801 is used to acquire surveillance video frames in a specified area.

[0125] The identification module 802 is used to identify the target bag information existing in the human object area in the monitoring video frame.

[0126] The detection module 803 is used to detect the bagging posture and the object holding behavior of the human object.

[0127] The determination module 804 is used to determine that the human object has performed a packing behavior in a specified area and issue a prompt message if the packing posture is a preset posture corresponding to the target bag information and the human object has held an item before the packing posture occurs.

[0128] In one embodiment, the recognition module 802 is used to recognize human body trajectory information in the surveillance video frame. A human body image is captured from the surveillance video frame according to the human body trajectory information. The human body image is input into a preset bag detection model so that the preset bag detection model outputs target bag information present in the human body image, and the target bag information includes bag location and / or bag type.

[0129] In one embodiment, the method further includes: a first training module, which is used to obtain a bag sample image set before inputting the human body image into the preset bag detection model, wherein the locations and types of different bag objects are marked in the bag sample image, and train the preset detector using the bag sample image set to obtain the preset bag detection model.

[0130] In one embodiment, the detection module 803 is used to identify human body trajectory information in the surveillance video frame. According to the human body trajectory information, a human body image is captured from the surveillance video frame. The human body image is input into a preset posture detection model, so that the preset posture detection model outputs the packing posture existing in the human body image, the type of bag in which the packing posture occurs, and the occurrence time of the packing posture.

[0131] In one embodiment, the method further includes: a second training module for obtaining a set of packaging sample images before inputting the human body image into the preset posture detection model, wherein a plurality of packaging postures and non-packaging postures corresponding to different target bag information are marked in the packaging sample images. The preset detector is trained using the packaging sample image set to obtain a preset posture detection model.

[0132] In one embodiment, the detection module 803 is used to identify human trajectory information in the monitoring video frame. A human image is captured from the monitoring video frame according to the human trajectory information. The human image is input into a preset behavior detection model so that the preset behavior detection model outputs the hand of the human object and the target object held by the hand in the human image. If the distance between the position of the target object and the position of the hand of the human object is less than a preset threshold, it is determined that the human object has a hand-held object behavior, and the occurrence time of the hand-held object behavior is determined.

[0133] In one embodiment, the method further includes: a third training module for obtaining a set of sample images of handheld items before inputting the human body image into the preset behavior detection model, wherein the handheld item sample images are marked with items of different appearance features. The handheld item sample image set is used to train the preset detector to obtain the preset behavior detection model.

[0134] In one embodiment, the bag types include one or more of a backpack, a shoulder bag with the bag located above the waist, a shoulder bag with the bag located below the waist, a forearm bag, and a handbag.

[0135] For a detailed description of the above-mentioned behavior information recognition device 800, please refer to the description of the relevant method steps in the above-mentioned embodiment. Its implementation principle and technical effect are similar, and will not be repeated here in this embodiment.

[0136] Fig. 9 The following is a schematic diagram of the structure of a cloud device 90 provided in an exemplary embodiment of the present application. The cloud device 90 can be used to run the method provided in any of the above embodiments. Fig. 9 As shown, the cloud device 90 may include: a memory 904 and at least one processor 905, Fig. 9 A processor is taken as an example.

[0137] The memory 904 is used to store computer programs and can be configured to store various other data to support operations on the cloud device 90. The memory 904 can be an object storage service (OSS).

[0138] The memory 904 may be implemented by any type of volatile or nonvolatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0139] The processor 905 is coupled to the memory 904 and is used to execute the computer program in the memory 904 to implement the solution provided by any of the above method embodiments. The specific functions and technical effects that can be achieved are not repeated here.

[0140] Furthermore, if Fig. 9 The cloud device also includes: a firewall 901, a load balancer 902, a communication component 906, a power supply component 903 and other components. Fig. 9 Only some components are shown schematically, which does not mean that the cloud device only includes Fig. 9 Components shown.

[0141] In one embodiment, the above Fig. 9 The communication component 906 in the communication component 906 is configured to facilitate wired or wireless communication between the device where the communication component 906 is located and other devices. The device where the communication component 906 is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, LTE (Long Term Evolution, Long Term Evolution, referred to as LTE), 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component 906 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 906 also includes a near field communication (Near Field Communication, referred to as NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (Radio Frequency Identification, referred to as RFID) technology, Infrared Data Association (Infrared Data Association, referred to as IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (Bluetooth, referred to as BT) technology and other technologies.

[0142] In one embodiment, the above Fig. 9 The power supply component 903 provides power to various components of the device where the power supply component 903 is located. The power supply component 903 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply component is located.

[0143] An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the method of any of the aforementioned embodiments is implemented.

[0144] An embodiment of the present application also provides a computer program product, including a computer program, which implements the method of any of the aforementioned embodiments when executed by a processor.

[0145] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0146] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the methods of various embodiments of the present application.

[0147] It should be understood that the above-mentioned processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The memory may include a high-speed RAM (Random Access Memory) memory, and may also include non-volatile storage NVM (Nonvolatile memory, NVM for short), such as at least one disk storage, and can also be a USB flash drive, a mobile hard disk, a read-only memory, a disk or an optical disk, etc.

[0148] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0149] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0150] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, clothing or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, clothing or device. In the absence of more restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, clothing or device including the element.

[0151] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0152] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0153] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of user data and other information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0154] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for identifying behavior information, characterized in that: The method comprises: Get surveillance video frames in the specified area; Identify target bag information existing in the human object area in the surveillance video frame; Detecting the bagging posture and the object holding behavior of the human subject; If the packing posture is a preset posture corresponding to the target bag information, and the human subject has held an item before the packing posture occurs, it is determined that the human subject has performed a packing behavior in the designated area, and a prompt message is issued.

2. The method according to claim 1, characterized in that: The identifying target bag information existing in the human body object area in the surveillance video frame includes: Identifying human body trajectory information in the surveillance video frame; Intercepting a human body image from the monitoring video frame according to the human body trajectory information; The human body image is input into a preset bag detection model so that the preset bag detection model outputs target bag information existing in the human body image, wherein the target bag information includes a bag position and / or a bag type.

3. The method according to claim 2, characterized in that Before inputting the human body image into the preset bag detection model, the method further includes: Acquire a bag sample image set, wherein the bag sample images are marked with positions and types of different bag objects; The preset detector is trained using the bag sample image set to obtain the preset bag detection model.

4. The method according to claim 1, characterized in that: The detecting of the packing posture of the human object comprises: Identifying human body trajectory information in the surveillance video frame; Intercepting a human body image from the monitoring video frame according to the human body trajectory information; The human body image is input into a preset posture detection model, so that the preset posture detection model outputs the packing posture existing in the human body image, the type of bag in which the packing posture occurs, and the occurrence time of the packing posture.

5. The method according to claim 4, characterized in that Before inputting the human body image into a preset posture detection model, the method further includes: Acquire a set of packaging sample images, wherein the packaging sample images are marked with a plurality of packaging postures and non-packaging postures corresponding to different target bag information; The packaged sample image set is used to train a preset detector to obtain the preset posture detection model.

6. The method according to claim 1, characterized in that Detecting the human subject's behavior of holding an object, including: Identifying human body trajectory information in the surveillance video frame; Intercepting a human body image from the monitoring video frame according to the human body trajectory information; Inputting the human body image into a preset behavior detection model so that the preset behavior detection model outputs the hands of the human object in the human body image and the target object held by the hands; If the distance between the position of the target object and the hand position of the human subject is less than a preset threshold, it is determined that the human subject has performed the hand-holding object behavior, and the occurrence time of the hand-holding object behavior is determined.

7. The method according to claim 6, characterized in that Before inputting the human body image into a preset behavior detection model, the method further includes: Acquire a set of sample images of handheld objects, wherein objects with different appearance features are marked in the sample images of handheld objects; The preset detector is trained using the handheld object sample image set to obtain the preset behavior detection model.

8. The method according to any one of claims 2 to 5, characterized in that: The bag types include one or more of a backpack, a shoulder bag with the bag located above the waist, a shoulder bag located below the waist, a forearm bag and a handbag.

9. A supermarket anti-theft detection method, characterized in that: include: Get surveillance video frames in the supermarket; Identify target bag information existing in the human object area in the surveillance video frame; Detecting the packing posture and commodity holding behavior of the human subject; If the packing posture is a preset posture corresponding to the target bag information, and the human subject has held the commodity before the packing posture occurs, it is determined that the human subject has packed the commodity in the supermarket, and an anti-theft prompt message is issued.

10. A system for identifying behavioral information, characterized in that: include: A bag detection module is used to obtain surveillance video frames in a specified area and identify target bag information existing in the human object area in the surveillance video frames; A hand-held object detection module, used to detect hand-held object behavior of the human subject; A packing posture detection module, used to detect the packing posture of the human object; The packing behavior decision module is respectively connected to the bag detection module, the handheld item detection module and the packing posture detection module, and is used to determine that the human object has a packing behavior in the specified area, and issue a prompt message if the packing posture is a preset posture corresponding to the target bag information and the human object has a handheld item behavior before the packing posture occurs.

11. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the electronic device to perform the method according to any one of claims 1 to 9.

12. A cloud device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the cloud device to execute the method described in any one of claims 1-9.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the method according to any one of claims 1 to 9 is implemented.

14. A computer program product, characterized in that The method comprises a computer program, which implements the method according to any one of claims 1 to 9 when the computer program is executed by a processor.

Images