Recognition method of left article and related device

Abstract

The invention discloses a method for identifying a left article and a related device, and relates to the field of artificial intelligence or finance. A customer takes out an article through a hand, so that the article should be taken out by the hand if the article contour of the article is closest to the hand contour of the hand, and the article also should belong to an owner of the hand. In this way, the corresponding relation among the hand contour, the article contour and the whole body contour can be constructed, whether the article contour of the article in the video image corresponds to the whole body contour of a person or not is further determined, and if not, it is considered that the article belongs to the left article; if so, the article does not belong to the left article. And if only the article contour of the article is in the video image and the whole body contour of the person is not in the video image, considering that the article belongs to the left article. Therefore, the method for automatically and quickly detecting the left articles is realized.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method for identifying leftover items and related devices. Background Art

[0002] After handling business at a bank branch, customers may leave behind important items, such as passbooks, bank cards, ID cards and seals, electronic devices such as mobile phones, or cash. If these items are lost, customers may face security risks and financial losses.

[0003] In the related art, staff manually check the video collected by the camera to check where there are items left behind in the bank branch. The manual inspection method is labor-intensive and time-consuming when there are too many customers and insufficient staff. The inspection efficiency is low, which increases the risk of lost items. Summary of the invention

[0004] In view of the above problems, the present application provides a method for identifying left-behind items and a related device to realize a method for automatically detecting left-behind items, so as to improve inspection efficiency and save manpower. The specific scheme is as follows:

[0005] The first aspect of the present application provides a method for identifying a left-behind item, comprising:

[0006] Inputting a video image collected in real time for a set area into a pre-built contour recognition model, and obtaining contour information of an object located in the video image and a region position located in the set area of ​​the video image through the contour recognition model, wherein the contour information of the object includes a whole body contour of the object and a hand contour of the object, and the object is a person;

[0007] Determining that the object has entered the set area based on contour information of the object corresponding to the plurality of frames of video images and area positions of the set area corresponding to the plurality of frames of video images;

[0008] For each frame of video image acquired in real time in the set area, based on the contour information of the object located in the video image, determining the hand contour included in the whole body contour of the object and closest to the whole body contour of the object as the hand contour of the object;

[0009] Inputting the video image collected in real time for the set area into a pre-built universal object recognition model, and obtaining the object contour of the object located in the video image through the universal object recognition model;

[0010] For each frame of video image acquired in real time in the set area, if the contour position of the object contour of the first object in the objects in the video image is the same as the contour position of the pre-stored target existing object contour, search for the first identification ID corresponding to the contour position of the target existing object contour from the preset correspondence between the contour position of the existing object contour and the identification ID;

[0011] For each frame of video image collected in real time in the set area, if there is no full-body outline with the first identification ID in the video image and there is an outline of the target existing item with the first identification ID, determining that the first item is a left-behind item;

[0012] For each frame of video image acquired in real time in the set area, if the contour position of the contour of a second object among the objects in the video image is different from the contour position of the pre-stored existing object contour, the hand contour closest to the contour of the second object in the video image, the whole body contour to which the hand contour belongs, and the contour of the second object are set to the same second identification ID;

[0013] Based on the contour information of the objects respectively corresponding to the multiple frames of the video images and the area positions of the set areas respectively corresponding to the multiple frames of the video images, determining that all the objects leave the set area;

[0014] The video image collected in real time for the set area is input into the universal object recognition model, and the object contour of the object located in the video image is obtained through the universal object recognition model; if the object contour exists in the video image, it is determined that the object corresponding to the object contour is a left-behind object.

[0015] In a possible implementation, the method further includes:

[0016] For any of the video images, if it is determined that there is a leftover object in the video image, intercepting an image of the outline of the object in the video image to obtain a intercepted image;

[0017] Inputting the captured image into a pre-built specific object recognition model, and obtaining the object type of the object corresponding to the object outline through the specific object recognition model;

[0018] If the item type is a preset type, a warning operation is performed.

[0019] In a possible implementation, the method further includes:

[0020] Determine a first moving trend of the whole body contour with the first identification ID based on contour positions of the whole body contour with the first identification ID respectively corresponding to a plurality of frames of video images whose acquisition time is earlier than and closest to the target acquisition time;

[0021] Based on the first movement trend, determining whether the video image corresponding to the target acquisition moment has a full body outline of the first identification ID;

[0022] If the video image corresponding to the target acquisition moment does not include a full-body contour having the first identification ID, it is determined that the full-body contour having the first identification ID does not exist in the video image corresponding to the target acquisition moment.

[0023] In a possible implementation, after the step of setting the hand contour closest to the object contour of the second object in the video image, the whole body contour to which the hand contour belongs, and the object contour of the second object as the same second identification ID, the method further includes:

[0024] Based on the contour information of the objects respectively corresponding to the plurality of frames of video images collected in real time in the set area, the contour position of the object contour having the second identification ID is tracked;

[0025] The corresponding relationship between the second identification ID and the contour position of the object contour having the second identification ID is updated in real time.

[0026] In a possible implementation, the step of determining that the object has entered the set area based on contour information of the object corresponding to the multiple frames of the video images and area positions of the set areas corresponding to the multiple frames of the video images includes:

[0027] For each of the objects, based on the whole body contour of the object corresponding to the multiple frames of the video images and the area positions of the set areas corresponding to the multiple frames of the video images, a second movement trend of the whole body contour of the object is obtained;

[0028] For each of the objects, if the second movement trend of the object indicates that the object needs to enter the set area, it is determined whether the object has entered the set area based on the positional relationship between the whole body contour of the object corresponding to multiple frames of the video images and the area position of the set area.

[0029] In a possible implementation, the step of determining whether the object has entered the set area based on the positional relationship between the whole body contour of the object corresponding to the multiple frames of the video images and the area position of the set area includes:

[0030] For each frame of the video image, obtaining a positional relationship between the whole body contour of the object in the video image and the area position of the set area in the video image to obtain the positional relationship corresponding to the video image;

[0031] Sorting the positional relationships respectively corresponding to the multiple frames of the video images from early to late according to the acquisition time of the multiple frames of the video images to obtain a sorting result;

[0032] If the ranking result indicates that the number of pixel points at the area positions of the left and right sides of the whole body contour of the object entering the set area increases, it is determined that the object has entered the set area;

[0033] If the sorting result indicates that the number of pixel points at the area positions of the left and right sides of the whole body contour of the object entering the set area are both decreasing, it is determined that the object has left the set area.

[0034] A second aspect of the present application provides a device for identifying a left-behind object, comprising:

[0035] A first acquisition module is used to input a video image collected in real time for a set area into a pre-built contour recognition model, and obtain contour information of an object located in the video image and a region position located in the set area of ​​the video image through the contour recognition model, wherein the contour information of the object includes a whole body contour of the object and a hand contour of the object, and the object is a person;

[0036] A first determination module is used to determine that the object has entered the set area based on the contour information of the object corresponding to the multiple frames of the video images and the area position of the set area corresponding to the multiple frames of the video images;

[0037] A second determination module is configured to determine, for each frame of video image acquired in real time in the set area, based on the contour information of the object located in the video image, that the hand contour included in the whole body contour of the object and closest to the whole body contour of the object is the hand contour of the object;

[0038] A second acquisition module is used to input the video image collected in real time for the set area into a pre-built universal object recognition model, and obtain the object contour of the object located in the video image through the universal object recognition model;

[0039] a search module, configured to search, for each frame of video image acquired in real time in the set area, a first identification ID corresponding to the outline position of the target existing object outline from a preset correspondence between the outline position of the existing object outline and the identification ID, if the outline position of the first object in the objects in the video image is the same as the outline position of the pre-stored target existing object outline;

[0040] A third determination module is configured to determine, for each frame of video image captured in real time in the set area, if there is no full-body outline with the first identification ID in the video image and there is an outline of the target existing object with the first identification ID, determine that the first object is a left-behind object;

[0041] A setting module, for setting, for each frame of video image acquired in real time in the set area, the hand contour closest to the contour of the second object in the video image, the whole body contour to which the hand contour belongs, and the contour of the second object to the same second identification ID if the contour position of the contour of the second object in the objects in the video image is different from the contour position of the pre-stored existing object contour;

[0042] A fourth determination module, configured to determine that all the objects leave the set area based on the contour information of the objects respectively corresponding to the multiple frames of the video images and the area positions of the set areas respectively corresponding to the multiple frames of the video images;

[0043] The third acquisition module is used to input the video image collected in real time for the set area into the universal object recognition model, and obtain the object contour of the object located in the video image through the universal object recognition model; if there is an object contour in the video image, determine that the object corresponding to the object contour is a left-behind object.

[0044] A fourth aspect of the present application provides an edge computing box, comprising at least one processor and a memory connected to the processor, wherein:

[0045] The memory is used to store computer programs;

[0046] The processor is used to execute the computer program so that the edge computing box can implement the method for identifying left-behind items of the above-mentioned first aspect or any implementation method of the first aspect.

[0047] In a fifth aspect, the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an edge computing box, the edge computing box can implement the method for identifying abandoned items according to the first aspect or any implementation of the first aspect.

[0048] By means of the above technical solution, the present application provides a method for identifying leftover items, which determines whether an object enters the set area through a video image collected in real time for the set area. If there is an object, it means that a customer needs to enter the set area to handle business. In the process of handling business, the customer may take out some items. The video image may include one or more objects, so it is necessary to determine which object the item belongs to. It can be understood that the customer takes out the item by hand, so the item contour of the item is closest to the hand contour of which hand, and the item should be taken out by this hand, so the item should also belong to the owner of this hand. Based on this, if the contour position of the item contour of the second item in the item in the video image is different from the contour position of the pre-stored existing item contour, the hand contour closest to the item contour of the second item in the video image, the whole body contour to which the hand contour belongs, and the item contour of the second item are set to the same second identification ID. If the contour position of the item contour of the first item in the item is the same as the contour position of the pre-stored target existing item contour, the first identification ID corresponding to the contour position of the target existing item contour is found from the correspondence between the preset contour position of the existing item contour and the identification ID. For each frame of video image captured in real time in the set area, if there is no full-body outline with the first identification ID in the video image and there is an existing object outline of the target with the first identification ID, it means that the object to which the first object belongs has left the set area, and the first object is determined to be a left-behind object. If all objects have left the set area, the video image captured in real time for the set area is input into a pre-built universal object recognition model, and the object outline of the object in the video image is obtained through the universal object recognition model; if there is an object outline in the video image, it is determined that the object corresponding to the object outline is a left-behind object. Thus, an automatic and rapid detection method for left-behind objects is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0050] Figure 1 A schematic diagram of a system architecture provided for this application;

[0051] Figure 2 A schematic diagram of the structure of a server 200 provided in this application;

[0052] Figure 3 A schematic diagram of a flow chart of a method for identifying a left-behind item provided in an embodiment of the present application;

[0053] Figure 4 A schematic diagram of the positional relationship between the whole body contour and the hand contour provided in an embodiment of the present application;

[0054] Figure 5 A schematic diagram of a whole body outline and a set area of ​​an object provided in an embodiment of the present application;

[0055] Figure 6 A schematic diagram of a whole body outline and a set area of ​​an object provided in an embodiment of the present application;

[0056] Figure 7 A schematic diagram of the structure of a device for identifying left-behind items provided in an embodiment of the present application;

[0057] Figure 8 A schematic diagram of the structure of an edge computing box provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0059] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0060] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0061] It should be noted that the user information involved in this application (including but not limited to full body outline and hand outline) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0062] See also Figure 1 , Figure 1 A schematic diagram of a system architecture is shown. The system may include an edge computing box 100, a server 200, and multiple cameras 300.

[0063] The server 200 may include one or more servers ( Figure 1 A server is included as an example for explanation).

[0064] The camera 300 is used to capture still images or videos. The subject (object or organism) generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then transmitted to the ISP for conversion into a digital image signal.

[0065] Exemplarily, multiple cameras 300 are deployed at a bank branch. It is understandable that different cameras 300 collect video images in real time for different set areas of the bank branch. The following example is used to illustrate.

[0066] For example, each counter window of a bank branch corresponds to a set area, and people who enter the set area should need to handle business at the counter window.

[0067] For example, each ATM (Automated Teller Machine) in a bank branch corresponds to a set area, and people who enter the set area should need to handle business through the ATM.

[0068] For example, multiple ATMs correspond to a set area, and people entering the set area should need to handle business through one or more of the multiple ATMs.

[0069] In summary, the setting area can be determined based on actual conditions and is not limited here.

[0070] Exemplarily, the camera 300 may be connected to an NVR (Network Video Recorder), which receives, stores and manages the video images captured by the camera.

[0071] Exemplarily, the edge computing box 100 can obtain corresponding real-time video images from the video streams transmitted to the NVR by multiple cameras 300. Thus, the method for identifying leftover items provided in the embodiment of the present application can be executed to issue an early warning after determining that there are leftover items in the set area.

[0072] Exemplarily, the edge computing box 100 may include a storage unit, a power distribution unit, a computing unit, and the like.

[0073] Exemplarily, the edge computing box 100 can send the warning results and the video images containing the abandoned items to the server 200.

[0074] Next describe Figure 1 The product form of the server 200;

[0075] Figure 2 A structural diagram of a server 200 is provided, such as Figure 2 As shown, the server 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate with each other via the bus 201.

[0076] The bus 201 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0077] The processor 202 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0078] The memory 204 may include a volatile memory, such as a random access memory (RAM). The memory 204 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0079] It should be understood that the above-mentioned edge computing box 100 and server 200 can be centralized or distributed devices, and the processors in the above-mentioned edge computing box 100 and server 200 (such as processor 170 and processor 202) can be hardware circuits (such as application specific integrated circuit (ASIC), field-programmable gate array (FPGA), general-purpose processor, digital signal processor (DSP), microprocessor or microcontroller, etc.), or a combination of these hardware circuits. For example, the processor can be a hardware system with the function of executing instructions, such as CPU, DSP, etc., or a hardware system without the function of executing instructions, such as ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without the function of executing instructions and hardware systems with the function of executing instructions.

[0080] Reference Figure 3 , Figure 3 A flowchart of a method for identifying a leftover item provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, a method for identifying left-behind items provided in an embodiment of the present application may include steps S301 to S309, and these steps are described in detail below.

[0081] The set areas mentioned in step S301 to step S309 in the embodiment of the present application are the same set area. Exemplarily, the camera that collects video images in real time for the same set area is the same camera. As time passes, the camera collects video images of the set area in real time at different times. Therefore, the video images mentioned in the embodiment of the present application are all collected by the camera in real time.

[0082] Step S301: inputting a video image captured in real time for a set area into a pre-built contour recognition model, and obtaining contour information of an object located in the video image and a region position located in the set area of ​​the video image through the contour recognition model.

[0083] Exemplarily, the contour recognition model may be in working state all the time.

[0084] The contour information of the object includes the whole body contour of the object and the hand contour of the object, and the object is a person.

[0085] For example, one or more objects may exist in a video image.

[0086] It can be understood that the whole body outline of the object refers to the outline including the head, torso, limbs, hands and feet of the object. The hand outline of the object refers to the outline of the hand of the object.

[0087] Exemplarily, the contour recognition model is obtained by training with a sample video image as input, and with the annotated contour information of an object in the sample video image and the position of an annotated area in a set area of ​​the sample video image as training targets.

[0088] Exemplarily, the shapes of different set areas can be the same or different. Exemplarily, the shape of the set area can be an irregular polygon or a regular polygon.

[0089] Exemplarily, the set area is a virtual area, and the subject may not be able to see the set area at the bank branch.

[0090] Exemplarily, the set area is an actual area that the subject can see at a bank branch.

[0091] Exemplarily, the set area is a closed area.

[0092] Step S302: determining that the object has entered the set area based on the contour information of the object corresponding to the multiple frames of the video images and the area positions of the set areas corresponding to the multiple frames of the video images.

[0093] Exemplarily, for each object, the movement trend of the object's whole body contour can be obtained based on the whole body contour of the object corresponding to multiple frames of video images and the area position of the set area corresponding to multiple frames of video images, thereby determining whether the object has entered the set area.

[0094] Exemplarily, in order to improve the processing speed, the multiple frames of video images in step S302 can be multiple frames of discontinuous video images extracted from the video. Exemplarily, multiple frames of video images are extracted from the video in a frame-per-second manner, such as 30 frames of video images can be collected in 1 second, and 5 frames of video images are extracted at intervals. This method can reduce system consumption and improve system computing efficiency.

[0095] Step S303: For each frame of video image acquired in real time in the set area, based on the contour information of the object located in the video image, determine that the hand contour included in the whole body contour of the object and closest to the whole body contour of the object is the hand contour of the object.

[0096] For example, as time goes by, some customers have completed their business and left the set area, while some customers have just entered the set area to complete their business, so the number of objects contained in the video images captured at different times may be the same or different; the objects contained in the video images captured at different times may be the same objects or different objects.

[0097] The following example illustrates the positional relationship between the object's whole body outline and hand outline.

[0098] like Figure 4 , which is a schematic diagram of the positional relationship between the whole body contour and the hand contour provided in an embodiment of the present application.

[0099] It is understandable that if there are multiple objects in the set area, the camera has a certain shooting angle when capturing video images, which may cause the hand contours of different objects to be close, such as Figure 4 In the figure, the whole body outline of the object 41 is marked with a dotted line, and the hand outline 411 of the object 41 is marked with a dotted line. Figure 4 It can be seen that there are four hand contours located within the whole body contour of object 41 , two of which are the hand contours of object 41 itself, and two are the hand contours of object 42 .

[0100] from Figure 4 It can be seen that the hand contour 411 is located in the whole body contour of the object 41 and is closest to the whole body contour of the object 41 , so the hand contour 411 is the hand contour of the object 41 .

[0101] Figure 4 The overlapping part of the hand contours of object 41 and object 42 is enlarged, which can be seen in Figure 4 A magnified image of; Figure 4 The thick solid line in the middle is the hand outline of object 41. Figure 4 The thin solid line is the hand outline of the object 42. It can be understood that an object has only two hands, so Figure 4 In the enlarged image shown, only one hand contour belongs to object 41 . At this time, the distance between the hand contour and the whole body contour of object 41 can be calculated to determine that the hand contour closest to the whole body contour of object 41 belongs to object 41 .

[0102] It is understandable that, in the process of training the contour recognition model, the more accurately the contour recognition model is trained, the more accurate the whole body contour of the object is, and thus the probability of including the hand contours of other objects is lower.

[0103] Step S304: inputting the video image collected in real time for the set area into a pre-built universal object recognition model, and obtaining the object contour of the object located in the video image through the universal object recognition model.

[0104] It is understandable that after determining that at least one object has entered the set area, the universal object recognition model will be activated. After the universal object recognition model is activated, the real-time collected video image is input into both the contour recognition model and the universal object recognition model.

[0105] Exemplarily, the general object recognition model is trained by taking sample video images as input and annotated object contours of the sample video images as training targets.

[0106] For example, the number of objects in one video image may be 0, 1, or more, so the number of object contours in one video image may be 0, 1, or more.

[0107] Step S305: For each frame of video image captured in real time in the set area, if the contour position of the object contour of a first object existing in the objects in the video image is the same as the contour position of the pre-stored target existing object contour, search for the first identification ID corresponding to the contour position of the target existing object contour from the correspondence between the preset contour position of the existing object contour and the identification ID.

[0108] Step S306: For each frame of video image collected in real time in the set area, if there is no full-body outline with the first identification ID in the video image and there is an outline of the target existing object with the first identification ID, determine that the first object is a left-behind object.

[0109] Step S307: For each frame of video image captured in real time in the set area, if the contour position of the contour of a second object among the objects in the video image is different from the contour position of the pre-stored existing object contour, the hand contour closest to the object contour of the second object in the video image, the whole body contour to which the hand contour belongs, and the object contour of the second object are set to the same second identification ID.

[0110] It is understandable that after the same video image is input into the contour recognition model and the universal object recognition model, the whole body contour and hand contour belonging to the same object can be obtained through the contour recognition model, and the object contour can be obtained through the universal object recognition model.

[0111] It is understandable that before an object enters the set area, other objects may have entered the set area, and steps S301 to S309 provided in the embodiment of the present application will also be executed for other objects. Therefore, an identification ID will also be set for other objects, and the corresponding relationship between the identification ID of the item and the outline position of the item outline of the item will be stored. If step S305 determines that the outline position of the item outline of the first item in the item obtained in step S304 is the same as the outline position of the pre-stored target item outline, there is no need to set the identification ID of the item outline of the first item again, and it can be directly searched. If there is a full-body outline with a first identification ID in the video image and there is an existing item outline of the target with the first identification ID, it means that the object to which the first item belongs has not left the set area, and the first item is not a lost item at this time. If there is no full-body outline with a first identification ID in the video image and there is an existing item outline of the target with the first identification ID, it means that the object to which the first item belongs has left the set area, and the first item is a left-behind item. If step S305 determines that the contour position of the second object among the objects obtained in step S304 is different from the contour position of the pre-stored existing object contour, it means that a new object has appeared in the set area and should be an object that has just entered the set area. Therefore, step S307 needs to be executed.

[0112] It can be understood that the object contour of the same object in multiple frames of video images can be determined based on the video tracking technology.

[0113] Illustratively, the number of the second items may be one or more.

[0114] It is understandable that a video image may include one or more second objects, and a video image may include one or more hand outlines. A video image may include one or more objects, so it is necessary to determine which object the object belongs to. It is understandable that the customer takes out the object by hand, and when the object first appears, the object outline of the object is closest to the hand outline of the hand, and the object should be taken out by this hand, and the object should also belong to the owner of this hand.

[0115] In summary, when the second item appears for the first time, the hand contour closest to the second item should be the hand contour of the object to which the second item belongs. In this way, an association relationship can be established between the object contour of the second item, the hand contour closest to the object contour of the second item, and the whole body contour to which the hand contour belongs. Figure 4 The object outline 412 in FIG. 1 is used as an example for explanation.

[0116] The hand contour closest to the object contour 412 is the hand contour 411, and the hand contour 411 belongs to the object 41. In this way, the correspondence between the object contour 412, the hand contour 411 and the whole body contour of the object 41 can be set, that is, the same identification ID can be set.

[0117] Step S308: determining that all the objects leave the set area based on the contour information of the objects respectively corresponding to the multiple frames of the video images and the area positions of the set areas respectively corresponding to the multiple frames of the video images.

[0118] It is understandable that as long as there is an object in the set area, the universal object recognition model is always in operation. If all objects leave the set area, the universal object recognition model will stop running after running for a set time. It is understandable that the contour recognition model in the embodiment of the present application is always in operation. Therefore, it is possible to monitor in real time whether there are any leftover items in the set area.

[0119] Exemplarily, in order to improve the processing speed, the multiple frames of video images in step S308 can be multiple frames of discontinuous video images extracted from the video. Exemplarily, multiple frames of video images are extracted from the video in a frame-by-frame manner per second, such as 30 frames of video images can be collected in 1 second, and 5 frames of video images are extracted at intervals. This method can reduce system consumption and improve system computing efficiency.

[0120] Step S309: input the video image collected in real time for the set area into the universal object recognition model, and obtain the object contour of the object located in the video image through the universal object recognition model; if there is an object contour in the video image, determine that the object corresponding to the object contour is a leftover item.

[0121] It can be understood that, since all objects have left the set area, as long as the video image includes the outline of an object, it means that there is an object left behind.

[0122] The embodiment of the present application provides a method for identifying leftover items, which determines whether an object enters the set area through a video image collected in real time for a set area. If there is an object, it means that a customer needs to enter the set area to handle business. In the process of handling business, the customer may take out some items. The video image may include one or more objects, so it is necessary to determine which object the item belongs to. It can be understood that the customer takes out the item by hand, so the item outline of the item is closest to the hand outline of which hand, and the item should be taken out by this hand, so the item should also belong to the owner of this hand. Based on this, if the outline position of the item outline of the second item in the item in the video image is different from the outline position of the pre-stored existing item outline, the hand outline closest to the item outline of the second item in the video image, the whole body outline to which the hand outline belongs, and the item outline of the second item are set to the same second identification ID. If the outline position of the item outline of the first item in the item is the same as the outline position of the pre-stored target existing item outline, the first identification ID corresponding to the outline position of the target existing item outline is found from the correspondence between the preset outline position of the existing item outline and the identification ID. For each frame of video image captured in real time in the set area, if there is no full-body outline with the first identification ID in the video image and there is an existing object outline of the target with the first identification ID, it means that the object to which the first object belongs has left the set area, and the first object is determined to be a left-behind object. If all objects have left the set area, the video image captured in real time for the set area is input into a pre-built universal object recognition model, and the object outline of the object in the video image is obtained through the universal object recognition model; if there is an object outline in the video image, it is determined that the object corresponding to the object outline is a left-behind object. Thus, an automatic and rapid detection method for left-behind objects is realized.

[0123] It is understandable that the universal object recognition model can recognize the object outline of any object, such as printed receipts, pens, bank cards, keys, U shields, ID cards, waste paper, etc. There are some leftover items such as waste paper or printed receipts that customers do not pay much attention to, and even if they are lost, they will not cause any impact on customers. Based on this, the embodiment of the present application also provides the following method, which includes the following steps A1 to A3 during implementation.

[0124] Step A1: For any of the video images, if it is determined that there is a leftover object in the video image, an image of the outline of the object in the video image is captured to obtain a captured image.

[0125] Step A2: input the captured image into a pre-built specific object recognition model, and obtain the object type of the object corresponding to the object outline through the specific object recognition model.

[0126] Exemplarily, the specific object recognition model is trained by taking a cropped image of a sample video image as input and taking the labeled object type corresponding to the cropped image as a training target.

[0127] Exemplarily, the captured image is processed by the nearest neighbor interpolation method in the super-resolution image processing technology to obtain a captured image with higher clarity and resolution.

[0128] Step A3: If the item type is a preset type, perform a warning operation.

[0129] Exemplarily, the preset types include but are not limited to: ID card, bank card, seal, U shield, cash, passbook, mobile phone, PAD.

[0130] It is understandable that the preset types are all types of items that customers are concerned about, and if these items are lost, it may cause serious losses to the customers.

[0131] In an optional implementation, if the subject forgets to take an item with him, the distance between the subject and the item will become farther and farther. Since the identification ID of the item and its object is the same, for example, both are the first identification ID, the contour position of the whole body contour with the first identification ID in the video image and the contour position of the object contour with the first identification ID can be determined based on the video tracking technology. Assuming that it is necessary to determine whether the video image captured at the target capture time includes the whole body contour with the first identification, the method includes the following steps B1 to B3.

[0132] Step B1: determining a first moving trend of the whole body contour with the first identification ID based on contour positions of the whole body contour with the first identification ID corresponding to a plurality of video frames whose acquisition times are earlier than and closest to the target acquisition time.

[0133] For example, the whole body contours with the first identification ID are sorted in order from early to late according to the collection time, so that the first movement trend of the whole body contour with the first identification ID can be obtained, for example, the first movement trend is a regional position far away from the set area, or the first movement trend is a regional position close to the set area.

[0134] It can be understood that if the first movement trend is a regional position away from the set area, it means that the purpose of the whole body contour with the first identification ID is to leave the set area; if the first movement trend is a regional position close to the set area, it means that the whole body contour with the first identification ID is to enter the set area.

[0135] Step B2: Based on the first movement trend, determine whether the video image corresponding to the target acquisition moment has a full-body outline of the first identification ID.

[0136] It can be understood that the contour position of the whole body contour having the first identification ID in the video image captured at the target capture moment can be determined based on the first moving area.

[0137] Step B3: If the video image corresponding to the target acquisition moment does not include the full-body contour with the first identification ID, determine that the full-body contour with the first identification ID does not exist in the video image corresponding to the target acquisition moment.

[0138] In an optional implementation, after step S307, the embodiment of the present application may further include the following method, which includes the following steps C1 to C2 during implementation.

[0139] Step C1: based on the contour information of the objects respectively corresponding to the plurality of frames of video images acquired in real time in the set area, the contour position of the object contour having the second identification ID is tracked.

[0140] Step C2: updating the correspondence between the second identification ID and the contour position of the object contour having the second identification ID in real time.

[0141] It is understandable that, as time passes, if a frame of video image captured in real time includes an object contour with the second identification ID, the previously stored correspondence between the second identification ID and the contour position of the object contour with the second identification ID is updated. In this way, it is possible to identify which object contours in the video image are contour positions of existing object contours.

[0142] It is understandable that there are multiple ways to implement step S302. The embodiment of the present application provides but is not limited to the following method, which includes the following steps D11 to D12 during implementation.

[0143] Step D11: for each object, based on the whole body contour of the object corresponding to the multiple frames of video images and the area positions of the set areas corresponding to the multiple frames of video images, a second movement trend of the whole body contour of the object is obtained.

[0144] For example, the whole body contours of the same object are sorted in order from early to late according to the acquisition time, so as to obtain the second movement trend of the object, for example, the second movement trend is the regional position far away from the set area, or the second movement trend is the regional position close to the set area.

[0145] It can be understood that if the second movement trend of the whole body contour of the same object is a regional position away from the set area, it means that the object does not want to enter the set area; if the second movement trend of the whole body contour of the same object is a regional position close to the set area, it means that the object wants to enter the set area, or wants to pass through the set area and enter other areas.

[0146] Step D12: For each of the objects, if the second movement trend of the object indicates that the object needs to enter the set area, determine whether the object has entered the set area based on the positional relationship between the whole body contour of the object corresponding to multiple frames of the video images and the area position of the set area.

[0147] It can be understood that if the object does not want to enter the set area but wants to pass through the set area, generally, the left side or the right side of the object's whole body outline will enter the set area, but the probability of the left and right sides of the object's whole body outline entering the set area at the same time is very low; if the object wants to enter the set area, generally, both the left and right sides of the object's whole body outline will enter the set area.

[0148] The left and right sides of the full body outline of the subject are described below. Figure 5 and Figure 6 , which is a schematic diagram of the whole body contour and set areas of an object provided in an embodiment of the present application.

[0149] like Figure 5 and Figure 6 As shown, Figure 5 and Figure 6 A setting area 51 and an object 52 are shown in FIG. Figure 5 and Figure 6 The whole body outline of the object 52 is indicated by a dotted line.

[0150] For example, the moving direction of the object can be the X-axis, the left side of the X-axis is the left side of the object's body outline, and the right side of the X-axis is the right side of the object's body outline. It can be understood that Figure 5 As shown, if the object 52 passes through the set area 51 to go to another area to handle business, in general, the left or right side of the whole body outline of the object 52 may enter the set area, but the probability that the left and right sides of the whole body outline of the object 52 enter the set area at the same time is small. Figure 6 As shown, if the object 52 needs to enter the set area 51 to handle business, generally, the left and right sides of the whole body outline of the object 52 should enter the set area.

[0151] In summary, if part or all of the left side and part or all of the right side of the object's whole body contour enter the set area, it is determined that the object has entered the set area. If the left side or the right side of the object's whole body contour enters the set area, or if neither the left side nor the right side of the object's whole body contour enters the set area, it is determined that the object has not entered the set area.

[0152] It is understandable that there are multiple ways to implement step D12. The embodiment of the present application provides but is not limited to the following method, which includes the following steps D21 to D24.

[0153] Step D21: for each frame of the video image, obtaining the positional relationship between the whole body contour of the object in the video image and the area position of the set area in the video image, so as to obtain the positional relationship corresponding to the video image.

[0154] Exemplarily, the positional relationship between the object's body contour and the area position of the set area includes: the number of pixel points on the left side of the object's body contour entering the set area and the number of pixel points on the right side of the object's body contour entering the set area.

[0155] In the embodiment of the present application, the cameras used to capture video images for the same set area are the same, so the sizes of the video images captured for the same set area at different times are the same.

[0156] Step D22: sorting the positional relationships respectively corresponding to the multiple frames of the video images from early to late according to the acquisition time of the multiple frames of the video images to obtain a sorting result.

[0157] Step D23: If the sorting result indicates that the number of pixel points at the left and right sides of the whole body contour of the object entering the set area is increasing, it is determined that the object has entered the set area.

[0158] Step D24: If the sorting result indicates that the number of pixel points at the area positions of the left and right sides of the whole body contour of the object entering the set area is decreasing, it is determined that the object has left the set area.

[0159] In the embodiment of the present application, after the same object enters the set area, an identification ID will be assigned to it. After the object leaves the set area and enters the set area again, the identification ID assigned to the object is different from the identification ID previously assigned to the object.

[0160] In the embodiment of the present application, after the same object enters the set area, an identification ID will be assigned to it. After the object leaves the set area and enters the set area again, the identification ID assigned to the object is different from the identification ID previously assigned to the object.

[0161] A method for identifying a left-behind item provided in an embodiment of the present application is introduced above, and a device for executing the method for identifying a left-behind item will be introduced below.

[0162] See also Figure 7 , Figure 7 This is a schematic diagram of the structure of a device for identifying leftover items provided in an embodiment of the present application. Figure 7 As shown, the identification device for the left-behind articles comprises:

[0163] The first acquisition module 701 is used to input the video image collected in real time for the set area into a pre-built contour recognition model, and obtain the contour information of the object located in the video image and the area position located in the set area of ​​the video image through the contour recognition model, wherein the contour information of the object includes the whole body contour of the object and the hand contour of the object, and the object is a person;

[0164] A first determination module 702 is used to determine that the object has entered the set area based on the contour information of the object corresponding to the multiple frames of the video images and the area position of the set area corresponding to the multiple frames of the video images;

[0165] A second determination module 703 is configured to determine, for each frame of video image acquired in real time in the set area, based on the contour information of the object located in the video image, that the hand contour included in the whole body contour of the object and closest to the whole body contour of the object is the hand contour of the object;

[0166] A second acquisition module 704 is used to input the video image collected in real time for the set area into a pre-built universal object recognition model, and obtain the object contour of the object located in the video image through the universal object recognition model;

[0167] A search module 705 is configured to search, for each frame of video image captured in real time in the set area, for a first object in the objects in the video image whose outline position is the same as the outline position of a pre-stored target existing object outline, for a first identification ID corresponding to the outline position of the target existing object outline from a preset correspondence between the outline position of the existing object outline and the identification ID;

[0168] A third determination module 706 is configured to determine, for each frame of video image collected in real time in the set area, if there is no full-body outline with the first identification ID in the video image and there is an outline of the target existing item with the first identification ID, that the first item is a left-behind item;

[0169] A setting module 707 is configured to set, for each frame of video image acquired in real time in the set area, if the contour position of the contour of a second object among the objects in the video image is different from the contour position of the pre-stored existing object contour, the hand contour closest to the contour of the second object in the video image, the whole body contour to which the hand contour belongs, and the contour of the second object to the same second identification ID;

[0170] A fourth determining module 708, configured to determine that all the objects leave the set area based on the contour information of the objects respectively corresponding to the multiple frames of the video images and the area positions of the set areas respectively corresponding to the multiple frames of the video images;

[0171] The third acquisition module 709 is used to input the video image collected in real time for the set area into the universal object recognition model, and obtain the object contour of the object located in the video image through the universal object recognition model; if there is an object contour in the video image, determine that the object corresponding to the object contour is a leftover item.

[0172] In an optional implementation, the method further includes:

[0173] A fourth acquisition module is used for capturing an image of the outline of the object in any of the video images to obtain a captured image if it is determined that the video image contains a leftover object;

[0174] a fifth acquisition module, configured to input the captured image into a pre-built specific object recognition model, and acquire the object type of the object corresponding to the object outline through the specific object recognition model;

[0175] The early warning execution module is used to execute an early warning operation if the item type is a preset type.

[0176] In an optional implementation, the method further includes:

[0177] A fifth determining module, configured to determine a first moving trend of the whole body contour having the first identification ID based on contour positions of the whole body contour having the first identification ID respectively corresponding to a plurality of frames of video images whose acquisition times are earlier than and closest to the target acquisition time;

[0178] a sixth determination module, configured to determine, based on the first movement trend, whether the video image corresponding to the target acquisition moment has a full body outline of the first identification ID;

[0179] The seventh determination module is configured to determine that the full body contour with the first identification ID does not exist in the video image corresponding to the target acquisition moment if the video image corresponding to the target acquisition moment does not include the full body contour with the first identification ID.

[0180] In an optional implementation, the method further includes:

[0181] A tracking module, configured to track the contour position of the contour of the object having the second identification ID based on the contour information of the objects respectively corresponding to the multiple frames of video images collected in real time for the set area;

[0182] An updating module is used to update in real time the corresponding relationship between the second identification ID and the contour position of the object contour having the second identification ID.

[0183] In an optional implementation, the first determining module includes:

[0184] An acquisition unit, configured to acquire, for each of the objects, a second movement trend of the whole body contour of the object based on the whole body contour of the object respectively corresponding to the multiple frames of the video images and the area positions of the set areas respectively corresponding to the multiple frames of the video images;

[0185] A judgment unit is used to judge, for each of the objects, whether the object has entered the set area based on a positional relationship between the whole body contour of the object corresponding to multiple frames of the video images and the area position of the set area if the second movement trend of the object indicates that the object needs to enter the set area.

[0186] In an optional implementation, the judging unit includes:

[0187] A first acquisition subunit is used to acquire, for each frame of the video image, a positional relationship between the whole body contour of the object located in the video image and the area position of the set area located in the video image, so as to obtain the positional relationship corresponding to the video image;

[0188] A sorting subunit, used to sort the positional relationships respectively corresponding to the multiple frames of the video images according to the acquisition time of the multiple frames of the video images from early to late, so as to obtain a sorting result;

[0189] A first determining subunit is configured to determine that the object has entered the set area if the sorting result indicates that the number of pixel points at the area positions of the left and right sides of the whole body contour of the object entering the set area are both increasing.

[0190] The second determining subunit is configured to determine that the object has left the set area if the sorting result indicates that the number of pixel points at the area positions of the left and right sides of the whole body contour of the object entering the set area are both decreasing.

[0191] The third aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an edge computing box, the edge computing box implements the method for identifying abandoned items according to the first aspect or any implementation of the first aspect.

[0192] The present application also provides an edge computing box. Figure 8 As shown, it shows a schematic diagram of the structure of an edge computing box suitable for implementing the embodiment of the present application. The edge computing box in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Figure 8 The edge computing box shown is merely an example and should not bring any limitations to the functionality and scope of use of the embodiments of the present application.

[0193] like Figure 8 As shown, the edge computing box may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 to a random access memory (RAM) 803. When the edge computing box is powered on, various programs and data required for the operation of the edge computing box are also stored in the RAM 803. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0194] Typically, the following devices may be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a memory card, a hard disk, etc.; and communication devices 809. The communication device 809 may allow the edge computing box to communicate wirelessly or wired with other devices to exchange data. Although Figure 8 An edge computing box with various devices is shown, but it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have instead.

[0195] Also provided in an embodiment of the present application is a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an edge computing box, the edge computing box implements any method for identifying left-behind items provided in an embodiment of the present application.

[0196] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When one or more computer programs are executed by the edge computing box, the edge computing box can implement any method for identifying left-behind items provided in the embodiment of the present application.

[0197] It should be noted that the method for identifying a leftover item and the related device provided by the present invention can be used in the field of artificial intelligence or finance. The above is only an example and does not limit the application field of the method for identifying a leftover item and the related device provided by the present invention.

[0198] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0199] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0200] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0201] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A method for identifying leftover items, characterized in that: include: Inputting a video image collected in real time for a set area into a pre-built contour recognition model, and obtaining contour information of an object located in the video image and a region position located in the set area of ​​the video image through the contour recognition model, wherein the contour information of the object includes a whole body contour of the object and a hand contour of the object, and the object is a person; Determining that the object has entered the set area based on contour information of the object corresponding to the plurality of frames of video images and area positions of the set area corresponding to the plurality of frames of video images; For each frame of video image acquired in real time in the set area, based on the contour information of the object located in the video image, determining the hand contour included in the whole body contour of the object and closest to the whole body contour of the object as the hand contour of the object; Inputting the video image collected in real time for the set area into a pre-built universal object recognition model, and obtaining the object contour of the object located in the video image through the universal object recognition model; For each frame of video image acquired in real time in the set area, if the contour position of the object contour of the first object in the objects in the video image is the same as the contour position of the pre-stored target existing object contour, search for the first identification ID corresponding to the contour position of the target existing object contour from the preset correspondence between the contour position of the existing object contour and the identification ID; For each frame of video image collected in real time in the set area, if there is no full-body outline with the first identification ID in the video image and there is an outline of the target existing item with the first identification ID, determining that the first item is a left-behind item; For each frame of video image acquired in real time in the set area, if the contour position of the contour of a second object among the objects in the video image is different from the contour position of the pre-stored existing object contour, the hand contour closest to the contour of the second object in the video image, the whole body contour to which the hand contour belongs, and the contour of the second object are set to the same second identification ID; Based on the contour information of the objects respectively corresponding to the multiple frames of the video images and the area positions of the set areas respectively corresponding to the multiple frames of the video images, determining that all the objects leave the set area; The video image collected in real time for the set area is input into the universal object recognition model, and the object contour of the object located in the video image is obtained through the universal object recognition model; if the object contour exists in the video image, it is determined that the object corresponding to the object contour is a left-behind object.

2. The method for identifying leftover items according to claim 1, characterized in that: Also includes: For any of the video images, if it is determined that there is a leftover object in the video image, intercepting an image of the outline of the object in the video image to obtain a intercepted image; Inputting the captured image into a pre-built specific object recognition model, and obtaining the object type of the object corresponding to the object outline through the specific object recognition model; If the item type is a preset type, a warning operation is performed.

3. The method for identifying leftover items according to claim 1, characterized in that: Also includes: Determine a first moving trend of the whole body contour with the first identification ID based on contour positions of the whole body contour with the first identification ID respectively corresponding to a plurality of frames of video images whose collection times are earlier than the target collection time and closest to the target collection time; Based on the first movement trend, determining whether the video image corresponding to the target acquisition moment has a full body outline of the first identification ID; If the video image corresponding to the target acquisition moment does not include a full-body contour having the first identification ID, it is determined that the full-body contour having the first identification ID does not exist in the video image corresponding to the target acquisition moment.

4. The method for identifying leftover items according to claim 1, characterized in that: After the step of setting the hand contour closest to the object contour of the second object in the video image, the whole body contour to which the hand contour belongs, and the object contour of the second object as the same second identification ID, the step further includes: Based on the contour information of the objects respectively corresponding to the plurality of frames of video images collected in real time in the set area, the contour position of the object contour having the second identification ID is tracked; The corresponding relationship between the second identification ID and the contour position of the object contour having the second identification ID is updated in real time.

5. The method for identifying leftover items according to claim 1, characterized in that: The step of determining that the object has entered the set area based on the contour information of the object corresponding to the multiple frames of the video images and the area positions of the set areas corresponding to the multiple frames of the video images comprises: For each of the objects, based on the whole body contour of the object corresponding to the multiple frames of the video images and the area positions of the set areas corresponding to the multiple frames of the video images, a second movement trend of the whole body contour of the object is obtained; For each of the objects, if the second movement trend of the object indicates that the object needs to enter the set area, it is determined whether the object has entered the set area based on the positional relationship between the whole body contour of the object corresponding to multiple frames of the video images and the area position of the set area.

6. The method for identifying leftover items according to claim 5, characterized in that: The step of judging whether the object has entered the set area based on the positional relationship between the whole body contour of the object corresponding to the multiple frames of the video images and the area position of the set area comprises: For each frame of the video image, obtaining a positional relationship between the whole body contour of the object in the video image and the area position of the set area in the video image to obtain the positional relationship corresponding to the video image; Sorting the positional relationships respectively corresponding to the multiple frames of the video images from early to late according to the acquisition time of the multiple frames of the video images to obtain a sorting result; If the ranking result indicates that the number of pixel points at the area positions of the left and right sides of the whole body contour of the object entering the set area increases, it is determined that the object has entered the set area; If the sorting result indicates that the number of pixel points at the area positions of the left and right sides of the whole body contour of the object entering the set area are both decreasing, it is determined that the object has left the set area.

7. A device for identifying left-behind items, characterized in that: include: A first acquisition module is used to input a video image collected in real time for a set area into a pre-built contour recognition model, and obtain contour information of an object located in the video image and a region position located in the set area of ​​the video image through the contour recognition model, wherein the contour information of the object includes a whole body contour of the object and a hand contour of the object, and the object is a person; A first determination module is used to determine that the object has entered the set area based on the contour information of the object corresponding to the multiple frames of the video images and the area position of the set area corresponding to the multiple frames of the video images; A second determination module is configured to determine, for each frame of video image acquired in real time in the set area, based on the contour information of the object located in the video image, that the hand contour included in the whole body contour of the object and closest to the whole body contour of the object is the hand contour of the object; A second acquisition module is used to input the video image collected in real time for the set area into a pre-built universal object recognition model, and obtain the object contour of the object located in the video image through the universal object recognition model; a search module, configured to search, for each frame of video image acquired in real time in the set area, a first identification ID corresponding to the outline position of the target existing object outline from a preset correspondence between the outline position of the existing object outline and the identification ID, if the outline position of the first object in the objects in the video image is the same as the outline position of the pre-stored target existing object outline; A third determination module is configured to determine, for each frame of video image captured in real time in the set area, if there is no full-body outline with the first identification ID in the video image and there is an outline of the target existing object with the first identification ID, determine that the first object is a left-behind object; A setting module, for setting, for each frame of video image acquired in real time in the set area, the hand contour closest to the contour of the second object in the video image, the whole body contour to which the hand contour belongs, and the contour of the second object to the same second identification ID if the contour position of the contour of the second object in the objects in the video image is different from the contour position of the pre-stored existing object contour; A fourth determination module, configured to determine that all the objects leave the set area based on the contour information of the objects respectively corresponding to the multiple frames of the video images and the area positions of the set areas respectively corresponding to the multiple frames of the video images; The third acquisition module is used to input the video image collected in real time for the set area into the universal object recognition model, and obtain the object contour of the object located in the video image through the universal object recognition model; if there is an object contour in the video image, determine that the object corresponding to the object contour is a left-behind object.

8. A computer program product, characterized in that It includes computer-readable instructions, which, when executed on an edge computing box, enable the edge computing box to implement the method for identifying abandoned items as described in any one of claims 1 to 6.

9. An edge computing box, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the edge computing box can implement the method for identifying left-behind items as described in any one of claims 1 to 6.

10. A computer storage medium, characterized in that: The storage medium carries one or more computer programs, which, when executed by the edge computing box, enable the edge computing box to implement the method for identifying abandoned items as described in any one of claims 1 to 6.

Images