Method and device for determining a behavior of a person, equipment and storage medium

Abstract

Embodiments of the present application provide a method, device and storage medium for determining a stay behavior, and relate to the technical field of image recognition. The method comprises: obtaining a current image collected at a current time for a preset area, the preset area comprising at least one sub-area; if it is determined that the current image comprises an imaging area of at least one stay object, obtaining a target pixel coordinate of each stay object mapped in the current image, and determining a sub-area where a corresponding stay object is located at the current time according to each target pixel coordinate; for each sub-area, determining a stay identifier of the sub-area at the current time indicating whether a stay object exists in the sub-area according to whether a stay object exists in the sub-area; and determining whether a stay behavior exists in the sub-area at the current time according to the stay identifiers of the sub-area at the current time and at least one historical time before the current time. The embodiments of the present application achieve the purpose of accurately identifying whether a stay behavior exists in each sub-area of the preset area.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and more specifically, to a method, apparatus, device, and storage medium for determining loitering behavior. Background Technology

[0002] Loitering behavior is one of the main abnormal behaviors in security monitoring and also a challenging aspect of surveillance. Related technologies for monitoring loitering behavior all rely on image recognition technology. Specifically, after obtaining video data of the monitored area, image recognition technology is used to process the video data to obtain monitoring results showing that the same object has been present in the monitored area for a preset duration. This monitoring result indicates the presence of loitering behavior in the monitored area. This type of monitoring solution targets the same object and is generally suitable for monitoring public scenarios such as roads, residential areas, shopping malls, and industrial parks.

[0003] Monitoring of special scenarios such as key computer rooms, warehouses, and substations is to ensure the safety of server racks, storage areas, and power distribution cabinets. Therefore, the monitoring of loitering behavior in such scenarios needs to be precise, specifically to the point that loitering behavior is present around which server rack area, which storage area, and which power distribution cabinet area.

[0004] Currently, the monitoring solutions in related technologies cannot meet the monitoring needs for loitering behavior in the aforementioned special scenarios. Summary of the Invention

[0005] The purpose of this application is to solve one of the above-mentioned problems.

[0006] On one hand, embodiments of this application provide a method for determining loitering behavior, the method comprising:

[0007] The system acquires the current image of a preset region, which includes at least one sub-region. If the current image is determined to include the imaging region of at least one loitering object, the system obtains the target pixel coordinates mapped to each loitering object in the current image and determines the sub-region where the corresponding loitering object is located at the current time based on each target pixel coordinate. The system determines a loitering identifier for each sub-region at the current time based on the sub-region where each loitering object is located; the loitering identifier indicates the presence of a loitering object. For each sub-region, the system determines whether loitering behavior exists at the current time based on the loitering identifiers of the sub-region at the current time and at least one historical time prior to the current time.

[0008] Optionally, determining that the current image includes an imaging region containing at least one lodged object includes:

[0009] Object recognition is performed on the current image to obtain a set of recognition boxes. The set of recognition boxes is a collection of recognition boxes used to mark the imaging region of a loitering object in the current image. If the set of recognition boxes includes at least one recognition box, it is determined that there is an imaging region of at least one loitering object in the current image.

[0010] Optionally, obtain the target pixel coordinates mapped to each lingering object in the current image, including:

[0011] For each bounding box, determine the pixel coordinates of the bounding box in the current image. Perform joint point recognition on the imaging region marked by the bounding box to obtain the pixel coordinates mapped to at least one preset joint point of the corresponding object in the corresponding imaging region. For each object, determine the pixel coordinates mapped to the object in the corresponding imaging region based on the pixel coordinates mapped to at least one preset joint point of the object in the corresponding imaging region. For each object, determine the target pixel coordinates mapped to the object in the current image based on the pixel coordinates of the bounding box and the pixel coordinates mapped to the object in the corresponding imaging region.

[0012] Optionally, the sub-region where the corresponding lingering object is located at the current time is determined based on each target pixel coordinate, including: for each target pixel coordinate, determining the image sub-region to which the target pixel coordinate belongs, and taking the corresponding sub-region of the image sub-region as the sub-region where the lingering object corresponding to the target pixel coordinate is located.

[0013] The current image includes the corresponding image sub-region for each sub-region, and the image sub-region is the imaging area of ​​the corresponding sub-region in the current image.

[0014] Optionally, the dwelling identifier of each sub-region at the current moment is determined based on the sub-region where each dwelling object is located. This includes: determining whether each sub-region has dwelling objects. For each sub-region with dwelling objects, a real-time dwelling identifier carrying the current time's timestamp is created, and this created real-time dwelling identifier is used as the dwelling identifier of the sub-region at the current moment. For each sub-region without dwelling objects, a preset dwelling identifier is used as the dwelling identifier of the sub-region at the current moment.

[0015] The lingering identifier includes a real-time lingering identifier and a preset lingering identifier. The real-time lingering identifier indicates that there is a lingering object; the preset lingering identifier indicates that there is no lingering object.

[0016] Optionally, the method also includes;

[0017] If it is determined that the current image does not include the imaging area of ​​the lingering object, then the preset lingering identifier is used as the lingering identifier for each sub-region at the current moment.

[0018] Optionally, based on the lingering identifiers of the sub-region at at least one historical time prior to the current time, determine whether the sub-region exhibits lingering behavior at the current time, including:

[0019] Determine the first number of real-time detained identifiers and the second number of preset detained identifiers among all detained identifiers, as well as the sum of the first and second numbers. If the following conditions are met, it is determined that there is a detained behavior in the sub-region at the current time: the first ratio of the sum to the preset number is not less than a first preset threshold; the second ratio between the first number and the sum is not less than a second preset threshold; and the difference between the maximum and minimum timestamps corresponding to each real-time detained identifier is not less than a third preset threshold.

[0020] The sum of the number of current lingering identifiers and the number of lingering identifiers at least one historical time is a preset number.

[0021] Optionally, the method further includes:

[0022] For each sub-region, if there is lingering behavior in the sub-region at the current time, the sub-region is identified as the target sub-region, and an alarm message is output. The alarm message includes the identification information of the target sub-region and the current time.

[0023] On the other hand, embodiments of this application provide a device for determining loitering behavior, the device comprising:

[0024] An acquisition device is used to acquire a current image of a preset region at the current time, the preset region including at least one sub-region.

[0025] The first determining module is used to, if it is determined that the current image includes the imaging area of ​​at least one loitering object, obtain the target pixel coordinates mapped by each loitering object in the current image, and determine the sub-region where the corresponding loitering object is located at the current time based on each target pixel coordinate.

[0026] The second determination module is used to determine the retention identifier of each sub-region at the current time based on the sub-region where each retention object is located. The retention identifier is used to indicate whether there is a retention object.

[0027] The third determination module is used to determine whether there is any lingering behavior in each sub-region at the current time, based on the lingering identifiers of the sub-region at the current time and at least one historical time before the current time.

[0028] This application also provides an electronic device, which includes: a memory, a processor, and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the steps of a method for determining a dwelling behavior.

[0029] This application also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of a method for determining a dwelling behavior.

[0030] The beneficial effects of the technical solutions provided in this application are:

[0031] First, the monitoring area in a special scenario can be understood as a pre-defined area. The cabinet area, storage area, or power distribution cabinet area within the monitoring area can be considered as at least one sub-area within this pre-defined area. By pre-locating each area in the special scenario, subsequent identification of loitering behavior can be precise down to the area level, such as the cabinet area, storage area, or power distribution cabinet area.

[0032] One method for monitoring a preset area is image acquisition and analysis. Specifically, this involves acquiring the current image of the preset area at the current moment, where the preset area includes at least one sub-region. If the current image is determined to include the imaging area of ​​at least one loitering object, the target pixel coordinates mapped to each loitering object in the current image are obtained, and the sub-region where the corresponding loitering object is located at the current moment is determined based on each target pixel coordinate. That is, when a loitering object exists in the preset area, it is necessary to determine the sub-region where each loitering object is located. Further, a loitering identifier is determined for each sub-region at the current moment based on the sub-region where each loitering object is located. This loitering identifier is used to indicate whether a loitering object exists. For each sub-region, based on the loitering identifiers of the sub-region at the current moment and at least one historical moment prior to the current moment, it is determined whether loitering behavior exists in the sub-region at the current moment. For example, since one loitering identifier is obtained per moment, by judging whether the number of identifiers indicating the presence of a loitering object among the loitering identifiers reaches a standard, it is indirectly determined whether the duration of loitering behavior reaches a standard. Since the determination of loitering behavior is the behavior of an object staying in a certain area or around a certain thing for a long time, if the duration reaches the standard of "long time", then loitering behavior is determined to exist; if it does not reach the standard, then loitering behavior is determined not to exist.

[0033] This application's embodiments can accurately identify whether there is lingering behavior in each sub-area of ​​a preset area. Especially for monitoring areas in special scenarios, this application's embodiments can accurately identify whether there is lingering behavior in each cabinet area, warehouse area, and power distribution cabinet area of ​​the monitoring area, thereby ensuring the safety of each device in special scenarios. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0035] Figure 1A flowchart illustrating a method for determining a loitering behavior provided in an embodiment of this application;

[0036] Figure 2 A schematic diagram illustrating the correspondence between a sub-region in a preset area and an image sub-region in an image, provided in an embodiment of this application;

[0037] Figure 3 An identification frame provided for an embodiment of this application;

[0038] Figure 4a A schematic diagram of 18 key parts or joints of the human body provided in the embodiments of this application;

[0039] Figure 4b An image including a stationary object is provided as an embodiment of this application;

[0040] Figure 5 A schematic diagram of a sub-region's state queue provided in an embodiment of this application;

[0041] Figure 6 A schematic diagram of the structure of a device for determining dwell behavior provided in an embodiment of this application;

[0042] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0043] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0044] Those skilled in the art will understand that, unless otherwise stated, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” indicates implementation as “A,” or implementation as “A,” or implementation as “A and B.”

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0046] First, let's introduce and explain several terms used in this application:

[0047] The behavior of loitering refers to the act of a loitering object staying in a certain area or around a certain thing for an extended period of time.

[0048] A lingering object; an object that stays or lingers around a certain area or thing.

[0049] YOLO is an object detection model that primarily uses a single CNN model to achieve end-to-end object detection. Its core idea is to use the entire image as input to the network and directly output the location of the regressed bounding box and its corresponding category at the output layer. Currently, YOLOv5, part of the YOLO algorithm series, can specifically output objects of target categories, such as people, cats, and dogs.

[0050] The open-pose model is a human pose estimation model that can estimate poses such as human movements, facial expressions, and finger movements. This model takes an image containing a human object as input and outputs the coordinates of each joint of the human object.

[0051] The technical solutions of this application and their effects are described below through several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0052] Figure 1 A flowchart illustrating a method for determining loitering behavior is shown. This method includes the following steps S110–140.

[0053] S110, acquire the current image of a preset region at the current time, the preset region including at least one sub-region.

[0054] Specifically, a camera device is set up at multiple fixed locations, with each device's camera facing a preset area to ensure that the captured image includes the complete image of the preset area. Each captured image includes a corresponding image sub-region for each sub-region, which is the imaged area of ​​that sub-region within the captured image. For images captured by the same camera device, since the camera device is in a fixed position, the position of the image sub-region corresponding to each sub-region is consistent within the captured image; that is, the pixel coordinates of the image sub-region corresponding to the same sub-region are consistent across all captured images.

[0055] For each camera device, images are acquired at a preset frequency. For example, 5 frames are acquired per second, or one frame is acquired every 200ms.

[0056] Optionally, the first frame of the image is acquired, and the corresponding image sub-regions in each sub-region of the first frame are determined, along with the pixel coordinates of each image sub-region in the first frame. For methods of obtaining the pixel coordinates of image sub-regions, please refer to relevant technologies.

[0057] To illustrate the relationship between sub-regions and image sub-regions, this application also provides an example.

[0058] In this example, Figure 2 The left half shows a preset region, which comprises a total of m*n sub-regions. The identifier and location of each sub-region are as follows: Figure 2 As shown, each sub-region is square in shape. Figure 2 The right half of the image shows an image comprising multiple image sub-regions, each trapezoidal in shape. The image sub-region labeled "A0B0" corresponds to the sub-region labeled "A0B0", and the image sub-region labeled "AmBn" corresponds to the sub-region labeled "AmBn". Due to perspective, when imaging a preset area, the imaging of the preset area and its sub-regions undergoes the following changes: Figure 2 The deformation shown.

[0059] S120, if it is determined that the current image includes the imaging area of ​​at least one lingering object, then the target pixel coordinates mapped by each lingering object in the current image are obtained, and the sub-region where the corresponding lingering object is located at the current time is determined based on each target pixel coordinate.

[0060] Specifically, object recognition is performed on the current image to obtain a set of recognition boxes. The set of recognition boxes is a collection of recognition boxes used to mark the imaging area of ​​a loitering object in the current image. For example, each recognition box includes the identifier of the corresponding loitering object and the pixel coordinates of the recognition box in the current image.

[0061] If the set of recognition boxes includes at least one recognition box, it is determined that the current image contains an imaging region of at least one loitering object. If the set of recognition boxes does not include any recognition boxes, it is determined that the current image does not contain an imaging region of a loitering object.

[0062] To better understand the recognition box, an example is also provided in this application embodiment.

[0063] In this example, the YOLO object detection model is called to detect the current image and output a set of bounding boxes. When the bounding box set is not empty, its contents are as follows:

[0064]

[0065] The current image includes n figures, where "figure n" is the identifier of a lingering object, and "[x0,y0,x1,y1]" represents the pixel coordinates of the bounding box in the current image. In this embodiment, the bounding box is rectangular, and (x0,y0) and (x1,y1) are the pixel coordinates of two opposite vertices of the bounding box. Therefore, by determining the pixel coordinates of these two opposite vertices, the specific location of the bounding box can be determined.

[0066] In this example, Figure 3 The image also shows two bounding boxes, each containing an object.

[0067] S130, determine the retention identifier of each sub-region at the current time based on the sub-region where each retention object is located. The retention identifier is used to indicate whether there is a retention object.

[0068] S140, For each sub-region, determine whether there is any lingering behavior in the sub-region at the current time based on the lingering identifiers of the sub-region at the current time and at least one historical time before the current time.

[0069] This application's embodiments can accurately identify whether there is lingering behavior in each sub-area of ​​a preset area. Especially for monitoring areas in special scenarios, this application's embodiments can accurately identify whether there is lingering behavior in each cabinet area, warehouse area, and power distribution cabinet area of ​​the monitoring area, thereby ensuring the safety of each device in special scenarios.

[0070] For a lingering object, any part of its body can represent its location, but whether this representation method can correctly establish the association between the lingering object and the preset sub-region is uncertain. Therefore, this application also provides an optional embodiment to solve this technical problem.

[0071] In one implementation of this embodiment, the following steps Sa1 to Sa3 are included.

[0072] Sa1, for each recognition box, determine the pixel coordinates of the recognition box in the current image, perform joint point recognition on the imaging area marked by the recognition box, and obtain the pixel coordinates of at least one preset joint point of the corresponding lingering object of the recognition box mapped in the corresponding imaging area.

[0073] Specifically, for each bounding box, the pixel coordinates of the bounding box in the current image are obtained from the bounding box set. The imaging region marked by the bounding box is determined based on its corresponding pixel coordinates, and this marked imaging region is used as the corresponding imaging region of the bounding box. Joint point recognition is performed on the corresponding imaging region in the current image to obtain the pixel coordinates of at least one preset joint point of the corresponding stagnant object in the current image.

[0074] For example, the open-pose model is called to process the corresponding imaging area of ​​the recognition box, obtaining the pixel coordinates of 18 key parts or joints of the stationary object. Figure 4a As shown, the following are included: 0, mouth; 1, neck; 2, left shoulder joint; 3, left elbow joint; 4, left wrist joint; 5, right shoulder joint; 6, right elbow joint; 7, right wrist joint; 8, left hip; 9, left knee joint; 10, left ankle joint; 11, right hip; 12, right knee joint; 13, right ankle joint; 14, left eye; 15, right eye; 16, left ear; 17, right ear.

[0075] Optionally, at least one joint is associated with the standing position of the object being held, and the pixel coordinates of that standing position in the current image are used as the target pixel coordinates. For example, when the object being held is a human body, the preset joints are the left and right ankle joints of the human body.

[0076] To better understand the technical effects of selecting the left and right ankle joints as preset joint points, this application also provides a specific example.

[0077] This example includes Figure 4b The image shown represents a sub-region, with each grid on the ground. Within a bounding box, a human figure is depicted, with the center point of the box designated as the figure's center position S' and the midpoint of the ankle as S. Regions A0, A1, and A2 are defined as the three sub-regions. Compared to region A2, which contains the figure's center position S', region A0, containing the midpoint of the ankle S, more accurately represents the actual standing area A0 of the human figure.

[0078] Sa2, for each lingering object, determine the pixel coordinates mapped in the corresponding imaging region based on the pixel coordinates mapped in the corresponding imaging region for at least one preset joint of the lingering object.

[0079] Specifically, based on the pixel coordinates of the left and right ankle joints mapped in the current image, the pixel coordinates of the standing position of the stranded object mapped in the corresponding imaging area are determined.

[0080] Sa3: For each lingering object, determine the target pixel coordinates mapped by the lingering object in the current image based on the pixel coordinates of the corresponding recognition box of the lingering object in the current image and the pixel coordinates mapped by the lingering object in the corresponding imaging area.

[0081] In another implementation of this embodiment, the association between the lingering object and the sub-region can be established by the target pixel coordinates of each lingering object, such as determining the sub-region where the corresponding lingering object is located at the current time based on the target pixel coordinates.

[0082] Specifically, for each target pixel coordinate, the image sub-region to which the target pixel coordinate belongs is determined, and the corresponding sub-region of the image sub-region is taken as the sub-region where the object corresponding to the target pixel coordinate is located.

[0083] Since determining loitering behavior involves a "long period of time," the images analyzed are those acquired at multiple consecutive moments, and recording the identification results at each moment is the basis for the judgment. Therefore, this application also provides an optional embodiment to mark the loitering information of each sub-region at each moment.

[0084] In one implementation of this embodiment, the dwelling identifier of each sub-region at the current moment can be determined based on the sub-region where the dwelling object is located. This implementation includes steps Sb1 to Sb2.

[0085] Sb1 determines whether there are any stranded objects in each sub-region based on the sub-region where each stranded object is located.

[0086] Optionally, if the number of at least one stranded object is less than the number of at least one sub-region, then each stranded object is traversed, and the corresponding sub-region containing the stranded object is determined based on the sub-region where each stranded object is located. For other sub-regions, it is determined that no stranded object exists.

[0087] Optionally, each sub-region is also associated with statistical data of the stranded objects. After determining the sub-region where the stranded object corresponding to the target pixel coordinates is located, the statistical data associated with the sub-region can be updated. If it is determined that there are updated records for the statistical data associated with each sub-region, then it is determined that there are stranded objects in the corresponding sub-region.

[0088] Sb2, for each sub-region with a lingering object, create a real-time lingering identifier carrying the current time's timestamp, and use the created real-time lingering identifier as the lingering identifier of the sub-region at the current time; for each sub-region without a lingering object, use the preset lingering identifier as the lingering identifier of the sub-region at the current time.

[0089] Optionally, the current timestamp can be used as a real-time persistence identifier; or, for each sub-region, the concatenation of the current timestamp and the statistical data of the sub-region can be used as a real-time persistence identifier.

[0090] The loitering identifier includes a real-time loitering identifier and a preset loitering identifier. The real-time loitering identifier indicates the presence of a loitering object and can be obtained using the methods described above. The preset loitering identifier indicates the absence of a loitering object and can be set to a fixed identifier.

[0091] For each sub-region, the lingering marker at the time when a lingering object exists carries time information, which can provide a reference for identifying lingering behavior.

[0092] In another optional implementation of this embodiment, when there are no lingering objects in the preset area, the lingering identifier of each sub-area can be set to the preset lingering identifier.

[0093] Specifically, if it is determined that the current image does not include the imaging area of ​​the lingering object, then the preset lingering identifier is used as the lingering identifier of each sub-region at the current moment.

[0094] In another optional implementation of this embodiment, each sub-region is associated with a state queue. The state queue includes a preset number of lingering flag bits, which are arranged sequentially. Each lingering flag bit includes a lingering flag.

[0095] During the terminal initialization phase, the status queue uses a default retention flag for each retention flag. This default retention flag does not indicate the existence of any retention object; it is merely a placeholder.

[0096] After obtaining the stagnation flag of the sub-region at the current moment, each stagnation flag in the state queue is shifted forward by one. The first stagnation flag is then deleted, and a new stagnation flag is selected as the candidate. The stagnation flag of the new stagnation flag is the stagnation flag of the sub-region at the current moment.

[0097] In the state queue, each of the delay flags preceding the corresponding delay flag at the current moment is a delay flag corresponding to each historical moment at the current moment.

[0098] The definition of loitering behavior is the presence of an object in a region for an extended period. After obtaining the loitering identifiers of the sub-region at the current time and at least one historical time, determining the loitering behavior based on these identifiers is a technical problem that needs to be solved. To address this technical problem, this application also provides an optional embodiment.

[0099] In one possible implementation of this embodiment, step S140 specifically includes the following steps Sc1 to Sc2.

[0100] Among them, each lingering identifier of the sub-region at the current time and at least one historical time before the current time includes a preset number of lingering identifiers.

[0101] Optionally, after updating the lingering identifier of the sub-region to the state queue at the current time, the lingering identifier of each lingering identifier bit in the state queue can be used as the lingering identifier in step S140.

[0102] Sc1 determines the first number of real-time detained markers and the second number of preset detained markers among all detained markers, as well as the sum of the first number and the second number.

[0103] If Sc2 satisfies the following conditions 1 to 3, then it is determined that the sub-region exhibits lingering behavior at the current moment.

[0104] Condition 1: The ratio of the total to the first preset quantity is not less than the first preset threshold.

[0105] Since the preset number of lingering markers may include default lingering markers, and the default lingering markers may not have been used for "image acquisition" at the corresponding time, and "image acquisition" signifies a step of substantively identifying whether a lingering object exists in the preset area, it is necessary to determine whether the proportion of lingering markers representing "image acquisition" in the preset number of lingering markers reaches a first preset threshold. This proportion can be determined by the first ratio of the sum of the first and second numbers to the preset number. If the first ratio is not less than the first preset threshold, then it is determined that lingering behavior can be identified.

[0106] Condition 2: The second ratio between the first quantity and the total is not less than the second preset threshold.

[0107] Condition 3: The difference between the maximum and minimum timestamps corresponding to each real-time lingering identifier is not less than the third preset threshold.

[0108] Since the definition of loitering behavior includes "a period of loitering," conditions 2 and 3 can be used to identify loitering behavior. In condition 2, identification is only possible if a certain number of loitering instances occur. The ability to identify loitering behavior is determined by the ratio of the first number to the sum of the first and second numbers. For example, if the second ratio is not less than a second preset threshold, it indicates that loitering instances occur most of the time, and loitering behavior can be identified.

[0109] Next, in condition 3, obtain the comparison result where the difference between the maximum and minimum timestamps corresponding to each real-time lingering identifier is not less than the third preset threshold. If the comparison result is not less than the third preset threshold, it indicates that there is lingering behavior.

[0110] To better understand this implementation, an example is also provided in this application embodiment.

[0111] In this example, Figure 5 The update process of the state queue stack_AiBi of the sub-region identified as “AiBi” is shown. The “new state” is the current state of the sub-region, and the “old state” is the state of the sub-region in the past.

[0112] In this example, the preset retention flag in stack_AiBi is marked with "0", the default retention flag is marked with "null", and the real-time retention flag is the timestamp of the corresponding moment.

[0113] In this example, the first number of real-time loitering identifiers in stack_AiBi is n_int, and the second preset number of loitering identifiers is n_0. Among the timestamps corresponding to each real-time loitering identifier, the maximum timestamp is max_t, and the minimum timestamp is min_t. The preset number is L, the first preset threshold is 0.9, the second preset threshold is 0.8, and the third preset threshold is TS, where TS can be 5 seconds.

[0114] This example also provides the following three conditions:

[0115]

[0116] If each value meets the above conditions, then it is determined that there is lingering behavior in the sub-region of "AiBi".

[0117] To further meet the security requirements of special scenarios, an alarm needs to be triggered after the presence of loitering behavior is confirmed. Therefore, this application also provides an optional embodiment.

[0118] In one implementation of this embodiment, for each sub-region, if there is lingering behavior in the sub-region at the current time, the sub-region is identified as the target sub-region, and alarm information is output. The alarm information includes the identification information of the target sub-region and the current time.

[0119] Optionally, alarm information can be entered into a preset log for future reference in security work for the preset area.

[0120] Figure 6 An embodiment of this application illustrates a device 600 for determining loitering behavior. The device 600 includes an acquisition device 610, a first determining module 620, a second determining module 630, and a third determining module.

[0121] Acquisition device 610 is used to acquire a current image of a preset region at the current time, the preset region including at least one sub-region;

[0122] The first determining module 620 is used to, if it is determined that the current image includes the imaging area of ​​at least one lingering object, obtain the target pixel coordinates mapped by each lingering object in the current image, and determine the sub-region where the corresponding lingering object is located at the current time based on each target pixel coordinate;

[0123] The second determining module 630 is used to determine the retention identifier of each sub-region at the current time based on the sub-region where each retention object is located. The retention identifier is used to indicate whether there is a retention object.

[0124] The third determining module 640 is used to determine, for each sub-region, whether there is any lingering behavior in the sub-region at the current time, based on the lingering identifiers of the sub-region at the current time and at least one historical time before the current time.

[0125] The apparatus in this application embodiment can execute the method provided in this application embodiment, and the implementation principle is similar. The actions performed by each module in the apparatus of each embodiment of this application correspond to the steps in the method of each embodiment of this application. For detailed functional descriptions of each module of the apparatus, please refer to the descriptions in the corresponding methods shown above, which will not be repeated here.

[0126] This application provides an electronic device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of a method for determining loitering behavior. Compared with the prior art, this method can achieve the purpose of accurately identifying whether loitering behavior exists in each sub-region of a preset area.

[0127] See Figure 7 This application also provides a specific example of an electronic device. Figure 7The illustrated electronic device 7000 includes a processor 7001 and a memory 7003. The processor 7001 and the memory 7003 are connected, for example, via a bus 7002. Optionally, the electronic device 7000 may further include a transceiver 7004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 7004 is not limited to one type, and the structure of the electronic device 7000 does not constitute a limitation on the embodiments of this application.

[0128] Processor 7001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 7001 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0129] Bus 7002 may include a pathway for transmitting information between the aforementioned components. Bus 7002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 7002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0130] The memory 7003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0131] The memory 7003 is used to store computer programs that execute the embodiments of this application, and its execution is controlled by the processor 7001. The processor 7001 is used to execute the computer programs stored in the memory 7003 to implement the steps shown in the foregoing method embodiments.

[0132] Among them, electronic devices include, but are not limited to, computer terminals.

[0133] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the steps and corresponding content of the aforementioned method embodiments.

[0134] This application also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0135] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.

[0136] The above are only optional implementation methods for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application, without departing from the technical concept of this application, also fall within the protection scope of the embodiments of this application.

Claims

1. A method of determining a behavior of a retention, characterized in that, include: Acquire the current image of a preset region at the current time, wherein the preset region includes at least one sub-region; If it is determined that the current image includes the imaging area of ​​at least one lingering object, then the target pixel coordinates mapped by each lingering object in the current image are obtained, and the sub-region where the corresponding lingering object is located at the current time is determined based on each target pixel coordinate; The presence or absence of a stranded object is determined based on the sub-region where the stranded object is located at the current time. For each sub-region, based on the lingering identifiers of the sub-region at the current time and at least one historical time prior to the current time, it is determined whether the sub-region has lingering behavior at the current time; Wherein, determining that the current image includes the imaging region of at least one lingering object includes: Object recognition is performed on the current image to obtain a set of recognition boxes for the current image. The set of recognition boxes is a collection of recognition boxes, and the recognition boxes are used to mark the imaging area of ​​a lingering object in the current image. If the set of recognition boxes includes at least one recognition box, then it is determined that the current image contains an imaging region of at least one lingering object; Obtaining the target pixel coordinates mapped to each lingering object in the current image includes: For each recognition box, determine the pixel coordinates of the recognition box in the current image, perform joint point recognition on the imaging area marked by the recognition box, and obtain the pixel coordinates of at least one preset joint point of the corresponding lingering object of the recognition box mapped in the corresponding imaging area. For each stranded object, the pixel coordinates mapped by the stranded object in the corresponding imaging area are determined based on the pixel coordinates mapped by at least one preset joint point of the stranded object in the corresponding imaging area. For each lingering object, the target pixel coordinates mapped by the lingering object in the current image are determined based on the pixel coordinates of the corresponding recognition box of the lingering object in the current image and the pixel coordinates mapped by the lingering object in the corresponding imaging region.

2. The method of claim 1, wherein, The current image includes a corresponding image sub-region for each sub-region, and the image sub-region is the imaging area of ​​the corresponding sub-region in the current image; The step of determining the sub-region where the corresponding lingering object is located at the current time based on the coordinates of each target pixel includes: For each target pixel coordinate, determine the image sub-region to which the target pixel coordinate belongs, and use the corresponding sub-region of the image sub-region as the sub-region where the object corresponding to the target pixel coordinate is located.

3. The method of claim 1, wherein, The dwelling identifier includes a real-time dwelling identifier and a preset dwelling identifier. The real-time dwelling identifier indicates that there is a dwelling object; the preset dwelling identifier indicates that there is no dwelling object. The step of determining the retention identifier of each sub-region at the current time based on the sub-region where each retained object is located includes: Determine whether there are any stranded objects in each sub-region based on the sub-region where each stranded object is located; For each sub-region where the detained object exists, a real-time detention identifier carrying the timestamp of the current time is created, and the created real-time detention identifier is used as the detention identifier of the sub-region at the current time; For each sub-region where no such object exists, the preset retention identifier is used as the retention identifier of the sub-region at the current time.

4. The method of claim 3, wherein, The method further includes; If it is determined that the current image does not include the imaging area of ​​the lingering object, then the preset lingering identifier is used as the lingering identifier for each sub-region at the current moment.

5. The method according to claim 3, characterized in that, The sum of the number of the current time's lingering identifiers and the number of the at least one historical time's lingering identifiers is a preset number; The step of determining whether the sub-region has any lingering behavior at the current time based on the lingering identifiers of the sub-region at at least one historical time including the current time and before the current time includes: Determine the first number of real-time detained identifiers and the second number of preset detained identifiers in each detained identifier, as well as the sum of the first number and the second number; If the following conditions are met, it is determined that the sub-region exhibits lingering behavior at the current time: The first ratio of the sum to the preset quantity is not less than a first preset threshold. The second ratio between the first quantity and the sum is not less than a second preset threshold. The difference between the maximum and minimum timestamps corresponding to each real-time retention identifier is not less than the third preset threshold.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: For each sub-region, if there is lingering behavior in the sub-region at the current time, the sub-region is identified as the target sub-region, and an alarm message is output. The alarm message includes the identification information of the target sub-region and the current time.

7. A device for determining a behavior of a detainee, characterized in that include: An acquisition device is used to acquire a current image of a preset region at the current time, wherein the preset region includes at least one sub-region; The first determining module is configured to, if it is determined that the current image includes an imaging region of at least one lingering object, obtain the target pixel coordinates mapped by each lingering object in the current image, and determine the sub-region where the corresponding lingering object is located at the current time based on each target pixel coordinate; The second determining module is used to determine the retention identifier of each sub-region at the current time based on the sub-region where each retention object is located, and the retention identifier is used to indicate whether there is a retention object; The third determining module is used to determine, for each sub-region, whether there is any lingering behavior in the sub-region at the current time, based on the lingering identifiers of the sub-region at the current time and at least one historical time before the current time. Wherein, the first determining module determines that the current image includes an imaging region containing at least one loitering object, including: Object recognition is performed on the current image to obtain a set of recognition boxes for the current image. The set of recognition boxes is a collection of recognition boxes, and the recognition boxes are used to mark the imaging area of ​​a lingering object in the current image. If the set of recognition boxes includes at least one recognition box, then it is determined that the current image contains an imaging region of at least one lingering object; Obtaining the target pixel coordinates mapped to each lingering object in the current image includes: For each recognition box, determine the pixel coordinates of the recognition box in the current image, perform joint point recognition on the imaging area marked by the recognition box, and obtain the pixel coordinates of at least one preset joint point of the corresponding lingering object of the recognition box mapped in the corresponding imaging area. For each stranded object, the pixel coordinates mapped by the stranded object in the corresponding imaging area are determined based on the pixel coordinates mapped by at least one preset joint point of the stranded object in the corresponding imaging area. For each lingering object, the target pixel coordinates mapped by the lingering object in the current image are determined based on the pixel coordinates of the corresponding recognition box of the lingering object in the current image and the pixel coordinates mapped by the lingering object in the corresponding imaging region.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program, when executed by the processor, is arranged to perform the method of any one of claims 1 to 7. The processor executes the computer program to implement the steps of the method according to any one of claims 1-6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-6.

Images