Intelligent loss detection and comprehensive prompt method for business hall

Abstract

The invention discloses an intelligent loss detection and comprehensive prompt method and device for a business hall, a storage medium and computer equipment, and the method comprises the steps: collecting a first image of a first target region through a first image collection device in the business hall, and determining whether a foreground object exists in the first image or not based on a preset background image; if the foreground object exists, removing a character object from the foreground object to obtain a target tracking object, and creating a tracking thread corresponding to the target tracking object; monitoring the position and the state of the target tracking object based on a subsequent image of the first target area through the tracking thread, and when the monitoring duration reaches a first preset duration, determining whether the target tracking object is a lost object based on a position monitoring result and a state monitoring result within the first preset duration; and when the target tracking object is a lost object, marking the target tracking object in any image, sending any marked image to a preset supervision terminal, and destroying the tracking thread.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method and device for intelligent loss detection and comprehensive prompting in business halls, a storage medium, and a computer device. Background Art

[0002] With the rapid development of society and the increasingly accelerated pace of life, business halls, as key venues for providing various services, receive a large number of customers daily. Within business halls, customers may inadvertently lose personal items such as mobile phones, wallets, and ID cards while conducting business. These lost items not only cause significant inconvenience to the owner but can also lead to a series of security issues, such as personal information leakage and property loss. Furthermore, for business halls, frequent item loss incidents can negatively impact service quality and customer satisfaction.

[0003] To address the problem of lost items within business halls, traditional solutions rely primarily on manual inspections and proactive customer reporting. However, manual inspections are limited in efficiency, coverage, and susceptibility to human error, making comprehensive, real-time monitoring of every area within the business hall difficult. Proactive customer reporting, on the other hand, relies on the customer's awareness and memory, often leading to lags and hindering timely discovery and resolution of lost items.

[0004] In recent years, with the rapid development of computer vision and artificial intelligence technologies, intelligent monitoring systems based on image processing have been gradually applied to various fields, providing new solutions for addressing the problem of lost items in business halls. However, existing intelligent monitoring systems mostly focus on analyzing human behavior and security monitoring, and their automatic detection and notification capabilities for lost items are still incomplete. Especially in the complex and ever-changing business hall environment, how to accurately and efficiently identify lost items and promptly notify relevant personnel remains an urgent problem. Summary of the Invention

[0005] In light of this, this application provides a method and device for intelligent lost item detection and comprehensive notification in business halls, as well as storage media and computer equipment. These methods can automatically detect and notify lost items within business halls, enabling comprehensive, real-time monitoring of all areas within the business hall, accurately identifying lost items, and promptly sending notifications to pre-set monitoring terminals, enabling relevant personnel to take prompt action and improving customer satisfaction. Furthermore, by destroying no longer needed tracking threads, system resources are conserved, improving system stability and scalability.

[0006] According to one aspect of the present application, a method for intelligent loss detection and comprehensive prompting in a business hall is provided, comprising:

[0007] Capturing a first image of a first target area by a first image acquisition device in the business hall, and determining whether there is a foreground object in the first image based on a preset background image corresponding to the first target area;

[0008] In the case where there is a foreground object in the first image, removing the human object from the foreground object to obtain a target tracking object, and creating a tracking thread corresponding to the target tracking object;

[0009] monitoring, by the tracking thread, a position and a status of the target tracking object based on subsequent images of the first target area, and determining whether the target tracking object is a lost object based on position monitoring results and status monitoring results within the first preset time period when the monitoring time period reaches a first preset time period;

[0010] When the target tracking object is a lost object, the target tracking object is marked in any image, any marked image is sent to a preset supervision terminal, and the tracking thread is destroyed, wherein the any image is one of the first image or the subsequent image.

[0011] According to another aspect of the present application, there is provided an intelligent lost item detection and comprehensive prompting device for a business hall, comprising:

[0012] an image acquisition module, configured to acquire a first image of a first target area through a first image acquisition device in the business hall, and determine whether there is a foreground object in the first image based on a preset background image corresponding to the first target area;

[0013] a tracking object determination module, configured to, when a foreground object exists in the first image, remove a human object from the foreground object to obtain a target tracking object, and create a tracking thread corresponding to the target tracking object;

[0014] a monitoring module, configured to monitor, through the tracking thread, the position and status of the target tracking object based on subsequent images of the first target area, and, when a monitoring duration reaches a first preset duration, determine whether the target tracking object is a lost object based on position monitoring results and status monitoring results within the first preset duration;

[0015] a marking module for marking the target tracking object in any image when the target tracking object is a lost object, sending any marked image to a preset supervision terminal, and destroying the tracking thread, wherein the any image is one of the first image or the subsequent image.

[0016] According to another aspect of the present application, a storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the above-mentioned intelligent lost item detection and comprehensive prompt method for the business hall is implemented.

[0017] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor. When the processor executes the program, the above-mentioned intelligent lost item detection and comprehensive prompt method for the business hall is implemented.

[0018] By means of the above-mentioned technical solution, the present application provides a method and device for intelligent lost object detection and comprehensive notification in a business hall, a storage medium, and a computer device. First, a first image acquisition device installed in the business hall captures a first image of a first target area in real time. Next, using image processing technology, the captured first image is compared with a preset background image, and a background difference method or other method is used to determine whether there is a foreground object in the first image. After detecting the foreground object, more sophisticated image processing technology is used to analyze the foreground object and eliminate human objects, thereby obtaining the target tracking object. Subsequently, a corresponding tracking thread is created for each determined target tracking object. When the monitoring time of the target tracking object reaches a first preset time, a determination is made based on the monitoring results whether the target tracking object is a lost object. When the target tracking object is determined to be a lost object, the object is marked in any image, and the marked image is sent to a preset monitoring terminal. After sending the marked image, the corresponding tracking thread is destroyed to free up system resources and avoid unnecessary thread occupation. This embodiment of the present application can automatically detect and provide notifications for lost items within a business hall. It enables comprehensive, real-time monitoring of all areas within the business hall, accurately identifies lost items, and promptly sends notifications to pre-set monitoring terminals, enabling relevant personnel to take prompt action and improving customer satisfaction. Furthermore, by destroying no longer needed tracking threads, system resources are conserved, improving system stability and scalability.

[0019] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0021] Figure 1A flow chart of a method for intelligent loss detection and comprehensive prompting in a business hall provided by an embodiment of the present application is shown;

[0022] Figure 2 The following is a schematic diagram showing the structure of an intelligent lost item detection and comprehensive prompting device for a business hall provided in an embodiment of the present application;

[0023] Figure 3 A schematic diagram of the device structure of a computer device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that, unless there is a conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0025] In this embodiment, a method for intelligent loss detection and comprehensive prompting in a business hall is provided. Figure 1 As shown, the method includes:

[0026] Step 101: A first image of a first target area is captured by a first image capture device in a business hall, and based on a preset background image corresponding to the first target area, it is determined whether there is a foreground object in the first image.

[0027] An embodiment of the present application provides a method for intelligent lost item detection and comprehensive prompting in a business hall, which realizes real-time monitoring, intelligent analysis and lost item prompting of the business hall by combining image acquisition, image processing and thread concurrency technology. First, a first image of a first target area is acquired in real time by a first image acquisition device (such as a high-definition camera, etc.) installed in the business hall. Here, the first target area can be the entire lobby of the business hall, or an area where lost items appear more frequently, and the specific area can be determined according to actual needs. Then, using image processing technology, the acquired first image is compared with a preset background image (the preset background image is a static image when there are no people and dynamic objects in the first target area), and the background difference method and other methods are used to determine whether there is a foreground object in the first image. Among them, the background difference method is a commonly used image processing method, which detects foreground objects by calculating the pixel difference between the current first image and the preset background image.

[0028] Step 102: When there is a foreground object in the first image, remove the human object from the foreground object to obtain a target tracking object, and create a tracking thread corresponding to the target tracking object.

[0029] In this embodiment, after detecting foreground objects, more sophisticated image processing techniques (such as a human detection algorithm) are used to analyze the foreground objects and remove human objects, thereby obtaining target tracking objects. These target tracking objects are typically items left behind by customers in the business hall, such as mobile phones, wallets, and backpacks. The human detection algorithm is an algorithm that can identify the outline and posture of people in images. This distinguishes human objects from object objects, allowing for separate tracking and management of the items.

[0030] Subsequently, a corresponding tracking thread is created for each identified target tracking object. The tracking thread is responsible for continuously monitoring the position and status of the target tracking object based on subsequent images of the first target area. Different target tracking objects can create different threads to achieve thread concurrency. Thread concurrency is a technology that can execute multiple tasks simultaneously. In the embodiment of the present application, each tracking thread is independently responsible for the monitoring task of a target tracking object, improving the system's concurrent processing capabilities and response speed.

[0031] Step 103: Monitor the position and status of the target tracking object based on subsequent images of the first target area through the tracking thread, and when the monitoring time reaches a first preset time, determine whether the target tracking object is a lost object based on the position monitoring results and the status monitoring results within the first preset time.

[0032] In this embodiment, when the monitoring time of the target tracking object reaches a first preset time (such as the set item lost judgment time threshold), the target tracking object is judged to be a lost object based on the monitoring results (including position monitoring results and status monitoring results). Specifically, the lost object judgment may involve a comprehensive consideration of multiple factors, such as the position change of the target tracking object, the residence time, the interaction with the surrounding environment, etc. By setting reasonable judgment conditions, lost items can be accurately identified, false alarms and missed alarms can be avoided, and the accuracy of the system can be improved.

[0033] Step 104: When the target tracking object is a lost object, mark the target tracking object in any image, send any marked image to a preset monitoring terminal, and destroy the tracking thread, wherein the any image is one of the first image or the subsequent image.

[0034] In this embodiment, when the target tracking object is determined to be a lost object, the object is marked in any image (the first image or one of the subsequent images), and the marked image is sent to a preset monitoring terminal (such as the mobile phone or computer of the business hall manager). By marking and sending the image, the supervisor can quickly locate the lost item and take appropriate measures, improving the system's practicality and customer satisfaction. After sending the marked image, the corresponding tracking thread is destroyed to free up system resources, avoid unnecessary thread usage, improve the overall performance and stability of the system, and free up more resources for subsequent image acquisition and processing tasks.

[0035] By applying the technical solution of this embodiment, a first image capture device installed in the business hall first captures a first image of a first target area in real time. Next, using image processing techniques, the captured first image is compared with a preset background image, and the presence of a foreground object in the first image is determined using methods such as background subtraction. After detecting the foreground object, more sophisticated image processing techniques are used to analyze the foreground object and remove human objects from it, thereby obtaining the target tracking object. Subsequently, a corresponding tracking thread is created for each identified target tracking object. When the monitoring duration of the target tracking object reaches a first preset duration, a determination is made based on the monitoring results as to whether the target tracking object is a lost object. If the target tracking object is determined to be a lost object, the object is marked in any image, and the marked image is sent to a preset monitoring terminal. After the marked image is sent, the corresponding tracking thread is destroyed to free up system resources and avoid unnecessary thread usage. This embodiment of the present application can automatically detect and notify users of lost items within a business hall, enabling comprehensive, real-time monitoring of all areas within the business hall, accurately identifying lost items, and promptly sending a notification message to the preset monitoring terminal so that relevant personnel can take prompt action, thereby improving customer satisfaction. In addition, by destroying no longer needed tracking threads, system resources are saved, the stability and scalability of the system are improved, and more possibilities are provided for subsequent upgrades and optimizations.

[0036] In an embodiment of the present application, optionally, the "eliminating human objects from the foreground objects to obtain target tracking objects" in step 102 includes: inputting the foreground image corresponding to the foreground object into a preset detection model, identifying whether the foreground object contains human objects, and if the foreground object contains human objects, generating a bounding box corresponding to the human object, and using the foreground object outside the bounding box as the target tracking object.

[0037] In this embodiment, if the first image contains a foreground object, the foreground image corresponding to the foreground object can be input into a preset detection model. Here, the preset detection model can be a pre-trained neural network model specifically designed for identifying human objects in images. Specifically, the preset detection model can utilize deep learning techniques, such as a convolutional neural network (CNN), to be trained using a large amount of annotated data. During the training process, it learns the characteristics of human objects, thereby accurately identifying human objects in new images. The preset detection model then processes the input foreground image to determine whether the foreground object contains a human object. If the foreground object contains a human object, the preset detection model can further generate a bounding box for the human object in the foreground image based on the recognition results. Specifically, the bounding box can be a rectangular area that marks the location of the human object in the image. After obtaining the bounding box of the human object, the portion outside the bounding box is identified as the target object to be tracked. This portion is typically other foreground objects other than the human object, i.e., the object to be tracked. By eliminating the human object, the target object to be tracked is obtained, providing an accurate basis for subsequent target tracking and loss determination.

[0038] The embodiment of the present application uses a preset detection model to analyze and process the foreground image, which can accurately identify and eliminate human objects, thereby obtaining the target object that needs to be tracked. This not only improves the accuracy and robustness of the system, but also provides a reliable basis for subsequent target tracking and lost judgment.

[0039] In an embodiment of the present application, optionally, after "taking the foreground object outside the bounding box as the target tracking object", the method further includes: identifying the relative position between the target tracking object and the human object, and based on the relative position, calculating the confidence that the target tracking object belongs to the human object, and when the confidence is greater than a preset confidence threshold, eliminating the target tracking object.

[0040] In this embodiment, after removing human objects from the foreground objects, preliminary target tracking objects are obtained. However, sometimes some of the target tracking objects may be items carried by the person seeking to process the business. Therefore, to reduce invalid tracking, it is possible to further determine which target tracking objects may be items carried by the person seeking to process the business. Specifically, for each preliminary target tracking object, the relative positional relationship between it and the previously identified human object is calculated. For example, the distance, direction, and other parameters between the center point of the target tracking object and the nearest edge of the human object's bounding box are calculated to obtain the relative positional relationship. Next, based on the relative positional relationship between the target tracking object and the human object, a confidence score is calculated to measure the likelihood that the target tracking object is a human object. This confidence score can be based on a comprehensive consideration of multiple factors, such as distance, direction, and the size and shape of the target tracking object. By calculating the confidence score, the preliminary target tracking objects can be further screened to eliminate items that are closely related to the human object and are not truly required to be tracked. The calculated confidence score is then compared with a preset confidence threshold. If the confidence score is greater than the threshold, the target object is considered to be closely related to the person (such as a personal item) and not a real object to be tracked. Otherwise, the target object is considered independent and needs to be tracked. Target objects with a confidence score greater than the threshold are removed from the target object list. By eliminating potential misidentified objects, the accuracy of target tracking can be further improved, ineffective tracking can be reduced, and system resource waste can be avoided.

[0041] In an embodiment of the present application, optionally, the method further includes: acquiring a second image of a second target area through a second image acquisition device in the business hall, and extracting a person area from the second image; determining a first length of the person contained in the person area in a first direction and a second length in a second direction, and determining a minimum bounding box corresponding to the person area, calculating the length ratio corresponding to the person area based on the first length and the second length, and calculating the area and perimeter corresponding to the minimum bounding box; determining first posture information corresponding to the person through a first recognition model based on the length ratio, the area and perimeter corresponding to the minimum bounding box; when the first posture information indicates that the person is in a non-standing posture, determining the Zernike moment feature, the area corresponding to the person area, and the outer contour perimeter based on the extracted person area, and determining second posture information corresponding to the person through a second recognition model based on the Zernike moment feature, the area corresponding to the person area, and the outer contour perimeter; determining an abnormal posture type of the person based on the second posture information, and sending the abnormal posture type and the area identifier corresponding to the second target area to the preset supervision terminal.

[0042] In this embodiment, a second image of the second target area can also be captured by a second image capture device (camera or other image sensor) in the business hall. Here, the second target area can include the ground area of ​​the business hall. Then, the character area is extracted from the second image. The extraction of the character area can adopt image processing techniques such as background subtraction, edge detection, segmentation, etc. Next, the first length of the character area in the first direction (such as the horizontal direction) and the second length in the second direction (such as the vertical direction) are determined. At the same time, the minimum bounding box corresponding to the character area is determined, and the length ratio of the character area is calculated based on the first length and the second length. In addition, the area and perimeter corresponding to the minimum bounding box are also calculated. Specifically, the length ratio can be calculated by dividing the first length by the second length; the minimum bounding box can be determined by finding the minimum rectangular area containing the character area.

[0043] Afterwards, the first posture information corresponding to the person is determined through the first recognition model based on the length ratio of the person's area, the area corresponding to the minimum bounding box, and the perimeter. Here, the first recognition model can be a classifier based on machine learning or deep learning, which can be trained to recognize different posture types. The first recognition model can use a variety of algorithms, such as support vector machines (SVMs), neural networks (such as convolutional neural networks (CNNs), etc. The first recognition model can preliminarily determine the posture type of the person and output the first posture information of "standing posture" and "non-standing posture".

[0044] When the first posture information indicates that the person is in a non-standing posture, the Zernike moment feature, the area corresponding to the person area, and the outer contour circumference are further determined based on the extracted person area. Then, based on these feature information, the second posture information corresponding to the person is determined by the second recognition model. Among them, the Zernike moment is a mathematical tool for describing the shape of an image, which can capture the geometric features of the image. The second recognition model is also a classifier based on machine learning or deep learning, but is more focused on the recognition of non-standing postures. Based on the second posture information, the abnormal posture type of the person can be determined. Abnormal posture types may include abnormal behaviors such as falling and lying down. Then, the abnormal posture type and the area identification corresponding to the second target area are sent to the preset supervision terminal.

[0045] The embodiment of the present application adopts a dual recognition strategy: first, the first recognition model is used to identify whether it is a standing posture. If it is a non-standing posture, the second recognition model is used for more detailed recognition, which improves the flexibility and accuracy of recognition; through real-time monitoring and anomaly detection functions, abnormal behavior in the business hall can be discovered in time, providing timely and accurate information support for supervisors.

[0046] In an embodiment of the present application, optionally, the method also includes: after receiving the third image uploaded by the third image acquisition device corresponding to the business processing certificate collection device, extracting the facial image corresponding to the person to be processed from the third image, recording the time of collecting the business processing certificate corresponding to the person to be processed, and adding the certificate identifier corresponding to the business processing certificate to the business processing queue; if the duration between the current time and the time of collecting the business processing certificate is greater than the second preset duration, and there is no fourth image matching the facial image of the person to be processed in the fourth image captured by the fourth image acquisition device corresponding to each business processing window, then based on the preset priority adjustment strategy, adjust the position of the certificate identifier in the business processing queue.

[0047] In this embodiment, after receiving a third image uploaded by a third image capture device corresponding to a service voucher collection device, a facial image corresponding to the person waiting for service can be extracted from the third image. The service voucher collection device can be, for example, a number-taking machine. After entering the business hall, the person waiting for service can first obtain a number using the number-taking machine to queue up for service. Facial image extraction can utilize facial recognition technology, such as a deep learning-based face detection algorithm, to accurately identify and extract the facial region of the person waiting for service from the image. Next, the time the person waiting for service received their service voucher is recorded, and a voucher identifier corresponding to the service voucher is added to the service queue. For example, if the service voucher collection device is a number-taking machine, the service voucher can be a number plate printed by the number-taking machine, and the number corresponding to the number plate can be added to the service queue. The time the service voucher was collected can be recorded using a system timestamp or an external time source. The voucher identifier is a unique identifier for the service voucher, used to distinguish different vouchers in the service queue. The business processing queue is a data structure used to store the voucher identifiers corresponding to the pending business in a specific order (such as time order, priority order, etc.).

[0048] After recording the time when the business voucher corresponding to the pending business person is collected, timing can be started. If the timing finds that the time between the current time and the time when the business voucher is collected is greater than the second preset time, and there is no fourth image matching the facial image of the pending business person in the fourth image captured by the fourth image capture device corresponding to each business processing window, it means that the pending business person has not started to handle the business after waiting for a long time. At this time, the position of the voucher identifier in the business processing queue can be adjusted based on the preset priority adjustment strategy. Specifically, the preset priority adjustment strategy can take into account multiple factors, such as the urgency of the business processing, the busyness of the business processing window, the waiting time of the pending business person, etc. Adjusting the position of the voucher identifier can be achieved by modifying the data structure in the business processing queue, such as moving the voucher identifier to the front or back end of the queue, optimizing the scheduling order of the business processing queue, improving the business processing efficiency, and reducing the waiting time of the pending business person.

[0049] The embodiment of the present application uses image acquisition and processing technology in combination with business processing procedures to achieve effective management of business personnel to be processed and reasonable scheduling of business processing credentials, which helps to improve business processing efficiency and reduce the waiting time of business personnel to be processed.

[0050] In an embodiment of the present application, optionally, the method also includes: inputting the fourth image into a preset face extraction model, and extracting a target face from the fourth image through the preset face extraction model; based on the target face, extracting the emotional features corresponding to the target face through a preset emotional feature extraction model, and calculating the feature similarity between the emotional features and each preset emotional feature in a preset emotional feature database, and taking the emotional category corresponding to the preset emotional feature with the highest similarity as the target emotion corresponding to the target face; when the target emotion matches any preset alarm emotion, the window identifier of the business processing window corresponding to the fourth image, and the target emotion, are sent to the preset supervision terminal.

[0051] In this embodiment, the emotions of the business processing personnel while handling business can also be identified. Specifically, the fourth image captured by the fourth image acquisition device corresponding to the business processing window is input into a preset face extraction model. The preset face extraction model is used to extract the target face from the fourth image. The preset face extraction model can be a trained deep learning model that can accurately extract the facial area from the image. This model can adopt an architecture such as a convolutional neural network (CNN) and has powerful feature extraction and classification capabilities.

[0052] Next, based on the extracted target face, the emotional features corresponding to the target face are extracted using a preset emotional feature extraction model. The preset emotional feature extraction model is also a deep learning model specifically designed to extract emotion-related features from facial images. These features include muscle movements of facial expressions, the shape and position of the eyes, the degree of mouth opening, and so on. Furthermore, the feature similarity between the extracted emotional features and each preset emotional feature in the preset emotional feature database is calculated. The preset emotional feature database stores a variety of preset emotional features and their corresponding emotional categories, such as happiness, sadness, anger, surprise, etc. These preset emotional features are obtained through training with a large amount of labeled data. The emotional category corresponding to the preset emotional feature with the highest similarity is then used as the target emotion corresponding to the target face. Feature similarity calculations can use methods such as cosine similarity and Euclidean distance to measure the similarity between the extracted emotional features and the preset emotional features.

[0053] After the target emotion is determined, the target emotion is matched with the preset alarm emotion. When the target emotion matches any preset alarm emotion, the alarm operation is executed. Among them, the preset alarm emotions can be some emotion categories that require special attention set in advance, such as anger, sadness, etc. These emotions may indicate that the customer is dissatisfied with or has doubts about the business processing process. Subsequently, the window logo of the business processing window corresponding to the fourth image and the target emotion are sent to the preset supervision terminal, so that relevant personnel can be notified to pay attention to abnormal emotional states, take timely measures to improve business processing services, and improve customer satisfaction.

[0054] In an embodiment of the present application, optionally, the method further includes: acquiring a fifth image of the third target area by a fifth image acquisition device in the business hall, and splitting the fifth image into a plurality of sub-images; for each sub-image, calculating the similarity between the sub-image and the adjacent sub-image in terms of target indicators, and when the similarity is greater than a preset similarity threshold, merging the sub-image with the adjacent sub-image to obtain a merged sub-image, and outputting the merged sub-image; performing size normalization adjustment on each sub-image and each merged sub-image respectively, and inputting the normalized images into a preset classification model, and outputting the contents of each adjusted image. category of the item; based on the preset alarm category, identify whether there is a target alarm category in the category, and when the target alarm category exists, count the number of times the target alarm category appears; if the number of appearances is greater than once, obtain the detection frame corresponding to each target alarm category, when there are overlapping detection frames, determine the confidence corresponding to each target detection frame in the overlapping detection frame, retain the target detection frame with the largest confidence, and mark the target alarm item on the fifth image based on the retained target detection frame; if the number of appearances is once, mark the target alarm item on the fifth image based on the target detection frame corresponding to the target alarm category.

[0055] In this embodiment, it is also possible to identify whether there are any dangerous items in the current business hall, such as controlled knives, sticks, flames, smoke, etc. Specifically, a fifth image of the third target area is captured by a fifth image acquisition device in the business hall. Here, the fifth image acquisition device can be a high-definition camera installed on the ceiling or wall of the business hall, responsible for capturing the image of the third target area. The installation position of the fifth image acquisition device can be determined according to actual needs. Subsequently, the captured fifth image is split into multiple sub-images. Image splitting can be achieved through image processing algorithms, such as a grid-based splitting method, which evenly divides the image into multiple sub-images for subsequent processing.

[0056] For each sub-image, the similarity between the sub-image and the adjacent sub-image in terms of the target indicator is calculated. When the similarity is greater than the preset similarity threshold, the sub-image is merged with the adjacent sub-image to obtain a merged sub-image, and the merged sub-image is output. Among them, the target indicator can be an image feature such as color, texture, shape, etc., which is used to measure the similarity between sub-images. The similarity calculation can use a feature matching algorithm such as SIFT (Scale Invariant Feature Transform) or SURF (Speeded Robust Features). The preset similarity threshold is an empirical value used to judge whether the sub-images are similar enough to decide whether to merge them. It should be noted that sub-image merging can be a multi-round process. For example, the first round merges the split sub-images, the second round merges the merged sub-images again, and the third round merges the sub-images merged in the second round... until no sub-images can be merged.

[0057] Each sub-image, as well as each merged sub-image, is then resized and fed into a pre-set classification model, which outputs the category of the object contained in each resized image. Resizing can be achieved using an image scaling algorithm, such as bilinear interpolation or bicubic interpolation, to resize the sub-images to a uniform size. The pre-set classification model can be a trained machine learning model, such as a convolutional neural network (CNN), that accurately classifies objects in the image.

[0058] Furthermore, based on the preset alarm categories, the system identifies whether the target alarm category exists in the classification results. For example, preset alarm categories may include controlled knives, sticks, flames, smoke, etc. If a target alarm category exists, the number of occurrences of the target alarm category is counted. This count is used to determine the frequency of the alarm items and provide a basis for subsequent processing.

[0059] If the target warning category appears more than once, the detection frame corresponding to each target warning category is obtained. For example, if the classification results indicate that the categories corresponding to these sub-images and the merged sub-images contain controlled knives, and the categories corresponding to multiple images are all controlled knives, this may indicate that the same controlled knives are contained in multiple images (sub-images and / or merged sub-images). In this case, the detection frames in each image classified as controlled knives can be obtained and a determination can be made as to whether these detection frames contain overlapping areas, that is, whether there are overlapping detection frames. If overlapping detection frames exist, it indicates that the same controlled knives are contained in multiple images (sub-images and / or merged sub-images). The confidence level of each target detection frame in the overlapping detection frames is then determined. The target detection frame with the highest confidence level is retained, and based on the retained target detection frame, the target warning item is marked on the fifth image. Here, the detection frame is one of the outputs of a preset classification model, used to identify the location and range of the item in the image. The confidence level is the classification model's confidence level in the classification result of the item within the detection frame. Overlapping detection frames can be processed using the non-maximum suppression (NMS) algorithm to eliminate redundant detection frames. In the case where the target warning object appears multiple times, by processing overlapping detection frames, it is ensured that each target warning object is accurately marked on the fifth image to avoid repeated labeling.

[0060] If the target warning category appears only once, there is no need to process the overlapping detection frames, and the target warning item is directly marked on the fifth image based on the target detection frame corresponding to the target warning category.

[0061] The embodiment of the present application realizes automatic classification of items in specific areas of a business hall, identification of alarm items, and accurate marking of alarm items on images through steps such as image acquisition, splitting, merging, size normalization, classification, alarm category identification, frequency statistics, and target alarm item marking, which helps to timely discover potential dangers in the business hall.

[0062] Further, as Figure 1 The specific implementation of the method, the embodiment of the present application provides a business hall intelligent loss detection and comprehensive prompt device, such as Figure 2 As shown, the device includes:

[0063] an image acquisition module, configured to acquire a first image of a first target area through a first image acquisition device in the business hall, and determine whether there is a foreground object in the first image based on a preset background image corresponding to the first target area;

[0064] a tracking object determination module, configured to, when a foreground object exists in the first image, remove a human object from the foreground object to obtain a target tracking object, and create a tracking thread corresponding to the target tracking object;

[0065] a monitoring module, configured to monitor, through the tracking thread, the position and status of the target tracking object based on subsequent images of the first target area, and, when a monitoring duration reaches a first preset duration, determine whether the target tracking object is a lost object based on position monitoring results and status monitoring results within the first preset duration;

[0066] a marking module for marking the target tracking object in any image when the target tracking object is a lost object, sending any marked image to a preset supervision terminal, and destroying the tracking thread, wherein the any image is one of the first image or the subsequent image.

[0067] Optionally, the tracking object determination module is configured to:

[0068] The foreground image corresponding to the foreground object is input into a preset detection model to identify whether the foreground object contains a human object. If the foreground object contains a human object, a bounding box corresponding to the human object is generated, and the foreground object outside the bounding box is used as the target tracking object.

[0069] Optionally, the device further comprises a rejection module; the rejection module is configured to:

[0070] After taking the foreground object outside the bounding box as the target tracking object, the relative position between the target tracking object and the human object is identified, and based on the relative position, the confidence that the target tracking object belongs to the human object is calculated. When the confidence is greater than a preset confidence threshold, the target tracking object is eliminated.

[0071] Optionally, the device further includes a gesture recognition module; the gesture recognition module is configured to:

[0072] Capturing a second image of a second target area by a second image acquisition device in the business hall, and extracting a human figure area from the second image;

[0073] Determining a first length of a person contained in the person region in a first direction and a second length in a second direction, and determining a minimum bounding box corresponding to the person region, calculating a length ratio corresponding to the person region based on the first length and the second length, and calculating an area and a perimeter corresponding to the minimum bounding box;

[0074] Determining first posture information corresponding to the person using a first recognition model based on the length ratio, the area and perimeter corresponding to the minimum bounding box;

[0075] When the first posture information indicates that the person is in a non-standing posture, determining, based on the extracted person region, Zernike moment features, an area corresponding to the person region, and an outer contour perimeter, and determining, using a second recognition model, second posture information corresponding to the person based on the Zernike moment features, the area corresponding to the person region, and the outer contour perimeter;

[0076] Based on the second posture information, the abnormal posture type of the person is determined, and the abnormal posture type and the area identifier corresponding to the second target area are sent to the preset monitoring terminal.

[0077] Optionally, the device further includes an adjustment module; the adjustment module is configured to:

[0078] After receiving the third image uploaded by the third image acquisition device corresponding to the business processing voucher collection device, extract the facial image corresponding to the person to be processed from the third image, record the time when the business processing voucher corresponding to the person to be processed was collected, and add the voucher identifier corresponding to the business processing voucher to the business processing queue;

[0079] If the duration between the current time and the time for collecting the business processing certificate is greater than the second preset time, and there is no fourth image matching the facial image of the person to be processed in the fourth image captured by the fourth image acquisition device corresponding to each business processing window, then based on the preset priority adjustment strategy, the position of the certificate identifier in the business processing queue is adjusted.

[0080] Optionally, the device further includes an emotion recognition module; the emotion recognition module is configured to:

[0081] Inputting the fourth image into a preset face extraction model, and extracting a target face from the fourth image using the preset face extraction model;

[0082] Based on the target face, extracting the emotional features corresponding to the target face through a preset emotional feature extraction model, and calculating the feature similarity between the emotional features and each preset emotional feature in a preset emotional feature database, and taking the emotional category corresponding to the preset emotional feature with the highest similarity as the target emotion corresponding to the target face;

[0083] When the target emotion matches any preset alarm emotion, the window identifier of the business processing window corresponding to the fourth image and the target emotion are sent to the preset monitoring terminal.

[0084] Optionally, the device further includes an alarm object identification module; the alarm object identification module is configured to:

[0085] capturing a fifth image of the third target area by a fifth image capturing device in the business hall, and splitting the fifth image into a plurality of sub-images;

[0086] For each subgraph, calculate the similarity between the subgraph and the adjacent subgraph in terms of the target indicator; when the similarity is greater than a preset similarity threshold, merge the subgraph with the adjacent subgraph to obtain a merged subgraph, and output the merged subgraph;

[0087] Normalize the size of each sub-image and each merged sub-image, input the normalized images into the preset classification model, and output the category of the object contained in each adjusted image;

[0088] Based on the preset alarm categories, identifying whether the target alarm category exists in the categories, and if the target alarm category exists, counting the number of occurrences of the target alarm category;

[0089] If the number of occurrences is greater than one, obtaining a detection frame corresponding to each target warning category. If there are overlapping detection frames, determining the confidence level of each target detection frame in the overlapping detection frames, retaining the target detection frame with the highest confidence level, and marking the target warning item on the fifth image based on the retained target detection frames.

[0090] If the number of occurrences is one, the target warning item is marked on the fifth image based on the target detection frame corresponding to the target warning category.

[0091] It should be noted that for other corresponding descriptions of the functional units involved in the intelligent lost item detection and comprehensive prompt device for a business hall provided in the embodiment of the present application, please refer to Figure 1 The corresponding description in the method will not be repeated here.

[0092] The present application also provides a computer device, which can be a personal computer, a server, a network device, etc. Figure 3 As shown, the computer device includes a bus, a processor, a memory, and a communication interface, and may also include an input / output interface and a display device. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store location information. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the steps of each method embodiment are implemented.

[0093] Those skilled in the art will understand that Figure 3The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0094] In one embodiment, a computer-readable storage medium is provided. The computer-readable storage medium may be non-volatile or volatile, and stores a computer program thereon. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0095] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0096] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0097] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0098] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0099] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for intelligent lost item detection and comprehensive prompting in a business hall, characterized in that: include: Capturing a first image of a first target area by a first image acquisition device in the business hall, and determining whether there is a foreground object in the first image based on a preset background image corresponding to the first target area; In the case where there is a foreground object in the first image, removing the human object from the foreground object to obtain a target tracking object, and creating a tracking thread corresponding to the target tracking object; monitoring, by the tracking thread, a position and a status of the target tracking object based on subsequent images of the first target area, and determining whether the target tracking object is a lost object based on position monitoring results and status monitoring results within the first preset time period when the monitoring time period reaches a first preset time period; When the target tracking object is a lost object, the target tracking object is marked in any image, any marked image is sent to a preset supervision terminal, and the tracking thread is destroyed, wherein the any image is one of the first image or the subsequent image.

2. The method according to claim 1, characterized in that The step of removing the human object from the foreground object to obtain the target tracking object includes: The foreground image corresponding to the foreground object is input into a preset detection model to identify whether the foreground object contains a human object. If the foreground object contains a human object, a bounding box corresponding to the human object is generated, and the foreground object outside the bounding box is used as the target tracking object.

3. The method according to claim 2, characterized in that After taking the foreground object outside the bounding box as a target tracking object, the method further includes: Identify the relative position between the target tracking object and the human object, and based on the relative position, calculate the confidence that the target tracking object belongs to the human object. When the confidence is greater than a preset confidence threshold, eliminate the target tracking object.

4. The method according to claim 1, wherein The method further comprises: Capturing a second image of a second target area by a second image acquisition device in the business hall, and extracting a human figure area from the second image; Determining a first length of a person contained in the person region in a first direction and a second length in a second direction, and determining a minimum bounding box corresponding to the person region, calculating a length ratio corresponding to the person region based on the first length and the second length, and calculating an area and a perimeter corresponding to the minimum bounding box; Determining first posture information corresponding to the person using a first recognition model based on the length ratio, the area and perimeter corresponding to the minimum bounding box; When the first posture information indicates that the person is in a non-standing posture, determining, based on the extracted person region, Zernike moment features, an area corresponding to the person region, and an outer contour perimeter, and determining, using a second recognition model, second posture information corresponding to the person based on the Zernike moment features, the area corresponding to the person region, and the outer contour perimeter; Based on the second posture information, the abnormal posture type of the person is determined, and the abnormal posture type and the area identifier corresponding to the second target area are sent to the preset monitoring terminal.

5. The method according to claim 1, wherein The method further comprises: After receiving the third image uploaded by the third image acquisition device corresponding to the business processing voucher collection device, extract the facial image corresponding to the person to be processed from the third image, record the time when the business processing voucher corresponding to the person to be processed was collected, and add the voucher identifier corresponding to the business processing voucher to the business processing queue; If the duration between the current time and the time for collecting the business processing certificate is greater than the second preset time, and there is no fourth image matching the facial image of the person to be processed in the fourth image captured by the fourth image acquisition device corresponding to each business processing window, then based on the preset priority adjustment strategy, the position of the certificate identifier in the business processing queue is adjusted.

6. The method according to claim 5, characterized in that The method further comprises: Inputting the fourth image into a preset face extraction model, and extracting a target face from the fourth image using the preset face extraction model; Based on the target face, extracting the emotional features corresponding to the target face through a preset emotional feature extraction model, and calculating the feature similarity between the emotional features and each preset emotional feature in a preset emotional feature database, and taking the emotional category corresponding to the preset emotional feature with the highest similarity as the target emotion corresponding to the target face; When the target emotion matches any preset alarm emotion, the window identifier of the business processing window corresponding to the fourth image and the target emotion are sent to the preset monitoring terminal.

7. The method according to claim 1, characterized in that The method further comprises: capturing a fifth image of the third target area by a fifth image capturing device in the business hall, and splitting the fifth image into a plurality of sub-images; For each subgraph, calculate the similarity between the subgraph and the adjacent subgraph in terms of the target indicator; when the similarity is greater than a preset similarity threshold, merge the subgraph with the adjacent subgraph to obtain a merged subgraph, and output the merged subgraph; Normalize the size of each sub-image and each merged sub-image, input the normalized images into the preset classification model, and output the category of the object contained in each adjusted image; Based on the preset alarm categories, identifying whether the target alarm category exists in the categories, and if the target alarm category exists, counting the number of occurrences of the target alarm category; If the number of occurrences is greater than one, obtaining a detection frame corresponding to each target warning category. If there are overlapping detection frames, determining the confidence level of each target detection frame in the overlapping detection frames, retaining the target detection frame with the highest confidence level, and marking the target warning item on the fifth image based on the retained target detection frames. If the number of occurrences is one, the target warning item is marked on the fifth image based on the target detection frame corresponding to the target warning category.

8. An intelligent lost item detection and comprehensive prompting device for a business hall, characterized in that: include: an image acquisition module, configured to acquire a first image of a first target area through a first image acquisition device in the business hall, and determine whether there is a foreground object in the first image based on a preset background image corresponding to the first target area; a tracking object determination module, configured to, when a foreground object exists in the first image, remove a human object from the foreground object to obtain a target tracking object, and create a tracking thread corresponding to the target tracking object; a monitoring module, configured to monitor, through the tracking thread, the position and status of the target tracking object based on subsequent images of the first target area, and, when a monitoring duration reaches a first preset duration, determine whether the target tracking object is a lost object based on position monitoring results and status monitoring results within the first preset duration; a marking module for marking the target tracking object in any image when the target tracking object is a lost object, sending any marked image to a preset supervision terminal, and destroying the tracking thread, wherein the any image is one of the first image or the subsequent image.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images