Queuing order detection method and device, electronic equipment and medium

Abstract

The invention relates to a queuing order detection method and apparatus, an electronic device and a medium. The method comprises the steps of obtaining a real-time video of a team in a queuing process; identifying a figure image in each frame of image in the obtained real-time video by using the constructed deep learning neural network to obtain position information of all people in each frame of image; performing straight line fitting on the position information of the owner in each frame of image to obtain a team fitting straight line; acquiring the vertical distance between the position information of each person in each frame of image and the team fitting straight line; when the vertical distance between the personal position information and the fitting straight line in the team is out of the threshold range, an alarm is given out for reminding, and non-contact intelligent monitoring on the queuing order of the crowd is realized; the recognition accuracy of targets such as personnel is improved; through deep learning, when position information confirmation is carried out, identification is carried out through a deep learning neural network, a certain self-learning capability is possessed, and adaptability is stronger.

Description

Technical Field

[0001] The present application relates to the field of computer information processing technology, and in particular to a queue orderliness detection method, device, electronic equipment and medium. Background Art

[0002] In order to effectively provide data support for managers' decision-making and to further adjust and optimize the queue processing process, queue detection technology came into being.

[0003] In the prior art, the relevant queue detection technology is as follows: at least two cameras are used to respectively shoot the head and tail of the same queue, and the image of each camera is obtained; the human features in each camera image are detected and extracted, and a human feature to be matched is obtained from the head feature set in turn, and then each human feature to be matched is matched with the unmatched human features in the tail feature set in turn; and the queue status information of the queue is obtained according to the matching results. However, this method mainly detects the queue-jumping behavior of the queue in an orderly manner, but does not complete the detection of the formation of the queue;

[0004] Currently, in order to detect the orderliness (formation) of queues, image recognition technology can be used to identify and track individual objects in images. However, these technologies still face challenges when dealing with large-scale crowd dynamics, especially in complex or crowded environments. They have the following disadvantages: a. A large number of cameras are required to cover the entire area, which increases the cost and complexity of the system; b. It is susceptible to external interference due to complex environments; c. The real-time monitoring effect of the orderliness of group queues is limited. Summary of the invention

[0005] In view of the above-mentioned problems, the present invention provides a method, device, electronic equipment and medium for detecting the orderliness of a queue of a crowd, which can be non-contacted, intelligently monitored and have high detection accuracy.

[0006] The present application is implemented through the following technical solutions, a queue orderliness detection method, including:

[0007] Get real-time video of the queue during the queuing process;

[0008] The constructed deep learning neural network is used to identify the person images in each frame of the acquired real-time video, and the location information of all people in each frame is obtained;

[0009] Perform a straight line fitting on the position information of all persons in each frame of the image to obtain a team fitting straight line;

[0010] The vertical distance between the position information of each person in each frame of the image and the fitting straight line of the team is obtained. When the vertical distance between the position information of an individual in the team and the fitting straight line is outside the threshold range, an alarm is issued.

[0011] Preferably, after acquiring the real-time video image, the acquired video image is processed according to the following method:

[0012] Acquire each frame of the video image, preprocess each frame, and then obtain the preprocessed image;

[0013] The obtained preprocessed image is input into the convolutional neural network to obtain the crowd image in the preprocessed image.

[0014] Preferably, the method for obtaining the location information of all persons in each frame of the image includes:

[0015] Constructing a deep learning network model, and training the constructed deep learning network model to obtain a trained deep learning network model;

[0016] The crowd image in each frame is input into the trained deep learning network model to obtain the location information of all people in each frame.

[0017] Preferably, the method for obtaining the team fitting line includes fitting the position information of all persons in each frame of the image using a polynomial fitting algorithm to obtain the team fitting line.

[0018] Preferably, the method for obtaining real-time video of the queue during the queuing process includes: using a high-resolution camera to collect real-time video under good lighting conditions, and using an infrared camera to collect real-time video at night or in low light conditions.

[0019] Preferably, the method of preprocessing each frame of image is image denoising or grayscale conversion.

[0020] A queue orderliness detection device, comprising:

[0021] Video acquisition module, used to collect real-time video of the queue during the queuing process;

[0022] The location information acquisition module uses the constructed deep learning neural network to identify the person images in each frame of the real-time video collected by the video acquisition module, and obtains the location information of all people in each frame;

[0023] The straight line fitting module performs straight line fitting on the position information of all persons in each frame of the image obtained by the position information acquisition module to obtain a team fitting straight line;

[0024] The reminder module obtains the vertical distance between the position information of each person in each frame image and the fitting straight line of the team. When the vertical distance between the position information of an individual in the team and the fitting straight line is outside the threshold range, an alarm reminder is issued.

[0025] An electronic device, comprising:

[0026] one or more processors;

[0027] A storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the queue orderliness detection method.

[0028] A computer readable medium stores a computer program, and when the computer program is executed by a processor, the method for detecting the orderliness of a queue is implemented.

[0029] The beneficial effects of this application are:

[0030] During the queuing process, real-time images of the queue are collected, and the collected images are used to obtain the position information of each person in the queue. A straight line is fitted using the position information, and then the distance between each person's position information and the straight line is used to determine whether someone has left the queue during the queuing process. If someone is found to have left the queue, a timely reminder is given to achieve non-contact, intelligent monitoring of the orderliness of the crowd queue; the accuracy of identifying targets such as people is improved; through deep learning, when confirming position information, it is identified through a deep learning neural network, which has a certain self-learning ability and is more adaptable. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 is a flow chart of a queue orderliness detection method in an embodiment of the present invention;

[0033] Figure 2 is a schematic diagram of the structure of a deep learning neural network in an embodiment of the present invention;

[0034] Figure 3 is a block diagram of a queue orderliness detection device in an embodiment of the present invention;

[0035] Figure 4 Schematic diagram of the structure of the computer system of the electronic device in this embodiment. DETAILED DESCRIPTION

[0036] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0037] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided so as to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, systems, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0038] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0039] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0040] A queue orderliness detection method, comprising:

[0041] Obtain real-time video of the queue during the queuing process; the video of the queuing process is collected in the following manner. Due to different weather conditions or different lighting conditions, this embodiment uses a high-definition camera or an infrared camera to collect the queuing picture. Specifically, a visible light camera (high-definition camera) is used to capture high-resolution images during the day and under good lighting conditions, and an infrared camera is used at night or under low-light conditions to ensure the continuity of 24-hour monitoring. A multi-camera system can also be formed to combine cameras from multiple angles to provide a full range of viewing angles to ensure monitoring without blind spots; this embodiment combines visible light and infrared imaging, and compared with traditional systems that only rely on a single imaging technology, it can maintain high efficiency and accuracy under various lighting conditions. This enables the system to work effectively at night or in low-light environments, thereby significantly improving the applicability and reliability of the monitoring system. The high-definition camera and infrared camera in this embodiment use conventional cameras that meet the requirements, and this embodiment does not introduce the specific models and structures in detail.

[0042] After completing the video image acquisition, since the acquired video image is the original image, it contains a lot of noise, which brings great difficulties to the later recognition. Therefore, after obtaining the acquired video, each frame of the video needs to be preprocessed. During the preprocessing, the main actions are to perform image denoising and grayscale conversion on each frame of the image. Image denoising is mainly to remove the noise in the image. The main purpose of grayscale conversion after image denoising is to retain sufficient brightness and contrast information. When performing image denoising, the methods used in this embodiment include but are not limited to mean filtering, Gaussian filtering, median filtering, Fourier transform and other methods to achieve it. The methods used for grayscale conversion include but are not limited to one or more of the averaging method, weighted averaging method, maximum value method and the like. Since image denoising and grayscale conversion are conventional means in this field, this embodiment only describes the method used, and the steps of the method are not introduced in detail.

[0043] After completing the image preprocessing, since the image contains too many factors and the main purpose of this embodiment is to detect the formation situation, only the crowd image in the image is required, and the image in the surrounding environment is removed to reduce the processing amount for post-processing. When performing crowd image recognition on each frame of the processed image, this embodiment adopts the target detection algorithm of the convolutional neural network for recognition. Specifically, during the recognition, the preprocessed image is input into the convolutional neural network to obtain the crowd image.

[0044] After obtaining the team crowd image, this embodiment constructs a deep learning neural network. The structure of the constructed deep learning neural network is as follows: Figure 2 As shown, after completing Figure 2After the deep learning neural network shown in the figure is obtained, the deep learning neural network is trained. During the training, large-scale team crowd images are collected as training sets, and the collected training is manually annotated. The annotated content is the location information of each person. Then the image is used as input and the annotated information is used as output for training to obtain a deep learning neural network after training. Then the processed team crowd image is input into the deep learning neural network to obtain the location information of each person in the crowd image. In this embodiment, a deep learning network is used to determine the location information of each person in the crowd, which can more accurately identify and track individuals in the image. Compared with traditional rule-based image processing algorithms, deep learning models show higher flexibility and accuracy in processing complex scenes and dynamic changes.

[0045] After obtaining the position information of each person in the team, this embodiment is based on the fact that the people who form the team are basically in a straight line during the queuing process. This embodiment fits and obtains a straight line based on the position information of each person in the team. The straight line can be used as the formation state that the team should maintain at present. This embodiment uses the position information of all people in each frame of the image to perform straight line fitting to obtain the team fitting straight line in each frame of the image. This embodiment adopts the method of fitting the straight line to calculate a straight line representing the orderliness of the queue of the crowd. Since the fitted straight line is an orderly straight line for the queue, it is possible to judge whether someone is in the queue by judging whether the person's position information is on the fitted straight line. The judgment method of this embodiment is to judge by the vertical distance between each person's position information and the fitted straight line. It can also be judged by Euclidean distance. When the vertical distance between the position information of each person in the graph and the fitted straight line is less than the set threshold distance, it means that the team basically maintains a straight line at this time, no one has left the team, and no reminder is given. When the vertical distance is greater than the set threshold distance, it means that someone is away from the team at this time. At this time, a reminder is issued to remind the person who has left to return to the team as soon as possible. Since this embodiment processes each frame of the image to match the distance with the fitted straight line based on the collected position information of each person, the queuing situation can be quickly fed back during the real-time image acquisition process of the camera, which improves the orderly detection of the queuing situation and can give effective reminders in the first time to avoid the situation where the team is completely formed due to the failure to complete the reminder due to individual circumstances.

[0046] In this embodiment, a polynomial fitting or regression equation is used to fit a straight line to the detected target position, and the orderliness of the crowd is quantified by calculating the minimum distance from the individual position to the fitting straight line. This method is more accurate than traditional manual judgment or simple mechanical calculation, and provides a quantifiable and repeatable evaluation standard, which helps to achieve objective and consistent evaluation.

[0047] In the specific implementation process of this embodiment, the distance is explained by taking the need to queue up to enter the entrance of a large square as an example. A high-definition camera and an infrared camera for collecting images are installed above the entrance of the square to collect video images of the queue at the entrance. Then, each frame of the collected image is preprocessed and crowd recognition is performed, and the recognized crowd image is input into a deep learning neural network for position recognition to obtain the position information of each person in the queue of the square. A fitting straight line is fitted using the acquired position information of all the people in the queue. Then, the shortest distance between the position information of each person and the fitting straight line is used to determine whether there is anyone in the team who has left the team and is outside the team. If it is determined that there are people outside the team, a reminder device such as a loudspeaker or other voice reminder device will remind the people outside the team to join the team as soon as possible. In this embodiment, the method for determining whether the person is outside the team is that when the shortest distance between the person's position information and the fitting is greater than 5, it means that the person is outside the team, and an alarm reminder is issued.

[0048] In another embodiment, the deep learning neural network is trained based on a deep learning framework such as TensorFlow or PyTorch. During image transmission, the image transmission protocol is connected to the monitoring system, and the real-time transmission of the monitoring results can adopt a communication protocol such as MQTT or HTTP.

[0049] In another embodiment, since the formation of the team will not change in a short period of time due to the departure of an individual from the team during the queuing process, in view of this, this embodiment takes the images collected within 1 minute as a unit, and performs image fitting on the personnel position information in the first frame image in the unit to obtain a fitting straight line, and all subsequent images are not subjected to straight line fitting. The fitted straight line in the first frame image is used as the standard fitting straight line of the unit, and then the distance between the personnel position information in each frame image in the unit and the standard fitting straight line is used to determine whether someone has left the team, thereby effectively reducing the amount of data processed and improving processing efficiency.

[0050] like Figure 3 As shown, this embodiment provides a queue orderliness detection device, including:

[0051] The video acquisition module is used to collect real-time video of the queue during the queuing process; specifically, the video acquisition module is a high-definition camera and an infrared camera;

[0052] The location information acquisition module uses the constructed deep learning neural network to identify the person images in each frame of the real-time video collected by the video acquisition module, and obtains the location information of all people in each frame;

[0053] The straight line fitting module performs straight line fitting on the position information of all persons in each frame of the image obtained by the position information acquisition module to obtain a team fitting straight line;

[0054] The reminder module obtains the vertical distance between the position information of each person in each frame of the image and the fitting straight line of the team. When the vertical distance between the position information of an individual in the team and the fitting straight line is outside the threshold range, an alarm reminder is issued. The transmission of all image data in the system is designed to automatically receive and process image data and output orderly results in real time; this automation and real-time performance greatly improves the response speed, especially in application scenarios that require rapid decision-making and response (such as large-scale events, public safety, etc.), showing great application value.

[0055] In one embodiment, Figure 4 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.

[0056] It should be noted that the computer system 400 of the electronic device shown in the figure is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0057] The computer system 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 402 or the program loaded from the storage part 408 to the random access memory (RAM) 403, such as executing the queue orderliness detection method in the above embodiment. In the RAM 403, various programs and data required for system operation are also stored. The CPU 401, the ROM 402 and the RAM 403 are connected to each other through the bus 404. The input / output (I / O) interface 405 is also connected to the bus 404.

[0058] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed so that a computer program read therefrom is installed into the storage section 408 as needed.

[0059] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 409, and / or installed from a removable medium 411. When the computer program is executed by a central processing unit (CPU) 401, various functions defined in the system of the present application are executed.

[0060] It should be noted that the computer-readable medium shown in the embodiment of the present application is used to implement the queue orderliness detection method; the computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (Erasab le Programmab le ReadOn l yMemory, EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable computer program. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0061] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0062] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0063] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above-mentioned various optional implementations.

[0064] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0065] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0066] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0067] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0068] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A queue orderliness detection method, characterized in that: include: Get real-time video of the queue during the queuing process; The constructed deep learning neural network is used to identify the person images in each frame of the acquired real-time video, and the location information of all people in each frame is obtained; Perform a straight line fitting on the position information of all persons in each frame of the image to obtain a team fitting straight line; Get the vertical distance between the position information of each person in each frame of the image and the team's fitting straight line; When the vertical distance between the position information of an individual in the team and the fitting straight line is outside the threshold range, an alarm is issued.

2. A queue orderliness detection method according to claim 1, characterized in that; After acquiring the real-time video image, the acquired video image is processed as follows: Acquire each frame of the video image, preprocess each frame, and then obtain the preprocessed image; The obtained preprocessed image is input into the convolutional neural network to obtain the crowd image in the preprocessed image.

3. A queue orderliness detection method according to claim 2, characterized in that: The method of obtaining the location information of all persons in each frame of the image includes: Constructing a deep learning network model, and training the constructed deep learning network model to obtain a trained deep learning network model; The crowd image in each frame is input into the trained deep learning network model to obtain the location information of all people in each frame.

4. A queue orderliness detection method according to claim 1, characterized in that: The method for obtaining the team fitting straight line includes fitting the position information of all persons in each frame of the image using a polynomial fitting algorithm to obtain the team fitting straight line.

5. A queue orderliness detection method according to claim 1, characterized in that: Methods for obtaining real-time video of the queue during the queuing process include: using a high-resolution camera to collect real-time video under good lighting conditions, and using an infrared camera to collect real-time video at night or under low light conditions.

6. A queue orderliness detection method according to claim 2, characterized in that: The method for preprocessing each frame of image is image denoising or grayscale conversion.

7. A queue orderliness detection device, characterized in that: include: Video acquisition module, used to collect real-time video of the queue during the queuing process; The location information acquisition module uses the constructed deep learning neural network to identify the person images in each frame of the real-time video collected by the video acquisition module, and obtains the location information of all people in each frame; The straight line fitting module performs straight line fitting on the position information of all persons in each frame of the image obtained by the position information acquisition module to obtain a team fitting straight line; The reminder module obtains the vertical distance between the position information of each person in each frame image and the fitting straight line of the team. When the vertical distance between the position information of an individual in the team and the fitting straight line is outside the threshold range, an alarm reminder is issued.

8. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement a queue orderliness detection method as described in any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, a queue orderliness detection method according to any one of claims 1 to 6 is implemented.

Images