Multi-channel load balancing asynchronous target detection method, storage medium and processor

Abstract

The present invention provides a multi-channel load balancing asynchronous target detection method, storage medium and processor. The method includes: S11, the dealer monitors the port occupied by the video sending module to send the image frame, and obtains the input image; S12, uses the multi-channel balanced load and queue of ZMQ to share the image frame, and distributes the received image to the multi-process YOLO module opened concurrently; S13, each YOLO process processes the distributed image frame, and detects the target attributes in the image; S14, through the multi-channel balanced load of ZMQ, and the queue shares the detection result, and returns it to the downstream recognition module. Start multiple YOLO processes, share process information through queues, dictionaries, etc., can simultaneously process multiple video inputs in real time, and can make full use of hardware resources through load balancing, which can be generally applied to target recognition such as YOLO monitoring system, facenet, tinyface, etc.

Description

Technical Field

[0001] The present invention relates to the field of target recognition technology, and more specifically to a multi-path load balancing asynchronous target detection method, a storage medium and a processor. Background Art

[0002] Object detection can be used in many fields such as security, industry, and assisted driving. For example, in the field of security, people can be counted in key areas to prevent crowding and trampling, and abnormal object detection can be performed in sensitive areas to prevent regional intrusions. At the same time, object detection is also the upstream input of visual technologies such as object recognition, instance segmentation, and morphological analysis. The effectiveness of object detection directly determines the results of these more complex task processing.

[0003] The object detection technology based on computer vision is to give an understanding of the foreground and background of an image given an image. We need to separate the object of interest from the background and determine the classification and position of the object. Therefore, the output of the object detection is a list, and each data group element of the list gives the classification and position of a detected object. It is easy for humans to locate and classify objects from a given image, but for computers, it is more difficult to obtain the abstract concepts of target objects such as "people" and "cars" from a multi-channel pixel matrix and determine their positions. In addition, when multiple targets overlap or are covered, object detection becomes more difficult.

[0004] Currently, the mainstream target detection method is based on deep learning, and the multi-process YOLO target detection solution in PUSH / PULL mode achieves a good balance between speed and accuracy.

[0005] YOLO is the pioneer of end-to-end models. It is named for its speed because it only needs one process to obtain the target location and classification. Compared with the two-stage method, its real-time speed is impressive. The training process of YOLO_V3 is as follows:

[0006] 1. Prepare data: scale the image to the specified size and input it into the network; the feature extraction network will output feature maps of 3 different scales [13*13, 26*26, 52*52], and match the samples to different anchors according to IOU [only considering the area, not the position] [yolo_v3 sets 3 anchors for each feature map, so there are 9 anchors in total];

[0007] 2. Convolutional network: Darknet-53 is used to extract features. Each grid in the feature map of three scales predicts a conditional probability value for each category, and generates B boxes on each grid. Each box predicts [num_anchors*(5+num_classes+hum_properties)] regression values, that is, the output of the bounding box x, y, width, height, configuration score, and classes scores under different anchors. During testing, the scores are calculated as follows:

[0008]

[0009] The first term on the left side of the equation is predicted by the grid, and the last two terms are predicted by each box, and the scores of each box containing objects of different categories are obtained in the form of conditional probability. Therefore, the total number of predicted values ​​output by the convolutional network is S×S×(B×5+C), where S is the number of grids, B is the number of boxes generated for each grid, and C is the number of categories;

[0010] When YOLO_V3 makes forward predictions, the input is an image of any size. When the final detection is output, NMS (Non-Maximum Suppression) filtering is required to obtain the final prediction box.

[0011] When using ZMQ's PUSH / PULL mode, a server is responsible for PUSH data, and multiple clients queue up to PULL data. A piece of data pushed by the server will only be pulled by one of the clients. If either the server or the client is disconnected, it will not affect the other.

[0012] When applied to the YOLO target detection system, in order to improve the real-time performance of detection, the video provider module is used as the server and the YOLO detection module is used as the client. Multiple YOLO detection processes can be started at the same time, which can detect targets faster.

[0013] It can be seen that its shortcomings are:

[0014] (1) When the YOLO module is working, it just waits in a mechanical queue without performing load balancing;

[0015] (2) It is difficult to implement when multi-channel video detection is enabled;

[0016] (3) When the server pushes messages, they tend to accumulate, which may cause data loss, which is absolutely not allowed. Summary of the invention

[0017] The technical problem to be solved by the present invention is to provide a multi-channel load balancing asynchronous target detection method, a storage medium and a processor, which can start multiple YOLO processes, share process information through queues, dictionaries, etc., can simultaneously process multiple video inputs in real time, and can make full use of hardware resources through load balancing. The method can be generally applied to target recognition such as YOLO monitoring systems, facenet, tinyface, etc.

[0018] To solve the above technical problems, on the one hand, an embodiment of the present invention provides a multi-path load balancing asynchronous target detection method, comprising:

[0019] S11, the dealer monitors the port occupied by the video sending module to send the image frame and obtains the input image;

[0020] S12, use ZMQ's multi-channel load balancing and queue sharing to distribute the received images to the concurrently enabled multi-process YOLO modules;

[0021] S13, a multi-process YOLO module processes the distributed image frames and detects the target attributes in the image;

[0022] S14, through ZMQ's multi-channel load balancing and queue sharing detection results, it is returned to the downstream identification module.

[0023] Preferably, the dealer monitors the port occupied by the video sending module to send the image frame, and obtaining the input image includes:

[0024] The video sending module receives the video file from the camera and processes the video file from the camera.

[0025] Preferably, the multi-channel load balancing and queue sharing of image frames using ZMQ includes:

[0026] The input image is shared in the multi-way balanced load and queue of ZMQ and is processed by the multi-way balanced load and queue of ZMQ.

[0027] Preferably, the multi-process YOLO module processes and distributes the obtained image frames, including: the multi-process YOLO module receives the image frames distributed by the multi-path balanced load of ZMQ.

[0028] Preferably, the target attributes in the image detected by the multi-process YOLO module include: facial attributes.

[0029] Preferably, the target attributes in the image detected by the multi-process YOLO module include: license plate attributes.

[0030] Preferably, the facial attributes include: gender attribute, race attribute, age attribute, and facial decoration.

[0031] Preferably, the license plate attributes include: vehicle model, year, and vehicle body color.

[0032] On the other hand, an embodiment of the present invention provides a storage medium, wherein the storage medium includes a stored program, wherein the program executes the above-mentioned multi-path load balancing asynchronous target detection method when running.

[0033] On the other hand, an embodiment of the present invention provides a processor, which is used to run a program, wherein the program executes the above-mentioned multi-path load balancing guide step target detection method when running.

[0034] Compared with the prior art, the above technical solution has the following advantages: multiple YOLO processes are started, and process information is shared through queues, dictionaries, etc., so that multiple video inputs can be processed in real time at the same time. Hardware resources can be fully utilized through load balancing, and it can be widely used in target recognition such as YOLO monitoring system, facenet, tinyface, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 It is a schematic diagram of a load target detection method in the prior art.

[0037] Figure 2 It is a flow chart of the load balancing guide step target detection method of the present invention.

[0038] Figure 3 It is an analytical diagram of realizing multi-path load balancing through an intermediate proxy in ZMQ in the load balancing asynchronous target detection method of the present invention.

[0039] Figure 4 yes Figure 3 An embodiment of .

[0040] Figure 5 yes Figure 3 Another embodiment of . DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] Embodiment 1

[0043] Target detection is done by first identifying the target and then detecting it from a large number of targets. Target recognition refers to the process of distinguishing a specific target (or a type of target) from other targets (or other types of targets). It includes both the recognition of two very similar targets and the recognition of one type of target from other types of targets. Figure 1 It is a schematic diagram of the principle of the multi-process target detection method in the prior art. Figure 1 way.

[0044] Figure 2 : is a flow chart of the load balancing step target detection method of the present invention. Figure 2 As shown, a multi-channel load balancing asynchronous target detection method includes: S11, the dealer monitors the port occupied by the video sending module to send the image frame, and obtains the input image; S12, uses the multi-channel balanced load and queue of ZMQ to share the image frame, and distributes the received image to the concurrently opened multi-process YOLO module; S13, the multi-process YOLO module processes the distributed image frame, and detects the target attributes in the image; S14, returns the detection result to the downstream recognition module through the multi-channel balanced load of ZMQ and the queue sharing.

[0045] The dealer monitors the port occupied by the video sending module to send the image frame, and obtains the input image, including: the video sending module receives the video file from the camera and processes the video file from the camera. The multi-channel balanced load and queue sharing image frame of ZMQ includes: the input image is shared in the multi-channel balanced load and queue of ZMQ, and is processed by the multi-channel balanced load and queue of ZMQ. The multi-process YOLO module processes and distributes the obtained image frame, including: the multi-process YOLO module receives the image frame distributed by the multi-channel balanced load of ZMQ. The target attributes detected by the multi-process YOLO module in the image include: facial attributes or license plate attributes. Facial attributes include but are not limited to: gender attributes, ethnic attributes, age attributes, facial decorations, etc. License plate attributes include but are not limited to: car model, year, body color, etc. . The downstream facial recognition module extracts features from the detected image, and then compares it with the feature library of the human face to see how much difference they have. If the difference is less than a certain threshold, it is considered to be a certain person. The downstream license plate recognition module uses a deep learning method to perform OCR (optical character recognition) recognition, and then recognizes the license plate information.

[0046] Figure 3 This is a parsing diagram of multi-channel load balancing achieved through an intermediate proxy in ZMQ in the load balancing step target detection method of the present invention. There are multiple video inputs, and multiple YOLO detection processes are also opened; the video input acts as the client, and the YOLO detection process acts as the server; through ZMQ's multi-channel load balancing, a reasonable multi-process asynchronous communication that fully utilizes hardware resources is achieved; the server and the client are not directly connected, but through ROUTER [router] and DEALER [dealer]. All requests are queued fairly after arriving at the router, and then sent to the server after load balancing by the dealer, and the server's response result is returned to the client by the dealer and the router.

[0047] Figure 4 yes Figure 3 An embodiment of . Figure 5 yes Figure 3 Another embodiment of Figure 4 , Figure 5As shown in the figure, the image frame information of the DEALER port is received, and the targets such as faces and license plates are detected; the DEALER and ROUTER communication module: handles the asynchronous load balancing communication between the video sending module, the YOLO detection module, and the face recognition module; the face information sending module: extracts the face information from the detection results of the YOLO detection module, encodes it into a format suitable for network transmission, and sends it to the face recognition module; the license plate recognition sending module: extracts the license plate information from the detection results of the YOLO detection module, encodes it into a format suitable for network transmission, and sends it to the license plate recognition module. The YOLO detection module, the face information sending module, the DEALER and ROUTER communication module, and the vehicle information sending module interact with each other through communication methods such as queues and dictionaries.

[0048] Embodiment 2

[0049] An embodiment of the present invention further provides a storage medium, which includes a stored program, wherein the program executes the above-mentioned multi-path load balancing asynchronous target detection method process when running.

[0050] Optionally, in this embodiment, the storage medium may be configured to store program codes for executing the following facial attribute recognition method flow:

[0051] S11, the dealer monitors the port occupied by the video sending module to send the image frame and obtains the input image;

[0052] S12, use ZMQ's multi-channel load balancing and queue sharing to distribute the received images to the concurrently enabled multi-process YOLO modules;

[0053] S13, a multi-process YOLO module processes the distributed image frames and detects the target attributes in the image;

[0054] S14, through ZMQ's multi-channel load balancing and queue sharing detection results, it is returned to the downstream identification module.

[0055] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0056] It can be seen that by adopting the storage medium of the present invention, multiple YOLO processes are started, and process information is shared through queues, dictionaries, etc., multiple video inputs can be processed in real time at the same time, and hardware resources can be fully utilized through load balancing. It can be widely used in target recognition systems such as YOLO and tinyface detection systems.

[0057] Embodiment 3

[0058] An embodiment of the present invention further provides a processor, which is used to run a program, wherein the program executes the steps in the above-mentioned multi-path load balancing asynchronous target detection method when running.

[0059] Optionally, in this embodiment, the above program is used to perform the following steps:

[0060] S11, the dealer monitors the port occupied by the video sending module to send the image frame and obtains the input image;

[0061] S12, use ZMQ's multi-channel load balancing and queue sharing to distribute the received images to the concurrently enabled multi-process YOLO modules;

[0062] S13, a multi-process YOLO module processes the distributed image frames and detects the target attributes in the image;

[0063] S14, through ZMQ's multi-channel load balancing and queue sharing detection results, it is returned to the downstream identification module.

[0064] Optionally, the specific examples in this embodiment may refer to the above embodiments and the examples described in the specific implementation, and this embodiment will not be described in detail here.

[0065] It can be seen that by adopting the processor of the present invention, multiple YOLO processes are started, and process information is shared through queues, dictionaries, etc., multiple video inputs can be processed in real time at the same time, and hardware resources can be fully utilized through load balancing. It can be widely used in detection systems such as YOLO and tinyface, and target recognition systems such as facenet.

[0066] From the above description, it can be seen that by using the multi-channel load-balanced step-by-step target detection method, storage medium and processor according to the present invention, multiple YOLO processes are started, and process information is shared by queues, dictionaries, etc., so that multiple video inputs can be processed in real time at the same time. Through load balancing, hardware resources can be fully utilized, and it can be generally applicable to detection systems such as YOLO, facenet, and target recognition systems such as tinyface.

[0067] The embodiments of the present invention are described in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A multi-path load balancing asynchronous target detection method, characterized in that: include: S11, the dealer monitors the port occupied by the video sending module to send the image frame and obtains the input image; S12, use ZMQ's multi-channel load balancing and queue sharing to distribute the received images to the concurrently enabled multi-process YOLO modules; The sharing of image frames using the multi-way balanced load and queue of ZMQ comprises: the input image is shared in the multi-way balanced load and queue of ZMQ, and is to be processed by the multi-way balanced load and queue of ZMQ; S13, the multi-process YOLO module processes the distributed image frames to detect the target attributes in the image; the multi-process YOLO module processes the distributed image frames including: the multi-process YOLO module receives the image frames distributed by the multi-path balanced load of ZMQ; S14, through ZMQ's multi-channel load balancing and queue sharing detection results, it is returned to the downstream identification module.

2. The multi-path load balancing asynchronous target detection method according to claim 1, characterized in that: The dealer monitors the port occupied by the video sending module to send the image frame, and obtains the input image including: The video sending module receives the video file from the camera and processes the video file from the camera.

3. The multi-path load balancing asynchronous target detection method according to claim 1, characterized in that: The target attributes in the image detected by the multi-process YOLO module include: facial attributes.

4. The multi-path load balancing asynchronous target detection method according to claim 1, characterized in that: The multi-process YOLO module detects and obtains target attributes in the image including: license plate attributes.

5. The multi-path load balancing asynchronous target detection method according to claim 3, characterized in that: The facial attributes include: gender attribute, race attribute, age attribute, and facial decoration.

6. The multi-path load balancing asynchronous target detection method according to claim 4, characterized in that: The license plate attributes include: vehicle model, year, and vehicle body color.

7. A storage medium, characterized in that: The storage medium includes a stored program, wherein the program executes the multi-path load balancing asynchronous target detection method according to any one of claims 1 to 6 when running.

8. A processor, characterized in that: The processor is used to run a program, wherein the program executes the multi-path load balancing asynchronous target detection method according to any one of claims 1 to 6 when running.

Images