Target detection and attribute identification method based on multi-path parallel

Abstract

The invention discloses a multi-path parallel-based target detection and attribute identification method, and the method comprises the steps: obtaining collected data from a binocular depth camera, and carrying out the preprocessing of the collected data, and obtaining an RGB image and a depth image; performing target detection on the RGB image and generating an RGB detection result; performing target detection on the depth map and generating a depth detection result; carrying out fusion processing on the RGB detection result and the depth detection result, and generating a fused detection result; performing target management according to the fused detection result, and generating an attribute recognition task for each target; carrying out attribute identification polling, and updating an attribute identification result to a corresponding target; performing event judgment according to the updated target, and outputting an event record; and releasing the data. According to the method, a double-path parallel detection and attribute identification polling mechanism is superposed, so that the identification efficiency is greatly improved.

Description

Technical Field

[0001] This application belongs to the field of personnel detection technology, specifically referring to a target detection and attribute recognition method based on multi-path parallelism. Background Technology

[0002] In conducting customer flow statistics and gender preference analysis for large chain supermarkets, traditional supermarket customer flow statistics rely on single-modal cameras, which have problems such as high missed detection rate (≥8%) in low light (such as fresh food area, underground supermarket), single frame processing time of more than 34ms leading to insufficient real-time performance, and serial processing of customer gender recognition and customer flow statistics leading to data delay. They cannot support real-time shelf adjustment and promotional staff scheduling.

[0003] Existing binocular depth target detection technologies mostly employ a serial processing mode, detecting RGB images first and then depth images, or vice versa. This fails to fully utilize hardware parallel computing power, resulting in high overall processing latency, reaching 34ms per frame. Current solutions either perform multi-attribute recognition (gender, employee, gaze) serially per frame for the same target or repeatedly crop the target image in each frame, leading to a 41ms time consumption for recognizing 10 target attributes, and the repeated cropping wastes computing power. Furthermore, the lack of a design that "reuses the first frame cropped image + allocates attribute tasks by polling frame" means that when recognizing multiple attributes of the same target, either the cumulative latency is high or repeated calculations (multiple croppings) occur, failing to balance efficiency and computing power. The computing power of the ARM Linux architecture (CPU+NPU heterogeneous architecture) is not combined with a polling strategy scheduling, making it difficult to further reduce the marginal time consumption of single-target attribute recognition. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, this application provides a target detection and attribute recognition method based on multi-path parallelism, which adopts a dual-path parallel detection superimposed attribute recognition polling mechanism, greatly improving the recognition efficiency.

[0005] This invention provides a multi-parallel target detection and attribute recognition method, which uses a binocular depth camera to acquire data. The method includes: Data is acquired from a binocular depth camera, preprocessed, and then used to obtain RGB and depth maps. Perform target detection on the RGB image and generate RGB detection results; Target detection is performed on the depth map, and depth detection results are generated; The RGB detection results and the depth detection results are fused together to generate a fused detection result; Target management is performed based on the fused detection results, and attribute recognition tasks are generated for each target; Perform attribute recognition polling and update the attribute recognition results to the corresponding target; Based on the updated target, perform event analysis and output event logs; Release the data.

[0006] Furthermore, according to the multi-parallel target detection and attribute recognition method provided in this application, the step of acquiring data from a binocular depth camera, and preprocessing it to obtain an RGB image and a depth map includes: A frame of data is acquired from the hardware interface; the frame of data includes an RGB image and a depth image, as well as 16-bit depth data. Construct RGB Data objects and depth Data objects; The RGB Data object data and the depth Data object data are pushed to the RGB image queue and the depth image queue, respectively. If the queue is full, the oldest data in the queue is removed, and data with the same timestamp in both the RGB image queue and the depth image queue is removed.

[0007] Furthermore, according to the multi-parallel target detection and attribute recognition method provided in this application, the step of performing target detection on the RGB image and generating RGB detection results includes: Retrieve RGB image data from the RGB image queue; Perform object detection on RGB images; Generate RGB detection results. Furthermore, according to the multi-parallel target detection and attribute recognition method provided in this application, the step of performing target detection on the depth map and generating depth detection results includes: Retrieve depth image data from the depth image queue; Utilizing depth information for target detection; Generate depth detection results.

[0008] Furthermore, according to the multi-path parallel target detection and attribute recognition method provided in this application, the step of fusing the RGB detection results and the depth detection results to generate the fused detection result includes: Receive the RGB detection results and the depth detection results, and align them using timestamps; The RGB detection results and the depth detection results are fused using algorithms including but not limited to IoU matching, and a fused detection result is generated.

[0009] Furthermore, according to the multi-parallel target detection and attribute recognition method provided in this application, the step of performing target management based on the fused detection results and generating attribute recognition tasks for each target includes: Receive the fused detection results; Associating with the currently tracked target, including but not limited to using IoU matching algorithms, updating the target trajectory, or creating a new target; An attribute recognition task is generated for each target every preset number of frames; wherein, the attribute recognition task includes, but is not limited to, gender, employee, and gaze. The attribute recognition task is distributed to the corresponding attribute recognition queue; wherein, the attribute recognition queue includes, but is not limited to, the gender recognition queue, the employee recognition queue, and the gaze recognition queue.

[0010] Furthermore, according to the multi-parallel target detection and attribute recognition method provided in this application, the step of performing attribute recognition polling and updating the attribute recognition results to the corresponding target includes: Retrieve attribute recognition tasks from their respective attribute recognition queues; According to the requirements of the attribute recognition task, the target region is extracted from the raw data; the raw data is an RGB image and a depth image. Run the attribute recognition algorithm, perform attribute recognition polling, and obtain the results; Update the results to the corresponding target. Furthermore, according to the multi-parallel target detection and attribute recognition method provided in this application, the attribute recognition polling includes: For each detected target, the head and shoulder region is cropped from the RGB image based on the bounding box coordinates and stored in a "target ID - head and shoulder image" key-value pair cache. The cache validity period is tied to the target tracking lifecycle and is automatically cleared after the target disappears. Perform polling task scheduling, and allocate attribute tasks according to the system frame sequence number, starting from the frame in which the target first appears; Specifically, when the frame number is 1, the attribute task is gender recognition; when the frame number is 2, the attribute task is employee recognition; when the frame number is 3, the attribute task is gaze recognition; the attribute tasks are assigned in a cyclical manner according to the frame numbers 1, 2, and 3. When performing attribute tasks in subsequent frames, the "target ID - head and shoulders image" in the cache is read directly, skipping the cropping step; the attribute results of the target are updated in real time after each attribute recognition is completed. Multi-target parallel polling, multi-threaded tasks are identified simultaneously; threads are allocated according to "target ID", each thread independently executes the polling process of a single target, and polling tasks of different targets are processed in parallel.

[0011] Furthermore, according to the multi-parallel target detection and attribute recognition method provided in this application, the step of performing event judgment based on the updated target and outputting event records includes: Event determination is performed based on the updated target trajectory; the target trajectory includes attributes and location. The event determination includes events such as crossing the warning line and exceeding the time limit for staying; Generate and output event logs.

[0012] Furthermore, according to the multi-parallel target detection and attribute recognition method provided in this application, the data release includes: Use smart pointers to manage data; When data is no longer used by any task, the smart pointer's reference count is reset to zero, automatically calling the deallocation function to return the data memory to the hardware.

[0013] The beneficial effects of this invention are as follows: This application provides a target detection and attribute recognition method based on multi-path parallel processing. This method uses dual-path parallel detection of RGB images and depth maps to overcome the serial bottleneck, and superimposes an attribute recognition polling mechanism to further reduce latency. By using frame-by-frame polling to allocate attribute tasks, multiple tasks blocking the same target in a single frame are avoided, greatly improving efficiency. The processing time for a single frame image is reduced from 34ms to 24ms (a reduction of 10ms), and the time for identifying 10 target attributes is reduced from 41ms to 21ms (a reduction of 20ms). The polling mechanism further reduces the time spent on repeated cropping by approximately 3ms per target. Attached Figure Description

[0014] The technical solution and other beneficial effects of this application will become apparent from the following detailed description of specific embodiments in conjunction with the accompanying drawings.

[0015] Figure 1 This is a schematic diagram of the target detection and attribute recognition method based on multi-path parallelism provided in Embodiment 1 of the present invention.

[0016] Figure 2 This is a flowchart of the target detection and attribute recognition method based on multi-path parallelism provided in Embodiment 1 of the present invention.

[0017] Figure 3 This is a flowchart of the data acquisition and queue management method provided in Embodiment 1 of the present invention.

[0018] Figure 4 This is a flowchart of the target detection and result fusion method provided in Embodiment 1 of the present invention.

[0019] Figure 5 This is a flowchart of the target tracking and attribute recognition method provided in Embodiment 1 of the present invention.

[0020] Figure 6 This is a data timing flowchart provided in Embodiment 1 of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0022] In the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0023] The following disclosure provides many different embodiments or examples for implementing different structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. Of course, these are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples; such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed. In addition, various specific examples of processes and materials are provided in this application, but those skilled in the art will recognize the application of other processes and / or the use of other materials.

[0024] The embodiments of this application will now be further described in conjunction with the accompanying drawings and specific implementation details.

[0025] Example 1: This invention provides a method for target detection and attribute recognition based on multi-path parallel processing, such as... Figure 1 As shown, the method includes the following steps: S1. Acquire data from a binocular depth camera, and after preprocessing, obtain RGB and depth maps; S2. Perform target detection on the RGB image and generate RGB detection results; S3. Perform target detection on the depth map and generate depth detection results; S4. The RGB detection results and the depth detection results are fused together to generate the fused detection results; S5. Perform target management based on the fused detection results and generate attribute recognition tasks for each target; S6. Perform attribute recognition polling and update the attribute recognition results to the corresponding target; S7. Based on the updated target, perform event judgment and output event records; S8. Release the data.

[0026] Specifically, such as Figure 2 , Figure 3 As shown, the data acquired from the binocular depth camera, after preprocessing, yields an RGB image and a depth map, including: S101. Obtain a frame of data from the hardware interface; the frame of data includes an RGB image and a depth image, as well as 16-bit depth data; S102. Construct RGB Data objects and depth Data objects; the RGB Data objects and depth Data objects are constructed and managed using smart pointers, which contain release functions. S103. Push the RGB Data object data and the depth Data object data to the RGB image queue and the depth image queue, respectively; In step S104, if the queue is full, remove the oldest data in the queue and ensure that data with the same timestamp in both the RGB image queue and the depth image queue are removed.

[0027] like Figure 2 , Figure 4 As shown, the step of performing target detection on the RGB image and generating RGB detection results includes: S201. Retrieve RGB image data from the RGB image queue; S202. Perform target detection on the RGB image; S203, Generate RGB detection results. The step of performing target detection on the depth map and generating depth detection results includes: S301. Retrieve depth image data from the depth image queue; S302. Target detection using depth information; S303, Generate depth detection results.

[0028] The step of fusing the RGB detection results and the depth detection results to generate the fused detection results includes: S401. Receive the RGB detection result and the depth detection result, and align them using timestamps; S402. The RGB detection results and the depth detection results are fused, and the fusion is performed using algorithms including but not limited to IoU matching, and a fused detection result is generated.

[0029] Specifically, the core of the method provided in this embodiment is based on the dual logic of "multi-path parallelism + attribute polling", which utilizes the heterogeneous computing power of ARM Linux CPU + NPU to simultaneously process the RGB image and depth image of the binocular camera, thereby achieving efficient detection and recognition.

[0030] The system receives RGB and depth maps from the binocular camera as hardware input, performs preprocessing including but not limited to size normalization and noise filtering, and then adapts them to the parallel module.

[0031] The dual-path parallel detection includes RGB image detection assigned to NPU0 and depth image detection assigned to NPU1. The CPU synchronously performs detection result fusion and outputs the target bounding box (bbox), category, and confidence score. Among them, RGB image detection relies on neural network inference, while depth image detection relies on spatial feature calculation.

[0032] Two tasks are launched synchronously using an NPU and a multi-threaded framework, sharing memory to transfer preprocessed data and avoiding copying overhead. In this embodiment, RGB image detection takes ≤34ms / frame, and depth image detection takes ≤12ms / frame. Both output the target bounding box (x1, y1, x2, y2), category (e.g., "person"), and confidence score (≥0.5).

[0033] like Figure 2 , Figure 5 As shown, the step of performing target management based on the fused detection results and generating attribute recognition tasks for each target includes: S501, Receive the fused detection results; S502. Associate with the currently tracked target, including, but not limited to, using an IoU matching algorithm, to update the target trajectory or create a new target; S503. Generate an attribute recognition task for each target every preset number of frames; wherein, the attribute recognition task includes, but is not limited to, gender, employee, and line of sight; wherein, the preset number of frames can be set according to actual needs; the attribute recognition task can also be set with other recognition tasks according to actual needs.

[0034] S504. Distribute the attribute recognition tasks to the corresponding attribute recognition queues; wherein, the attribute recognition queues include, but are not limited to, gender recognition queues, employee recognition queues, and gaze recognition queues. Each attribute recognition queue corresponds one-to-one with an attribute recognition task.

[0035] The step of performing attribute identification polling and updating the attribute identification results to the corresponding target includes: S601. Retrieve attribute recognition tasks from their respective attribute recognition queues; S602. According to the attribute recognition task requirements, extract the target region from the original data; the original data is an RGB image and a depth image; S603. Run the attribute recognition algorithm, perform attribute recognition polling, and obtain the results; S604. Update the result to the corresponding target.

[0036] In this embodiment, each attribute recognition task processes one attribute task independently without interfering with each other.

[0037] Specifically, in this embodiment, the attribute identification polling includes: For each detected target, the head and shoulder region is cropped from the RGB image based on the bounding box coordinates (x1, y1, x2, y2) and stored in the "Target ID - Head and Shoulder Image" key-value pair cache. The cache validity period is tied to the target tracking lifecycle and is automatically cleared after the target disappears. In this embodiment, the cropping rule is set to expand by 10% pixels, centered on the 1 / 3 area at the bbox coordinates, to ensure a complete head.

[0038] Perform polling task scheduling, and allocate attribute tasks according to the system frame sequence number, starting from the frame in which the target first appears; Specifically, when the frame number is 1, the attribute task is gender recognition; when the frame number is 2, the attribute task is employee recognition; when the frame number is 3, the attribute task is gaze recognition; the attribute tasks are assigned in a cyclical manner according to the frame numbers 1, 2, and 3. When performing attribute tasks in subsequent frames, the "target ID - head and shoulders image" in the cache is read directly, skipping the cropping step; the attribute results of the target are updated in real time after each attribute recognition is completed; for example, after the employee recognition task is completed in frame number 2, the original "unrecognized" status is replaced with employee ID or non-employee; Multi-target parallel polling allows for simultaneous identification of multiple tasks using multiple threads. Threads are allocated according to "target ID," with each thread independently executing the polling process for a single target, allowing for parallel processing of polling tasks for different targets. For example, while target A is executing frame number 2 (employee identification), target B can simultaneously execute frame number 3 (gaze identification).

[0039] The step of determining events based on the updated target and outputting event records includes: S701. Perform event judgment based on the updated target trajectory; the target trajectory includes attributes and location; specifically, in this embodiment, it is necessary to perform event judgment based on the updated target trajectory and attributes. The event determination includes events such as crossing the warning line and exceeding the time limit for staying; S702, Generate event logs and output them.

[0040] like Figure 2 , Figure 6 As shown, the data release includes: S801, Use smart pointers to manage data, such as shared_ptr; S802. When data is no longer used by any task, the smart pointer's reference count is reset to zero, and the release function is automatically called to return the data memory to the hardware.

[0041] In summary, the attribute recognition polling of the multi-parallel target detection and attribute recognition method provided in this application mainly includes: First frame processing: For the detected target, the head and shoulder image is cropped from the RGB image by bounding box and cached in a dedicated region, while performing the first round of attribute recognition (gender); Subsequent frame polling: For the same target, tasks are assigned according to "frame number mod3" (frame number 1 represents gender, frame number 2 represents employee, frame number 3 represents gaze, and so on), and the cached head and shoulder image is directly called without repeated cropping. At the same time, a 10-thread pool synchronously processes multi-target polling tasks, and the polling process for different targets is executed independently.

[0042] This application provides a multi-parallel target detection and attribute recognition method. This method overcomes the serial bottleneck by employing dual-path parallel detection of RGB and depth maps, and further reduces latency by superimposing an attribute recognition polling mechanism. By using frame-by-frame polling to allocate attribute tasks, multiple tasks blocking the same target within a single frame are avoided, significantly improving efficiency. The processing time for a single frame image is reduced from 34ms to 24ms (a reduction of 10ms), and the time for recognizing 10 target attributes is reduced from 41ms to 21ms (a reduction of 20ms). The polling mechanism additionally reduces the time spent on repeated cropping by approximately 3ms per target.

[0043] Example 3: This embodiment also provides a multi-path parallel target detection and attribute recognition system. The system's hardware includes binocular depth cameras based on an ARM Linux architecture (these binocular depth cameras are compatible with the CPU+NPU heterogeneous computing power scheduling of the method in Embodiment 1) installed at key locations such as supermarket entrances, fresh produce sections, and snack shelves. The cameras are installed at a height of 3.5 meters and connect to the supermarket's ERP system via Ethernet communication. The software includes the multi-path parallel target detection and attribute recognition method provided in Embodiment 1.

[0044] The system can perform dual-path parallel detection: RGB image detection (NPU running a YOLOv8 lightweight model) and depth image detection are executed simultaneously, reducing the processing time per frame to within 24ms, achieving 20fps real-time detection, and solving the latency problem of traditional serial processing; Attribute recognition polling: For each customer entering the detection area, the head and shoulder image is cropped from the RGB image in the first frame and cached. Subsequent tasks are assigned according to the polling rule of "frame 1 → gender, frame 2 → gaze, frame 3 → gaze, loop". The cached image is reused to avoid repeated cropping, which reduces the computing power consumption by an additional 3ms / target. Data output: Integrate the detection results and the attribute information obtained from polling, and output them in JSON format to the smart retail backend (such as customer analysis and employee attendance modules); output structured data of "regional number of people - gender distribution - focus of attention on shelves" in real time, and connect to the supermarket's "smart operation platform".

[0045] The method of using the system includes: Hardware deployment: Binocular cameras collect scene data and connect to an ARM Linux processing device, including CPU+NPU.

[0046] System initialization: Load the detection / recognition model, configure the thread pool (10 threads) and polling rules (such as assigning tasks based on frame number mod3), and set the head and shoulder image cache validity period (default 30 frames to avoid redundant storage after the target disappears).

[0047] Startup and operation: The system automatically receives data, performs parallel detection → first frame crop buffering → frame-by-frame polling and identification, and outputs results in real time.

[0048] Business integration: Output data to connect with intelligent retail systems, supporting scenarios such as customer dwell time analysis on shelves (combined with eye-tracking recognition) and employee attendance tracking (combined with employee identification).

[0049] In this system, based on the ARM Linux platform architecture, computing power can be scheduled, including: the NPU is responsible for RGB image + depth map detection, gender / employee recognition (neural network intensive task), and gaze recognition, while the CPU is responsible for detection result fusion, target tracking, and event logic processing (logic calculation task).

[0050] Model optimization: The detection / recognition model uses INT8 quantization to adapt to NPU computing power, with an accuracy loss of ≤2%. Cache management: The head and shoulder graph cache uses a shared memory design, which can be directly accessed by the CPU / NPU, avoiding the time-consuming cross-processor data transfer.

[0051] In summary, the multi-parallel target detection and attribute recognition system provided in this embodiment is based on the core technology of "multi-parallel binocular depth target detection + attribute recognition polling mechanism". This system mainly serves the precise perception and operational optimization in smart retail scenarios, and can also be extended to related commercial space management scenarios. Specific applications include: Passenger flow statistics and behavior analysis: Real-time detection of people within the area, outputting attributes such as number of people, gender, and direction of gaze, supporting the statistics of operational data such as passenger flow density, dwell time, and movement trajectory; Intelligent employee management: accurately identifies employee identity, combines detection time sequence to realize attendance check-in and on-duty status monitoring, and distinguishes between employees and customers to avoid operational data interference; Precision operation of retail scenarios: Combining customer attributes (gender, line of sight) and location information to assist in optimizing shelf displays, evaluating the effectiveness of promotional activities, and analyzing regional popularity; Real-time edge awareness: Adapted to ARM Linux edge devices (CPU+NPU heterogeneous architecture), meeting the edge deployment requirements of "low latency, low power consumption, and high reliability" in retail scenarios.

[0052] The system provided in this embodiment is highly adaptable to various scenarios, designed specifically for smart retail, and combines real-time detection (such as customer movement tracking) with accurate attribute recognition (such as employee attendance and customer gender analysis), exhibiting high robustness.

[0053] Example 3: This embodiment also provides a target detection and attribute recognition terminal device based on multi-path parallelism. The terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the method embodiment described above in Embodiment 1 of the present invention.

[0054] Furthermore, as an executable solution, the multi-parallel target detection and attribute recognition terminal device can be a binocular camera, desktop computer, laptop, handheld computer, or cloud server, etc. The multi-parallel target detection and attribute recognition terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above-described structure of the multi-parallel target detection and attribute recognition terminal device is merely an example and does not constitute a limitation on the multi-parallel target detection and attribute recognition terminal device. It may include more or fewer components than described above, or combine certain components, or different components. For example, the multi-parallel target detection and attribute recognition terminal device may also include input / output devices, network access devices, buses, etc., which are not limited in this embodiment of the present invention.

[0055] Furthermore, as an executable solution, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices. The general-purpose processor can be a microprocessor or any conventional processor. This processor is the control center of the multi-parallel target detection and attribute recognition terminal device, connecting all parts of the device via various interfaces and lines.

[0056] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the multi-parallel target detection and attribute recognition terminal device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the mobile phone. In addition, the memory may include high-speed random access memory and non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0057] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the embodiments of the present invention.

[0058] If the module / unit integrated by the multi-parallel target detection and attribute recognition terminal device is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc.

[0059] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the present invention. Finally, it should be noted that in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0060] The above provides a detailed description of a multi-path parallel target detection and attribute recognition method provided by the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the technical solutions and core ideas of this application. Those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A target detection and attribute recognition method based on multi-path parallel processing, employing a binocular depth camera for data acquisition, characterized in that... The method includes: Data is acquired from a binocular depth camera, preprocessed, and then used to obtain RGB and depth maps. Perform target detection on the RGB image and generate RGB detection results; Target detection is performed on the depth map, and depth detection results are generated; The RGB detection results and the depth detection results are fused together to generate a fused detection result; Target management is performed based on the fused detection results, and attribute recognition tasks are generated for each target; Perform attribute identification polling and update the attribute identification results to the corresponding target; Based on the updated target, perform event analysis and output event logs; Release the data.

2. The target detection and attribute recognition method based on multi-path parallel processing according to claim 1, characterized in that, The data acquired from the binocular depth camera is preprocessed to obtain an RGB image and a depth map, including: A frame of data is acquired from the hardware interface; the frame of data includes an RGB image and a depth image, as well as 16-bit depth data. Construct RGB Data objects and depth Data objects; The RGB Data object data and the depth Data object data are pushed to the RGB image queue and the depth image queue, respectively. If the queue is full, the oldest data in the queue is removed, and data with the same timestamp in both the RGB image queue and the depth image queue is removed.

3. The target detection and attribute recognition method based on multi-path parallel processing according to claim 2, characterized in that, The step of performing target detection on the RGB image and generating RGB detection results includes: Retrieve RGB image data from the RGB image queue; Perform object detection on RGB images; Generate RGB detection results.

4. The target detection and attribute recognition method based on multi-path parallel processing according to claim 3, characterized in that, The step of performing target detection on the depth map and generating depth detection results includes: Retrieve depth image data from the depth image queue; Utilizing depth information for target detection; Generate depth detection results.

5. The target detection and attribute recognition method based on multi-path parallel processing according to claim 4, characterized in that, The step of fusing the RGB detection results and the depth detection results to generate the fused detection results includes: Receive the RGB detection results and the depth detection results, and align them using timestamps; The RGB detection results and the depth detection results are fused using algorithms including but not limited to IoU matching, and a fused detection result is generated.

6. The target detection and attribute recognition method based on multi-path parallel processing according to claim 5, characterized in that, The step of performing target management based on the fused detection results and generating attribute recognition tasks for each target includes: Receive the fused detection results; Associating with the currently tracked target, including but not limited to using IoU matching algorithms, updating the target trajectory, or creating a new target; An attribute recognition task is generated for each target every preset number of frames; wherein, the attribute recognition task includes, but is not limited to, gender, employee, and gaze. The attribute recognition task is distributed to the corresponding attribute recognition queue; wherein, the attribute recognition queue includes, but is not limited to, the gender recognition queue, the employee recognition queue, and the gaze recognition queue.

7. The target detection and attribute recognition method based on multi-path parallel processing according to claim 6, characterized in that, The step of performing attribute identification polling and updating the attribute identification results to the corresponding target includes: Retrieve attribute recognition tasks from their respective attribute recognition queues; According to the requirements of the attribute recognition task, the target region is extracted from the raw data; the raw data is an RGB image and a depth image. Run the attribute recognition algorithm, perform attribute recognition polling, and obtain the results; Update the results to the corresponding target.

8. The target detection and attribute recognition method based on multi-path parallel processing according to claim 7, characterized in that, The attribute recognition polling includes: For each detected target, the head and shoulder region is cropped from the RGB image based on the bounding box coordinates and stored in a "target ID - head and shoulder image" key-value pair cache. The cache validity period is tied to the target tracking lifecycle and is automatically cleared after the target disappears. Perform polling task scheduling, and allocate attribute tasks according to the system frame sequence number, starting from the frame in which the target first appears; Specifically, when the frame number is 1, the attribute task is gender recognition; when the frame number is 2, the attribute task is employee recognition; when the frame number is 3, the attribute task is gaze recognition; the attribute tasks are assigned in a cyclical manner according to the frame numbers 1, 2, and 3. When performing attribute tasks in subsequent frames, the "target ID - head and shoulders image" in the cache is read directly, skipping the cropping step; the attribute results of the target are updated in real time after each attribute recognition is completed; Multi-target parallel polling, multi-threaded tasks are identified simultaneously; threads are allocated according to "target ID", each thread independently executes the polling process of a single target, and polling tasks of different targets are processed in parallel.

9. The target detection and attribute recognition method based on multi-path parallel processing according to claim 7, characterized in that, The step of determining events based on the updated target and outputting event records includes: Event determination is performed based on the updated target trajectory; the target trajectory includes attributes and location. The event determination includes events such as crossing the warning line and exceeding the time limit for staying; Generate and output event logs.

10. The target detection and attribute recognition method based on multi-path parallel processing according to claim 9, characterized in that, The data release includes: Use smart pointers to manage data; When data is no longer used by any task, the smart pointer's reference count is reset to zero, automatically calling the deallocation function to return the data memory to the hardware.

Images