Pedestrian detection method and device, storage medium and electronic device

Abstract

The present disclosure specifically relates to a pedestrian detection method and device, a storage medium and an electronic device. The method comprises: performing head recognition on a first image collected by a first camera device in a first shooting direction to obtain first head coordinates of each head target in the first image; performing recognition on a second image collected by a second camera device in a second shooting direction to obtain second skeleton coordinates of a skeleton target and third pedestrian coordinates of a pedestrian target in the second image; using coordinate data to perform target comparison on any two or more of the head target, the skeleton target and the pedestrian target, and determining a pedestrian detection result based on a target comparison result. The present scheme can effectively improve the detection rate of pedestrians and avoid missed detection and false detection.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, specifically to a pedestrian detection method, a pedestrian detection device, a storage medium, and an electronic device. Background Technology

[0002] Pedestrian detection is an important research topic in object detection and has been practically applied in numerous scenarios, demonstrating its value in areas such as unmanned supermarkets, autonomous vehicles, intelligent transportation, and public area surveillance. However, as application scenarios become increasingly complex, existing pedestrian detection solutions exhibit certain shortcomings and deficiencies. For instance, when pedestrians overlap or when pedestrians overlap with human-like objects in the environment (such as lampposts), it becomes impossible to accurately detect every pedestrian, leading to missed detections and false positives. Therefore, providing a pedestrian detection solution with higher accuracy is a problem that those skilled in the art need to address.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] This disclosure provides a pedestrian detection method, a pedestrian detection device, a storage medium, and an electronic device, which can solve the problems existing in the prior art to a certain extent.

[0005] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0006] According to a first aspect of this disclosure, a pedestrian detection method is provided, the method comprising:

[0007] The head detection model is used to perform head recognition on the first image captured by the first camera device in the first shooting direction to obtain the first head coordinates of each head target in the first image; and the skeleton point detection model and the pedestrian detection model are used to identify the second image captured by the second camera device in the second shooting direction to obtain the second skeleton coordinates of the skeleton targets and the third pedestrian coordinates of the pedestrian targets in the second image.

[0008] The target comparison is performed on any two or more of the head target, skeletal target and pedestrian target using coordinate data, and the pedestrian detection result is determined based on the target comparison result.

[0009] In some exemplary embodiments, the step of using coordinate data to compare any two or more of the head target, skeletal target, and pedestrian target, and determining the pedestrian detection result based on the target comparison result, includes:

[0010] Based on the skeletal targets and pedestrian targets corresponding to the second image, the corresponding IOU values ​​are calculated, and the pedestrian targets in the second image are filtered using the IOU values ​​to obtain a first filtering result of target mismatch and a second filtering result of target matching;

[0011] Distance comparison is performed using the third pedestrian coordinates of the marked pedestrian targets in the first filtering result and the first head coordinates of the head target corresponding to the first image, so as to obtain the third filtering result based on the mismatched comparison result and generate the fourth filtering result based on the matching comparison result.

[0012] Based on the third pedestrian coordinates of the marked pedestrian targets in the second filtering result and the first head coordinates of the marked head targets in the third filtering result, a distance comparison is performed, and a fifth filtering result is generated based on the mismatch comparison results.

[0013] Based on the first head coordinates of the head target in the fifth screening result and the second bone coordinates of the marked bone target in the first screening result, a distance comparison is performed to obtain a sixth screening result that is successfully matched;

[0014] The pedestrian detection result is determined based on at least one of the second screening result, the fourth screening result, and the sixth screening result.

[0015] In some exemplary embodiments, the step of using a head detection model to perform head recognition on a first image captured by a first camera device in a first shooting direction, in order to obtain the first head coordinates of each head target in the first image, includes:

[0016] Coordinate system calibration is performed based on the first camera coordinate system of the first camera device and the site coordinate system corresponding to the target site to obtain the corresponding first transformation matrix;

[0017] A head detection model is used to perform head recognition on the first image captured by the first camera device to obtain the head target in the first image and the corresponding first detection box;

[0018] The center point coordinates in the first camera coordinate system are calculated based on the first detection frame, and the center point coordinates in the field coordinate system are calculated using the first transformation matrix and configured as the first head coordinates.

[0019] Configure a corresponding identity identifier for each of the head targets.

[0020] In some exemplary embodiments, the step of using a skeletal point detection model to identify the second image acquired by the second camera device in the second shooting direction, in order to obtain the second skeletal coordinates of the skeletal target in the second image, includes:

[0021] Coordinate system calibration is performed based on the second camera coordinate system of the second camera device and the site coordinate system corresponding to the target site to obtain the corresponding second transformation matrix;

[0022] A skeletal point detection model is used to detect skeletal points in the second image acquired by the second camera device in order to obtain the skeletal targets and corresponding second detection boxes in the second image.

[0023] When the skeletal target includes a head skeleton, the center point coordinates of the skeletal target in the second camera coordinate system are calculated by combining the coordinates of the head skeleton; or, when the skeletal target does not include a head skeleton, the center point coordinates of the skeletal target in the second camera coordinate system are calculated by using the average value of the coordinates of each bone point.

[0024] The center point coordinates in the second camera coordinate system are calculated using the second transformation matrix, and then configured as the second skeleton coordinates.

[0025] Configure a corresponding identity for each of the skeletal targets.

[0026] In some exemplary embodiments, the step of using a pedestrian detection model to identify the second image captured by the second camera device in the second shooting direction, in order to obtain the third pedestrian coordinates of the pedestrian target in the second image, includes:

[0027] Coordinate system calibration is performed based on the second camera coordinate system of the second camera device and the site coordinate system corresponding to the target site to obtain the corresponding second transformation matrix;

[0028] A pedestrian detection model is used to detect pedestrians in the second image captured by the second camera device, so as to obtain pedestrian targets in the second image and their corresponding third detection boxes;

[0029] The center point coordinates in the second camera coordinate system are calculated based on the third detection frame, and the center point coordinates in the field coordinate system are calculated using the second transformation matrix and configured as the third pedestrian coordinates.

[0030] Configure corresponding identity identifiers for each of the aforementioned pedestrian targets.

[0031] In some exemplary embodiments, the method further includes: calculating corresponding IOU values ​​based on the skeletal targets and pedestrian targets corresponding to the second image, and using the IOU values ​​to filter pedestrian targets in the second image to obtain a first filtering result of target mismatch and a second filtering result of target matching, including:

[0032] The corresponding IOU values ​​are calculated using the second detection boxes of each skeletal target in the second image and the third detection boxes of each pedestrian target.

[0033] For each pedestrian target, determine the skeletal target with the largest IOU value, and when the IOU value is greater than or equal to a first preset threshold, configure the pedestrian target to be matched with the skeletal target, add a mark and configure it as the second filtering result;

[0034] Alternatively, when the IOU value is less than the first preset threshold, the pedestrian target and the skeleton target are configured to be mismatched, and they are marked and configured as the first filtering result.

[0035] In some exemplary embodiments, the step of performing distance comparison using the third pedestrian coordinates of the marked pedestrian targets in the first filtering result and the first head coordinates of the head target corresponding to the first image, to obtain a third filtering result based on the mismatched comparison result and to generate a fourth filtering result based on the successfully matched comparison result, includes:

[0036] Based on the third pedestrian coordinates of the pedestrian target in the field coordinate system and the first head coordinates of each head target in the field coordinate system in the first screening result, the distance between the third detection box corresponding to each pedestrian target and the first detection box of each head target is calculated respectively.

[0037] For each pedestrian target, the head target with the smallest detection box distance is determined, and when the detection box distance is less than or equal to the second preset threshold, the pedestrian target is configured to match the head target, and a mark is added to it and configured as the fourth filtering result;

[0038] Alternatively, when the distance between the detection boxes is greater than the second preset threshold, the pedestrian target and the head target are configured to not match, and they are marked and configured as the third filtering result.

[0039] In some exemplary embodiments, the step of performing a distance comparison based on the third pedestrian coordinates of the marked pedestrian targets in the second filtering result and the first head coordinates of the marked head targets in the third filtering result, and generating a fifth filtering result based on the mismatch comparison results, includes:

[0040] Based on the first head coordinates corresponding to the head target in the third filtering result and the third pedestrian coordinates corresponding to the pedestrian target in the second filtering result, the distance between the first detection box corresponding to each head target and the third detection box corresponding to each pedestrian target is calculated respectively.

[0041] The pedestrian target with the smallest detection box distance corresponding to the head target is determined, and when the detection box distance is less than or equal to a third preset threshold, the head target is configured to match the pedestrian target;

[0042] Alternatively, when the distance between the detection boxes is greater than the third preset threshold, the head target and the pedestrian target are configured to not match, and the fifth filtering result is generated based on these mismatched head targets.

[0043] In some exemplary embodiments, the step of performing a distance comparison based on the first head coordinates of the head target in the fifth filtering result and the second bone coordinates of the marked skeletal target in the first filtering result to obtain a successfully matched sixth filtering result includes:

[0044] Based on the first head coordinates corresponding to each head target in the fifth screening result and the second pedestrian coordinates corresponding to the skeletal target in the first screening result, the distance between a detection box corresponding to each head target and the second detection box corresponding to each skeletal target is calculated respectively.

[0045] The skeletal target with the smallest detection box distance corresponding to the head target is determined, and when the detection box distance is less than or equal to a fourth preset threshold, the head target is configured to match the skeletal target, and this is used as the sixth screening result.

[0046] In some exemplary embodiments, the method further includes:

[0047] Repeat the above pedestrian detection method n times to obtain n sets of target comparison results;

[0048] Statistical analysis is performed on n sets of target comparison results. If a target is marked as a pedestrian detection result more than or equal to m times, then the target is marked as a pedestrian target; where m and n are positive integers.

[0049] According to a second aspect of this disclosure, a pedestrian detection device is provided, the device comprising:

[0050] The first camera coordinate calculation module is used to perform head recognition on the first image captured by the first camera device in the first shooting direction using a head detection model, so as to obtain the first head coordinates of each head target in the first image.

[0051] The second camera coordinate calculation module is used to identify the second image acquired by the second camera device in the second shooting direction using the skeleton point detection model and the pedestrian detection model, respectively, so as to obtain the second skeleton coordinates of the skeleton target and the third pedestrian coordinates of the pedestrian target in the second image;

[0052] The coordinate comparison module is used to compare any two or more of the head target, skeletal target and pedestrian target using coordinate data, and determine the pedestrian detection result based on the target comparison result.

[0053] According to a third aspect of this disclosure, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the pedestrian detection method described above.

[0054] According to a fourth aspect of this disclosure, an electronic device is provided, comprising:

[0055] Processor; and

[0056] Memory for storing the executable instructions of the processor;

[0057] The processor is configured to implement the pedestrian detection method described above when executing the executable instructions.

[0058] One embodiment of this disclosure provides a pedestrian detection method that acquires images of a target scene from different angles using cameras positioned in two different shooting directions. Head recognition is performed on the first image from the first direction to obtain a head target, and skeletal point detection and pedestrian detection are performed on the second image from the second direction to obtain corresponding skeletal targets and pedestrian targets. The skeletal targets and pedestrian targets can be matched, either by matching the head target with the skeletal target, or by cross-matching and verifying the head target, skeletal targets, and pedestrian targets. This effectively eliminates missed detections caused by overlapping people and significantly improves the detection rate of pedestrians in overlapping situations. Thus, by comparing and verifying pedestrian detection results within the target area using detection from different angles, the detection rate is improved, and missed or false detections are avoided.

[0059] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0060] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0061] Figure 1 The illustration shows a schematic diagram of a pedestrian detection method according to an exemplary embodiment of the present disclosure;

[0062] Figure 2 This schematic diagram illustrates a method for calculating a head target according to an exemplary embodiment of the present disclosure;

[0063] Figure 3 The illustration schematically shows a method for calculating skeletal targets according to an exemplary embodiment of the present disclosure;

[0064] Figure 4 This illustration schematically depicts a method for calculating pedestrian targets according to an exemplary embodiment of the present disclosure;

[0065] Figure 5 This illustration schematically depicts a method for matching screening results to determine pedestrian detection results in an exemplary embodiment of this disclosure;

[0066] Figure 6 This illustration schematically shows a method for matching pedestrian targets and skeletal targets corresponding to a second image in an exemplary embodiment of the present disclosure;

[0067] Figure 7 This illustration schematically depicts a method for matching pedestrian targets with head targets in a first screening result according to an exemplary embodiment of the present disclosure;

[0068] Figure 8 This illustration schematically depicts a method for matching a second screening result with a third screening result in an exemplary embodiment of the present disclosure;

[0069] Figure 9 This diagram illustrates a method for matching a fifth screening result with a first screening result in an exemplary embodiment of the present disclosure.

[0070] Figure 10 This schematic diagram illustrates the flow of a pedestrian detection method according to an exemplary embodiment of the present disclosure;

[0071] Figure 11 This schematic diagram illustrates the composition of a pedestrian detection device according to an exemplary embodiment of the present disclosure;

[0072] Figure 12 This schematic diagram illustrates the composition of a coordinate comparison module in an exemplary embodiment of the present disclosure;

[0073] Figure 13 This schematic diagram illustrates the composition of an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0074] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0075] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0076] To address the shortcomings and deficiencies of existing technologies, this example implementation provides a pedestrian detection method that can be applied to scenarios such as unmanned supermarkets, self-driving cars, intelligent transportation, and public area monitoring. (Reference) Figure 1 As shown, a pedestrian detection method may include:

[0077] Step S11: Use the head detection model to perform head recognition on the first image captured by the first camera device in the first shooting direction, so as to obtain the first head coordinates of each head target in the first image;

[0078] Step S12: The second image acquired by the second camera device in the second shooting direction is identified using the skeleton point detection model and the pedestrian detection model, respectively, so as to obtain the second skeleton coordinates of the skeleton target and the third pedestrian coordinates of the pedestrian target in the second image.

[0079] Step S13: Use coordinate data to compare any two or more of the head target, skeletal target and pedestrian target, and determine the pedestrian detection result based on the target comparison result.

[0080] The pedestrian detection method provided in this example implementation can be executed on the server side. By setting up cameras in two different shooting directions to capture images of the target scene from different angles, head recognition is performed on the first image from the first direction to obtain the head target, and skeletal point detection and pedestrian detection are performed on the second image from the second direction to obtain the corresponding skeletal target and pedestrian target. The skeletal target and pedestrian target can be matched, either by matching the head target with the skeletal target, or by cross-matching and verifying the head target, skeletal target, and pedestrian target; or, multiple matching and verifications can be performed, thereby effectively eliminating missed detections caused by overlapping people and effectively improving the detection rate of pedestrians in overlapping situations. This achieves the comparison and verification of pedestrian detection results in the target area using detection from different angles, improving the detection rate and avoiding missed or false detections.

[0081] The steps of the pedestrian detection method in this exemplary embodiment will now be described in more detail with reference to the accompanying drawings and embodiments.

[0082] In step S11, a head detection model is used to perform head recognition on the first image captured by the first camera device in the first shooting direction, so as to obtain the first head coordinates of each head target in the first image.

[0083] In this example embodiment, the first camera device is positioned at the top center of the target site, and the first shooting direction is a top-down shooting direction.

[0084] For example, camera A can be placed in the center of the venue so that it can shoot from above and the focal length of the camera can be adjusted to cover the entire venue.

[0085] In this example embodiment, step S11 involves using a head detection model to perform head recognition on the first image captured by the first camera device in the first shooting direction, in order to obtain the first head coordinates of each head target in the first image, with reference to... Figure 2 As shown, it can specifically include:

[0086] Step S21: Based on the first camera coordinate system of the first camera device and the site coordinate system corresponding to the target site, coordinate system calibration is performed to obtain the corresponding first transformation matrix;

[0087] Step S22: Use the head detection model to perform head recognition on the first image captured by the first camera device to obtain the head target in the first image and the corresponding first detection box;

[0088] Step S23: Calculate the center point coordinates in the first camera coordinate system according to the first detection frame, and calculate the center point coordinates in the field coordinate system according to the first transformation matrix, and configure them as the first head coordinates;

[0089] Step S24: Configure a corresponding identity identifier for each of the head targets.

[0090] Specifically, the first camera device can be the aforementioned camera A. Four points can be selected as marker points in the image A captured by camera A. At the same time, the actual positions of these four marker points can be calibrated in the actual field, and the actual coordinates of these four actual positions can be obtained in the actual field coordinate system based on the field. Based on the coordinates of these four marker points in the camera coordinate system and their corresponding actual coordinates, the transformation matrix S between the actual positions and the position calibration on the image can be calculated.

[0091] The head detection model mentioned above can be, for example, a trained FCHD-Fully convolutional head detector. Currently, in other exemplary embodiments of this disclosure, head detection models based on other algorithms can also be used, and this disclosure does not specifically limit the specific algorithm of the head detection model. Specifically, the image captured in real time by camera A can be used as the input to the head detection model to identify whether a head target exists in the image. If a head target is detected in the image, a rectangular coordinate box of the head target detection result can be output as the first detection box AA; the coordinates of the four corners of the first detection box are configured as follows: top left corner (Q11_x, Q11_y), top right corner (Q12_x, Q12_y), bottom left corner (Q13_x, Q13_y), and bottom right corner (Q14_x, Q14_y). The coordinates of the center point can be calculated using the coordinates of the top left and bottom right corners of the detection box, denoted as (center_x, center_y) = ((Q11_x + Q14_x) / 2), (Q11_y + Q14_y) / 2. Then, using the calculated transformation matrix S, the actual coordinates of the center point in the field coordinate system can be calculated from the coordinates of the center point in the first camera coordinate system. The calculation formula can include: (ca_x, ca_y) = (center_x, center_y) S. In addition, a corresponding identity identifier, such as an ID, can be assigned to the detection result of each head target; this facilitates statistics and calculation.

[0092] In step S12, the skeleton point detection model and the pedestrian detection model are used to identify the second image acquired by the second camera device in the second shooting direction, so as to obtain the second skeleton coordinates of the skeleton target and the third pedestrian coordinates of the pedestrian target in the second image.

[0093] In this example embodiment, the second camera device is set on one side of the target site, and the second shooting direction is the shooting direction facing the side of the target site.

[0094] In addition, camera B can be placed on the side of the venue, allowing it to capture images from the side and horizontally. The camera's focal length can be adjusted to cover the entire venue. Cameras A and B can be configured to capture images synchronously at the same frequency, ensuring that image A from camera A and image B from camera B capture the same content and scene at the same time.

[0095] refer to Figure 3 As shown, step S12, which involves using a skeletal point detection model to identify the second image captured by the second camera device in the second shooting direction to obtain the second skeletal coordinates of the skeletal target in the second image, may specifically include:

[0096] Step S31: Based on the second camera coordinate system of the second camera device and the site coordinate system corresponding to the target site, coordinate system calibration is performed to obtain the corresponding second transformation matrix;

[0097] Step S32: Use the skeletal point detection model to perform skeletal point detection on the second image acquired by the second camera device to obtain the skeletal targets and corresponding second detection boxes in the second image;

[0098] Step S33: When the skeletal target includes a head skeleton, calculate the center point coordinates of the skeletal target in the second camera coordinate system by combining the coordinates of the head skeleton; or, when the skeletal target does not include a head skeleton, calculate the center point coordinates of the skeletal target in the second camera coordinate system by using the average value of the coordinates of each bone point.

[0099] Step S34: Calculate the center point coordinates in the field coordinate system corresponding to the center point coordinates in the second camera coordinate system using the second transformation matrix, and configure them as the second skeleton coordinates;

[0100] Step S35: Configure a corresponding identity identifier for each of the skeletal targets.

[0101] Specifically, the second camera device can be camera B, which is positioned on the side of the field as described above. Camera B can capture images from the side, and its focal length can be adjusted to cover the field. Four marker points can be selected from the image captured by camera B, and the coordinates of each marker point in camera B's camera coordinate system can be determined. Simultaneously, the actual positions of these four marker points are marked on the field, and the true coordinates of each actual position are determined in the field coordinate system corresponding to the target field. Thus, the coordinate system can be calibrated based on the coordinates of the four marker points in the camera coordinate system and the field coordinates in the field coordinate system, resulting in the corresponding transformation matrix W.

[0102] Specifically, the aforementioned skeleton detection model can be, for example, a skeleton detection algorithm based on High-Resolution Net or a skeleton detection algorithm based on Compute First Networking (CFN). Of course, in other exemplary embodiments of this disclosure, a skeleton detection algorithm based on, for example, a convolutional neural network can also be used. This disclosure does not specifically limit the specific algorithmic content of the skeleton detection algorithm.

[0103] Specifically, the images captured in real-time by camera B can be used as input to the skeleton detection model, which then identifies human skeletal points in the image. When a skeletal target is detected, a rectangle can be created using the long diagonal line connecting the highest and lowest points of the target as a reference, selecting all skeletal points as a second detection box BB. The skeleton recognition results can be used to determine whether the head skeleton is included. If the head skeleton is included, the x-coordinate of the head of the skeletal point can be used as the x-coordinate of the target center point, and the y-coordinate of the lowest point of the target can be used as the y-coordinate of the target center point, denoted as (circle_x, circle_y) = (x-coordinate of the head skeleton, y-coordinate of the lowest point). Alternatively, if the head skeleton is not detected, the average of the x-coordinates of all skeletal points can be used as the x-coordinate of the target center point, and the y-coordinate of the lowest point can be used as the y-coordinate of the target center point, denoted as (circle_x, circle_y) = (average of the x-coordinates of all skeletal points, y-coordinate of the lowest point). After determining the center point coordinates in the camera coordinate system, the actual position coordinates in the corresponding scene coordinate system can be calculated using the calculated transformation matrix W, and used as the second skeleton coordinates; the formula can include (cb_x, cb_y) = (circle_x, circle_y). W. Additionally, a corresponding identity ID can be configured for each identified skeletal target.

[0104] In this example implementation, step S12 involves using a pedestrian detection model to identify the second image captured by the second camera device in the second shooting direction, thereby obtaining the third pedestrian coordinates of the pedestrian target in the second image, with reference to... Figure 4 As shown, it can specifically include:

[0105] Step S41: Based on the second camera coordinate system of the second camera device and the site coordinate system corresponding to the target site, coordinate system calibration is performed to obtain the corresponding second transformation matrix;

[0106] Step S42: Use the pedestrian detection model to perform pedestrian detection on the second image captured by the second camera device to obtain the pedestrian targets in the second image and the corresponding third detection boxes;

[0107] Step S43: Calculate the center point coordinates in the second camera coordinate system corresponding to the third detection frame, and use the second transformation matrix to calculate the center point coordinates in the corresponding site coordinate system, and configure them as the third pedestrian coordinates;

[0108] Step S44: Configure corresponding identity identifiers for each of the pedestrian targets.

[0109] Specifically, a pedestrian detection model can be used to detect pedestrians in the images captured in real time by camera B, determining whether the current image contains a pedestrian target. The pedestrian detection model can be, for example, a trained deep learning-based pedestrian detection model or a convolutional neural network-based pedestrian detection model, etc. Of course, in other exemplary embodiments of this disclosure, the pedestrian detection model can also be based on models using other frameworks or algorithms; this disclosure does not specifically limit the specific algorithmic content of the pedestrian detection model.

[0110] The image captured in real time from the side by camera B can be used as input to the pedestrian detection model to determine whether the current image contains a pedestrian target. When a pedestrian target is detected in the current image, the smallest rectangle can be used to select each pedestrian target as the third detection box CC. The coordinates of the four corners of the detection box are: top left corner (Q21_x, Q21_y), top right corner (Q22_x, Q22_y), bottom left corner (Q23_x, Q23_y), and bottom right corner (Q24_x, Q24_y). The coordinates of the center point of the detection box can be calculated using the coordinates of the top left and bottom right corners. The formula can include: (c3_x, c3_y) = ((Q21_x + Q24_x) / 2, (Q21_y + Q24_y) / 2). Using the second transformation matrix described above, the center point coordinates in the camera coordinate system can be converted into the actual position coordinates in the site coordinate system, and used as the third pedestrian coordinates; the formula can include: (cc_x, cc_y) = (c3_x, c3_y) Additionally, a corresponding ID can be configured for each pedestrian target to facilitate subsequent calculations.

[0111] In step S13, coordinate data is used to compare any two or more of the head target, skeletal target and pedestrian target, and the pedestrian detection result is determined based on the target comparison result.

[0112] In this example implementation, refer to Figure 5 As shown, step S13 above may specifically include:

[0113] Step S131: Calculate the corresponding IOU value based on the skeletal target and pedestrian target corresponding to the second image, and use the IOU value to filter the pedestrian targets in the second image to obtain the first filtering result of target mismatch and the second filtering result of target matching;

[0114] Step S132: Use the third pedestrian coordinates of the marked pedestrian targets in the first filtering result and the first head coordinates of the head target corresponding to the first image to perform distance comparison, so as to obtain the third filtering result based on the mismatched comparison result and generate the fourth filtering result based on the matching comparison result.

[0115] Step S133: Perform distance comparison based on the third pedestrian coordinates of the marked pedestrian targets in the second filtering result and the first head coordinates of the marked head targets in the third filtering result, and generate a fifth filtering result based on the mismatch comparison results;

[0116] Step S134: Perform a distance comparison based on the first head coordinates of the head target in the fifth filtering result and the second bone coordinates of the marked skeletal target in the first filtering result to obtain a sixth filtering result that is successfully matched;

[0117] Step S135: Determine the pedestrian detection result based on at least one of the second screening result, the fourth screening result, and the sixth screening result.

[0118] In this example implementation, in step S131 above, for the recognition results of the skeletal target and pedestrian target corresponding to the acquired second image, an image recognition result matching can be performed first. Specifically, refer to... Figure 6 As shown, step S131 above may include:

[0119] Step S61: Calculate the corresponding IOU value using the second detection box of each skeletal target in the second image and the third detection box of each pedestrian target.

[0120] Step S62: For each pedestrian target, determine the skeletal target with the largest IOU value, and when the IOU value is greater than or equal to the first preset threshold, configure the pedestrian target to match the skeletal target, add a mark and configure it as the second filtering result;

[0121] Step S63, or, when the IOU value is less than the first preset threshold, configure the pedestrian target and the skeleton target as mismatched, add a mark and configure it as the first filtering result.

[0122] Specifically, for the skeletal targets and pedestrian targets already identified in the current second image, the IOU (Intersection over Union) value can be calculated for each pedestrian target using the size of its corresponding third detection box CC and the size of each skeletal target's second detection box BB. This IOU represents the overlap between the two detection boxes. The calculation formula may include: For example, if the current second image contains M third detection boxes for pedestrian targets and N second detection boxes for skeletal targets, then M needs to be calculated. N IOU values ​​are required. For any pedestrian target, N IOU values ​​need to be calculated, corresponding to N skeletal targets. After calculating the IOU values, for each pedestrian target, the largest IOU value can be selected from the N IOU values, and the skeletal target corresponding to the largest IOU value is marked as the pre-pairing result. This largest IOU value is recorded as... .

[0123] For each pedestrian target, the maximum IOU value can be compared with a pre-configured first preset IOU threshold; for example, the first preset threshold can be 0.8 or other values. If the value is greater than or equal to 0.8, the pedestrian target is marked as matching the corresponding skeletal target. That is, the third detection box corresponding to the pedestrian target and the second detection box corresponding to the skeletal target are actually the detection results of the same pedestrian. Based on this judgment method, successfully matched pedestrian targets can be marked as PP1l, PP2l, ..., PPcl; and the corresponding matched skeletal targets can be marked as QQ1l, QQ2l, ..., QQdl. The matched pedestrian targets and their corresponding skeletal targets can be used as the second filtering results.

[0124] At the same time, if the maximum IOU value of the pedestrian target is less than the first preset threshold, i.e. If the value is less than 0.8, it is determined that the pedestrian target does not match the skeletal target, meaning that there is no skeletal target in the current image that matches the pedestrian target. In this case, the pedestrian target and the corresponding pre-matched skeletal target can be labeled separately and used as the first filtering result. The pedestrian target can be labeled as Y1, Y2, ..., Yn; the skeletal target can be labeled as Z1, Z2, ..., Zm.

[0125] By performing a matching process on the pedestrian targets and skeletal targets identified in the second image, false detections caused by "human-like objects," such as pillars, can be effectively eliminated; and "multiple overlapping pedestrian boxes detected from a single pedestrian" can be effectively eliminated, reducing the false detection rate. Furthermore, in some embodiments, the pedestrian targets marked in the first screening result can be used as the final pedestrian detection result.

[0126] In this example implementation, regarding step S132 above, after obtaining the first and second filtering results through the previous step's filtering, the head target can be used to verify the pedestrian targets that failed to match, thus achieving a second matching process. Specifically, refer to... Figure 7 As shown, step S132 above may include:

[0127] Step S71: Based on the third pedestrian coordinates of the pedestrian target in the field coordinate system and the first head coordinates of each head target in the field coordinate system in the first screening result, calculate the distance between the third detection box corresponding to each pedestrian target and the first detection box of each head target.

[0128] Step S72: For each pedestrian target, determine the head target with the smallest detection box distance, and when the detection box distance is less than or equal to the second preset threshold, configure the pedestrian target to match the head target, add a mark to it, and configure it as the fourth filtering result;

[0129] Step S73, or, when the distance of the detection box is greater than the second preset threshold, configure the pedestrian target and the head target as mismatched, add a mark and configure it as the third filtering result.

[0130] Specifically, in the first screening results obtained during the initial screening process where no match was found, the third detection boxes of the marked pedestrian targets are labeled as Y1, Y2, ..., Yn in the camera coordinate system; and as Y1l, Y2l, ..., Ynl in the corresponding site coordinate system. For example, the coordinates of the center point can be used as the coordinates of the detection boxes for calculation.

[0131] Furthermore, for the head target identified using the first image, the first detection box in the camera coordinate system is labeled S1, S2, ..., Sm; the corresponding first detection box in the field coordinate system is labeled S1l, S2l, ..., Sml. For example, the coordinates of the center point of the head target can be used as the coordinates of the first detection box.

[0132] In the site coordinate system, the center point coordinates can be used to compare each marked pedestrian target Y1l, Y2l, ..., Ynl in the first screening result with each head target S1l, S2l, ..., Sml, respectively, and calculate the distance d between the third detection box of the pedestrian target and the first detection box of each head target. Specifically, n can be calculated. The calculation results of m distances d. For each marked pedestrian target Y1l, Y2l, ..., Ynl, the distance d between the center points of each pedestrian target and m head targets can be calculated using the center point coordinates in the field coordinate system. The head target with the smallest distance is selected as the pre-matching head target for the pedestrian target.

[0133] If the distance *d* between the pedestrian target and the corresponding pre-matched head target is less than or equal to a second preset threshold (e.g., the second preset threshold could be 0.1 or other values), then the pedestrian target and the pre-matched head target are considered to be successfully matched. This indicates that the pedestrian target and the corresponding head target are the same pedestrian detection result. These pedestrian targets can be used as the fourth filtering result. In this case, the pedestrian target can be used as the pedestrian detection result.

[0134] Alternatively, if the distance d is greater than the second preset threshold, the match fails, indicating that the pedestrian target and the head target may be marked as different pedestrians. In this case, the head targets that failed to match can be used as the third filtering result.

[0135] By using the head target of the first image recognition to match the pedestrian target in the first screening result, a second matching process is achieved, which can effectively remove "missed detections caused by pedestrian overlap in the field" and improve the detection rate of "pedestrian detection boxes in the state of pedestrian overlap".

[0136] In some exemplary embodiments, pedestrian targets from the second and fourth screening results described above can be used as pedestrian detection results.

[0137] Alternatively, in some exemplary embodiments, steps S61-S63 and S71-S73 described above can be repeated n times to obtain n sets of statistical results; these n sets of statistical results are then compared. For example, n can be 3 or other values. If a pedestrian is detected as a pedestrian more than or equal to 2 times, then the pedestrian can be confirmed as the final pedestrian detection result.

[0138] In this example implementation, specifically for step S133, after filtering out the head targets that failed to match according to the second matching process in step S132, these head targets can be compared with the pedestrian targets in the second filtering result of the first matching process in step S131 to perform a third comparison, and use the head targets to verify another part of the matched pedestrian targets.

[0139] In this example implementation, refer to Figure 8 As shown, step S133 above may include:

[0140] Step S81: Based on the first head coordinates corresponding to the head target in the third filtering result and the third pedestrian coordinates corresponding to the pedestrian target in the second filtering result, calculate the distance between the first detection box corresponding to each head target and the third detection box of each pedestrian target respectively.

[0141] Step S82: Determine the pedestrian target with the smallest detection box distance corresponding to the head target, and when the detection box distance is less than or equal to a third preset threshold, configure the head target to match the pedestrian target;

[0142] Step S83, or, when the distance of the detection box is greater than the third preset threshold, configure the head target and the pedestrian target to be mismatched, and generate the fifth filtering result based on the mismatched head targets.

[0143] Specifically, for the head targets in the third filtering results, i.e., the head targets that failed to match in step S132, their first detection boxes in the scene coordinate system can be marked as G1l, G2l, ..., Gtl. The coordinates of the center point of the head target in the field coordinate system can be configured as the coordinates of the first detection box.

[0144] For the head targets in these third-selection results, they can be matched with the pedestrian targets already matched in the first-selection results, that is, the successfully matched pedestrian targets marked as PP1l, PP2l, ... PPcl. This allows us to determine whether the head targets in the third-selection results are the same pedestrians as the pedestrian targets in the first-selection results.

[0145] For example, the distance e between the first detection bounding boxes (labeled G1l, G2l, ..., Gtl) of these head targets and the third detection bounding boxes (labeled PP1l, PP2l, ..., PPcl) of the pedestrian targets in the first filtering result can be calculated in the scene coordinate system. Specifically, for each head target in the third filtering result, the distance e between it and the pedestrian targets in the first filtering result can be calculated using the center point coordinates, and the pedestrian target with the smallest distance can be selected as the pre-matching pedestrian target for the head target.

[0146] If the distance *e* between the head target and the corresponding pre-matched pedestrian target is less than or equal to the third preset threshold, it indicates that the head target and the pedestrian target have successfully matched and are marked as the same pedestrian detection result. For example, the aforementioned third preset threshold can be 0.1 or other values. In this case, these pedestrian targets that have been successfully verified using the head target can be used as the seventh screening result and as the final pedestrian detection result.

[0147] Alternatively, if the distance *e* between the head target and the corresponding pre-matched pedestrian target is greater than a third preset threshold, it indicates that the head target and the pedestrian target have failed to match, and different pedestrians are marked. For head targets that fail to match, the first detection box in the camera coordinate system can be marked as H1, H2, ..., Hj; the corresponding first detection box in the site coordinate system can be marked as H1l, H2l, ..., Hjl. These head targets that fail to match can be used as the fifth filtering result.

[0148] Through the above steps S81-S83, the third round of matching can be achieved, which uses the head target to verify the screened pedestrian target, thereby further improving the accuracy of pedestrian detection.

[0149] Based on the above, in some exemplary embodiments, steps S61-S63, S71-S73, and S81-S83 can be repeated n times to obtain n sets of statistical results; these n sets of statistical results are then compared. For example, n can be 3 or other values. If a target is detected as a pedestrian more than or equal to 2 times, then the target can be confirmed as the final pedestrian detection result, thus obtaining a more optimized pedestrian detection result.

[0150] In this example implementation, specifically for step S134 above, for the head target in the fifth screening result, the skeletal target in the first screening result can be used for matching again to achieve mutual verification between the head target and the skeletal target.

[0151] Specifically, refer to Figure 9 As shown, step S134 above may specifically include:

[0152] Step S91: Based on the first head coordinates corresponding to each head target in the fifth screening result and the second pedestrian coordinates corresponding to the skeletal targets in the first screening result, calculate the distance between a detection box corresponding to each head target and a second detection box corresponding to each skeletal target.

[0153] Step S92: Determine the skeletal target with the smallest detection box distance corresponding to the head target, and when the detection box distance is less than or equal to the fourth preset threshold, configure the head target to match the skeletal target, and use it as the sixth screening result.

[0154] Specifically, for the remaining head targets (labeled H1, H2, ..., Hj) after the previous step, i.e., the fifth screening result mentioned above, they can be matched and verified with the skeletal targets (labeled Z1, Z2, ..., Zm) in the first screening result generated in step S131. Specifically, for each head target in the fifth screening result, the distance p can be calculated between its first detection box (labeled H1l, H2l, ..., Hjl) in the field coordinate system and the second detection box (labeled Z1l, Z2l, ..., Zml) in the field coordinate system of each skeletal target in the first screening result. For example, the coordinates of the center point can be used as the coordinates of the detection box to calculate the distance p.

[0155] For each head target in the fifth screening result, the skeletal target with the smallest distance can be selected as the pre-matched skeletal target. If the distance p between the head target and the pre-matched skeletal target is less than or equal to the fourth preset threshold, it can be determined that the head target and the skeletal target are successfully matched and marked as the same pedestrian detection result, which can be used as the sixth screening result; and this result does not conflict with the above-mentioned second screening result, fourth screening result, and pedestrian detection result set. At this time, the sixth screening result can also be used as the final pedestrian detection result.

[0156] Alternatively, if the distance p is greater than the fourth preset threshold, then the match between the head target and the skeletal target is determined to be unsuccessful, and one or both are incorrect identification results. Regarding the distance, the aforementioned fourth preset threshold can be 0.1 or other values. This disclosure does not impose any special limitations on this.

[0157] In this example implementation, regarding step S135, after performing all or part of the matching actions described above, multiple filtering results can be obtained. Specifically, based on the above, the results of successfully pairing a pedestrian target with a skeletal target, a pedestrian target with a head target, and a head target with a skeletal target can be re-ID-labeled to obtain a final pedestrian target, and this target ID is labeled as ID.destination That is, the second, fourth, seventh, and sixth screening results can be used as the final pedestrian detection results. Through the above step S134, an additional target inspection can be performed, which can effectively remove "missed detections caused by overlapping people" and improve the detection rate of "pedestrian detection boxes in the case of overlapping people".

[0158] In this example implementation, the above method may further include: repeatedly executing the pedestrian detection method n times to obtain n sets of target comparison results; statistically analyzing the n sets of target comparison results, and if a target is marked as a pedestrian detection result more than or equal to m times, then the target is marked as a pedestrian target; where m and n are positive integers.

[0159] Specifically, steps S61-S63, S71-S73, S81-S83, and S91-S92 can be repeated n times to obtain n sets of statistical results; these n sets of statistical results are then compared. For example, n can be 3 or other values. If a target is detected as a pedestrian more than or equal to 2 times, then the target can be confirmed as the final pedestrian detection result; thus, a further optimized pedestrian detection result can be obtained.

[0160] In this example implementation, reference Figure 10As shown, after simultaneously acquiring first and second images using the first and second camera devices, and using a head detection model to identify the head target in the first image, a skeletal target to identify the skeletal target in the second image, and a pedestrian target to obtain the corresponding pedestrian target using a pedestrian detection model, the following steps can be performed: S111, calculate the IOU value for the detection box CC of each pedestrian target and the detection box BB of each skeletal target; S112, for the same pedestrian ID, find the skeletal ID with the largest IOU value; S113, determine whether the IOU value is greater than... 0.8; S114, if yes, then mark the two as detection results of the same target and complete the pairing; or, S115, if no, then separately mark the pedestrian targets and skeletal targets that have not completed the pairing, to obtain pedestrian targets Y1, Y2...Yn; skeletal targets Z1, Z2...Zn; S116, map the remaining pedestrian targets (Y1, Y2...Yn) to the field coordinate system (Y1l, Y2l...Ynl), compare them with the head target, and obtain the distance d between each pedestrian target and the head target; S117, determine whether the distance d is less than or equal to 0.1; S1 27. If yes, mark the pedestrian target and the head target as the same target; or, S118. If no, calculate the distance between the remaining head targets and the paired pedestrian targets, and record it as distance e; S119. Determine whether the distance e is less than or equal to 0.1; S120. If yes, mark the two as the detection results of the same target, and complete the pairing; or, S121. Mark the head targets and skeletal targets that have not been paired separately, and obtain head targets H1, H2, ..., Hj; skeletal targets I1, I2, ..., Ik; S122. Mark the unpaired head targets and skeletal targets separately, and obtain head targets H1, H2, ..., Hj; skeletal targets I1, I2, ..., Ik; S122. The coordinates H1l, H2l, ..., Hjl of the head target in the field coordinate system are compared with the coordinates I1l, I2l, ..., Ikl of the skeletal target in the field coordinate system to obtain the distance P between each head target and skeletal target; S123, determine whether the distance P is less than or equal to 0.1; S124, if yes, mark the two as the same target detection result and complete the pairing; or S125, if no, mark the two as different target detection results and the pairing is not completed; S126, reassign ID to each pedestrian target or skeletal target. Furthermore, the above steps can be repeated 3 times, and the pairing results of the 3 times are counted. If a target is marked as a confirmed pedestrian detection result more than or equal to 2 times, then the target can be identified as the final pedestrian target.

[0161] The pedestrian detection method provided in this disclosure involves placing camera A at the center of the target site to capture images from above, and simultaneously placing camera B on the side of the site to capture images from the side. The focal lengths of the two cameras are adjusted to ensure that their images cover the entire site. By using both cameras—camera A (from above) and camera B (from the side)—to acquire images in real time, a head model is used to perform head detection on the images captured by camera A, obtaining the head detection result and ID number. The actual ground position coordinates in the site coordinate system are then obtained through calibration. Simultaneously, a pedestrian detection model is used to perform pedestrian detection on the images captured by camera B, obtaining the pedestrian detection result, ID number, and the actual ground position coordinates in the site coordinate system through calibration. A skeletal point detection model is used to perform skeletal point detection on the images captured by camera B, obtaining the skeletal point detection result, ID number, and the actual ground position coordinates in the site coordinate system through calibration. By correlating the positional results of these three detections, the final pedestrian detection target and ID number are obtained. This solution effectively avoids false detections of distant pedestrians (e.g., pillars) and missed detections due to overlapping pedestrians.

[0162] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may, for example, be executed synchronously or asynchronously in multiple modules.

[0163] Further reference Figure 11 As shown, this example embodiment also provides a pedestrian detection device 110, which can be applied to a server or terminal device; the device includes: a first camera coordinate calculation module 1101, a second camera coordinate calculation module 1102, and a coordinate comparison module 1103. Wherein,

[0164] The first camera coordinate calculation module 1101 can be used to perform head recognition on the first image captured by the first camera device in the first shooting direction using a head detection model, so as to obtain the first head coordinates of each head target in the first image.

[0165] The second camera coordinate calculation module 1102 can be used to identify the second image acquired by the second camera device in the second shooting direction using the skeleton point detection model and the pedestrian detection model, respectively, so as to obtain the second skeleton coordinates of the skeleton target and the third pedestrian coordinates of the pedestrian target in the second image.

[0166] The coordinate comparison module 1103 can be used to compare any two or more of the head target, skeletal target and pedestrian target using coordinate data, and determine the pedestrian detection result based on the target comparison result.

[0167] Further reference Figure 12 As shown, the coordinate comparison module 1103 may include: a first comparison module 1201, a second comparison module 1202, a third comparison module 1203, a fourth comparison module 1204, and a result output module 1205. Among them,

[0168] The first comparison module 1201 can be used to calculate the corresponding IOU value based on the skeletal target and pedestrian target corresponding to the second image, and use the IOU value to filter the pedestrian targets in the second image to obtain the first filtering result of target mismatch and the second filtering result of target matching.

[0169] The second comparison module 1202 is used to perform distance comparison using the third pedestrian coordinates of the pedestrian target marked in the first filtering result and the first head coordinates of the head target corresponding to the first image, so as to obtain the third filtering result based on the mismatched comparison result and generate the fourth filtering result based on the matching comparison result.

[0170] The third comparison module 1203 can be used to perform distance comparison based on the third pedestrian coordinates of the marked pedestrian targets in the second filtering result and the first head coordinates of the marked head targets in the third filtering result, and generate a fifth filtering result based on the mismatch comparison result.

[0171] The fourth comparison module 1204 can be used to perform distance comparison based on the first head coordinates of the head target in the fifth screening result and the second bone coordinates of the marked bone target in the first screening result, so as to obtain a sixth screening result that is successfully matched.

[0172] The result output module 1205 can be used to determine the pedestrian detection result based on at least one of the second screening result, the fourth screening result, and the sixth screening result.

[0173] In some exemplary embodiments, the first camera coordinate calculation module 1101 may include a head target calculation module.

[0174] The head target calculation module can be used to perform coordinate system calibration based on the first camera coordinate system of the first camera device and the site coordinate system corresponding to the target site to obtain the corresponding first transformation matrix; use the head detection model to perform head recognition on the first image acquired by the first camera device to obtain the head target in the first image and the corresponding first detection box; calculate the center point coordinates in the corresponding first camera coordinate system according to the first detection box, and use the first transformation matrix to calculate the center point coordinates in the corresponding site coordinate system, and configure them as the first head coordinates; configure the corresponding identity identifier for each head target.

[0175] In some exemplary embodiments, the second camera coordinate calculation module 1102 may include a skeleton target calculation module.

[0176] The skeletal target calculation module can be used to perform coordinate system calibration based on the second camera coordinate system of the second camera device and the site coordinate system corresponding to the target site to obtain the corresponding second transformation matrix; use a skeletal point detection model to perform skeletal point detection on the second image acquired by the second camera device to obtain skeletal targets and corresponding second detection boxes in the second image; when the skeletal target includes a head skeleton, calculate the center point coordinates of the skeletal target in the second camera coordinate system by combining the coordinates of the head skeleton; or, when the skeletal target does not include a head skeleton, calculate the center point coordinates of the skeletal target in the second camera coordinate system by using the average value of the coordinates of each skeletal point; calculate the corresponding center point coordinates in the site coordinate system using the second transformation matrix for the center point coordinates in the second camera coordinate system, and configure them as the second skeletal coordinates; configure a corresponding identity identifier for each skeletal target.

[0177] In some exemplary embodiments, the second camera coordinate calculation module 1102 may further include a pedestrian target calculation module.

[0178] The pedestrian target calculation module can be used to perform coordinate system calibration based on the second camera coordinate system of the second camera device and the site coordinate system corresponding to the target site to obtain the corresponding second transformation matrix; use the pedestrian detection model to perform pedestrian detection on the second image acquired by the second camera device to obtain pedestrian targets in the second image and the corresponding third detection box; calculate the center point coordinates in the second camera coordinate system corresponding to the third detection box, and use the second transformation matrix to calculate the center point coordinates in the corresponding site coordinate system, and configure them as the third pedestrian coordinates; configure corresponding identity identifiers for each pedestrian target.

[0179] In some exemplary embodiments, the first comparison module 1201 may include: calculating corresponding IOU values ​​using the second detection boxes of each skeletal target corresponding to the second image and the third detection boxes of each pedestrian target; determining the skeletal target with the largest IOU value for each pedestrian target, and configuring the pedestrian target and the skeletal target to match when the IOU value is greater than or equal to a first preset threshold, and adding a mark to it and configuring it as a second filtering result; or, configuring the pedestrian target and the skeletal target to not match when the IOU value is less than the first preset threshold, adding a mark to it and configuring it as a first filtering result.

[0180] In some exemplary embodiments, the second comparison module 1202 may include: calculating the distance between the third detection box corresponding to each pedestrian target and the first detection box of each head target based on the third pedestrian coordinates of each pedestrian target in the field coordinate system and the first head coordinates of each head target in the field coordinate system; determining the head target with the smallest detection box distance for each pedestrian target, and configuring the pedestrian target and the head target to match when the detection box distance is less than or equal to a second preset threshold, and adding a mark to it and configuring it as a fourth filtering result; or, configuring the pedestrian target and the head target to not match when the detection box distance is greater than the second preset threshold, adding a mark to it and configuring it as a third filtering result.

[0181] In some exemplary embodiments, the third comparison module 1203 may include: calculating the distance between the first detection box corresponding to each head target and the third detection box of each pedestrian target based on the first head coordinates corresponding to the head target in the third filtering result and the third pedestrian coordinates corresponding to the pedestrian target in the second filtering result; determining the pedestrian target with the smallest detection box distance corresponding to the head target, and configuring the head target and the pedestrian target to match when the detection box distance is less than or equal to a third preset threshold; or configuring the head target and the pedestrian target to not match when the detection box distance is greater than the third preset threshold, and generating the fifth filtering result based on the mismatched head targets.

[0182] In some exemplary embodiments, the fourth comparison module 1204 may include: calculating the distance between a detection box corresponding to each head target and a second detection box corresponding to each skeletal target based on the first head coordinates corresponding to each head target in the fifth filtering result and the second pedestrian coordinates corresponding to each skeletal target in the first filtering result; determining the skeletal target with the smallest detection box distance corresponding to the head target, and configuring the head target to match the skeletal target when the detection box distance is less than or equal to a fourth preset threshold, and using it as the sixth filtering result.

[0183] In some exemplary embodiments, the pedestrian detection device 110 may further include an execution control module.

[0184] The execution control module can be used to repeatedly execute the above pedestrian detection method n times to obtain n sets of target comparison results; statistically analyze the n sets of target comparison results, and if a target is marked as a pedestrian detection result more than or equal to m times, then the target is marked as a pedestrian target; where m and n are positive integers.

[0185] In some exemplary embodiments, the first camera device is positioned at the top center of the target site, and the first shooting direction is a top-down shooting direction; the second camera device is positioned on one side of the target site, and the second shooting direction is a shooting direction facing the side of the target site.

[0186] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0187] Figure 13 A schematic diagram of an electronic device suitable for implementing embodiments of the present invention is shown.

[0188] It should be noted that, Figure 13 The illustrated electronic device 1000 is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0189] like Figure 13 As shown, the electronic device 1000 includes a Central Processing Unit (CPU) 1001, which can perform various appropriate actions and processes according to a program stored in a Read-Only Memory (ROM) 1002 or a program loaded from a storage section 1008 into a Random Access Memory (RAM) 1003. For example, the CPU 1001 can perform actions such as... Figures 1-11 The steps shown are used to implement the pedestrian detection method described above. RAM 1003 also stores various programs and data required for system operation. CPU 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. Input / output (I / O) interface 1005 is also connected to bus 1004.

[0190] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1010 as needed so that computer programs read from them can be installed into storage section 1008 as needed.

[0191] In particular, according to embodiments of the present invention, the processes described below with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs various functions defined in the system of this application.

[0192] Specifically, the aforementioned electronic device can be a smart device such as a server, tablet, or laptop, and can execute the aforementioned IoT device interaction management method applied to a proxy server or IoT platform. Alternatively, the aforementioned electronic device can also be an IoT device, capable of executing the aforementioned IoT device interaction management method applied to IoT devices.

[0193] It should be noted that the storage medium shown in the embodiments of the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, wherein computer-readable program code is carried. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the storage medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0194] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0195] The units described in the embodiments of the present invention can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0196] It should be noted that, as another aspect, this application also provides a storage medium, which may be included in an electronic device or may exist independently without being assembled into the electronic device. The aforementioned storage medium carries one or more programs, which, when executed by an electronic device, cause the electronic device to implement the methods described in the following embodiments. For example, the electronic device may implement various steps of a method applied to a proxy server, an IoT platform, or an IoT device.

[0197] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0198] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0199] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A pedestrian detection method, characterized in that, The method includes: A head detection model is used to perform head recognition on a first image captured by a first camera device in a first shooting direction, so as to obtain the first head coordinates of each head target in the first image; and... The second image acquired by the second camera device in the second shooting direction is identified using a skeleton point detection model and a pedestrian detection model, respectively, so as to obtain the second skeleton coordinates of the skeleton target and the third pedestrian coordinates of the pedestrian target in the second image; Based on the skeletal targets and pedestrian targets corresponding to the second image, the corresponding IOU values ​​are calculated, and the pedestrian targets in the second image are filtered using the IOU values ​​to obtain a first filtering result of target mismatch and a second filtering result of target matching; Distance comparison is performed using the third pedestrian coordinates of the marked pedestrian targets in the first filtering result and the first head coordinates of the head target corresponding to the first image, so as to obtain the third filtering result based on the mismatched comparison result and generate the fourth filtering result based on the matching comparison result. Based on the third pedestrian coordinates of the marked pedestrian targets in the second filtering result and the first head coordinates of the marked head targets in the third filtering result, a distance comparison is performed, and a fifth filtering result is generated based on the mismatch comparison results. Based on the first head coordinates of the head target in the fifth screening result and the second bone coordinates of the marked bone target in the first screening result, a distance comparison is performed to obtain a sixth screening result that is successfully matched; The pedestrian detection result is determined based on at least one of the second screening result, the fourth screening result, and the sixth screening result.

2. The pedestrian detection method according to claim 1, characterized in that, The step of using a head detection model to perform head recognition on the first image captured by the first camera device in the first shooting direction, to obtain the first head coordinates of each head target in the first image, includes: Coordinate system calibration is performed based on the first camera coordinate system of the first camera device and the site coordinate system corresponding to the target site to obtain the corresponding first transformation matrix; A head detection model is used to perform head recognition on the first image captured by the first camera device to obtain the head target in the first image and the corresponding first detection box; The center point coordinates in the first camera coordinate system are calculated based on the first detection frame, and the center point coordinates in the field coordinate system are calculated using the first transformation matrix and configured as the first head coordinates. Configure a corresponding identity identifier for each of the head targets.

3. The pedestrian detection method according to claim 1, characterized in that, The step of using a skeletal point detection model to identify the second image acquired by the second camera device in the second shooting direction, and obtaining the second skeletal coordinates of the skeletal target in the second image, includes: Coordinate system calibration is performed based on the second camera coordinate system of the second camera device and the site coordinate system corresponding to the target site to obtain the corresponding second transformation matrix; A skeletal point detection model is used to detect skeletal points in the second image acquired by the second camera device in order to obtain the skeletal targets and corresponding second detection boxes in the second image. When the skeletal target includes a head skeleton, the center point coordinates of the skeletal target in the second camera coordinate system are calculated by combining the coordinates of the head skeleton; or, when the skeletal target does not include a head skeleton, the center point coordinates of the skeletal target in the second camera coordinate system are calculated by using the average value of the coordinates of each bone point. The center point coordinates in the second camera coordinate system are calculated using the second transformation matrix, and then configured as the second skeleton coordinates. Configure a corresponding identity for each of the skeletal targets.

4. The pedestrian detection method according to claim 1, characterized in that, The step of using a pedestrian detection model to identify the second image captured by the second camera device in the second shooting direction, and obtaining the third pedestrian coordinates of the pedestrian target in the second image, includes: Coordinate system calibration is performed based on the second camera coordinate system of the second camera device and the site coordinate system corresponding to the target site to obtain the corresponding second transformation matrix; A pedestrian detection model is used to detect pedestrians in the second image captured by the second camera device, so as to obtain pedestrian targets in the second image and their corresponding third detection boxes; The center point coordinates in the second camera coordinate system are calculated based on the third detection frame, and the center point coordinates in the field coordinate system are calculated using the second transformation matrix and configured as the third pedestrian coordinates. Configure corresponding identity identifiers for each of the aforementioned pedestrian targets.

5. The pedestrian detection method according to claim 1, characterized in that, The step of calculating the corresponding IOU value based on the skeletal target and pedestrian target in the second image, and using the IOU value to filter pedestrian targets in the second image to obtain a first filtering result of target mismatch and a second filtering result of target matching includes: The corresponding IOU values ​​are calculated using the second detection boxes of each skeletal target in the second image and the third detection boxes of each pedestrian target. For each pedestrian target, determine the skeletal target with the largest IOU value, and when the IOU value is greater than or equal to a first preset threshold, configure the pedestrian target to be matched with the skeletal target, add a mark and configure it as the second filtering result; Alternatively, when the IOU value is less than the first preset threshold, the pedestrian target and the skeleton target are configured to not match, and the target is marked and configured as the first filtering result.

6. The pedestrian detection method according to claim 1, characterized in that, The step of performing distance comparison using the third pedestrian coordinates of the marked pedestrian targets in the first filtering result and the first head coordinates of the head target corresponding to the first image, to obtain a third filtering result based on the mismatched comparison results and to generate a fourth filtering result based on the successfully matched comparison results includes: Based on the third pedestrian coordinates of the pedestrian target in the field coordinate system and the first head coordinates of each head target in the field coordinate system in the first screening result, the distance between the third detection box corresponding to each pedestrian target and the first detection box of each head target is calculated respectively. For each pedestrian target, the head target with the smallest detection box distance is determined, and when the detection box distance is less than or equal to the second preset threshold, the pedestrian target is configured to match the head target, and a mark is added to it and configured as the fourth filtering result; Alternatively, when the distance between the detection boxes is greater than the second preset threshold, the pedestrian target and the head target are configured to not match, and they are marked and configured as the third filtering result.

7. The pedestrian detection method according to claim 1, characterized in that, The step of performing a distance comparison based on the third pedestrian coordinates of the marked pedestrian targets in the second filtering result and the first head coordinates of the marked head targets in the third filtering result, and generating a fifth filtering result based on the mismatch comparison results, includes: Based on the first head coordinates corresponding to the head target in the third filtering result and the third pedestrian coordinates corresponding to the pedestrian target in the second filtering result, the distance between the first detection box corresponding to each head target and the third detection box corresponding to each pedestrian target is calculated respectively. The pedestrian target with the smallest detection box distance corresponding to the head target is determined, and when the detection box distance is less than or equal to a third preset threshold, the head target is configured to match the pedestrian target; Alternatively, when the distance between the detection boxes is greater than the third preset threshold, the head target and the pedestrian target are configured to not match, and the fifth filtering result is generated based on these mismatched head targets.

8. The pedestrian detection method according to claim 1, characterized in that, The step of performing a distance comparison based on the first head coordinates of the head target in the fifth filtering result and the second bone coordinates of the marked skeletal target in the first filtering result to obtain a successfully matched sixth filtering result includes: Based on the first head coordinates corresponding to each head target in the fifth screening result and the second pedestrian coordinates corresponding to the skeletal targets in the first screening result, the distance between a detection box corresponding to each head target and the second detection box corresponding to each skeletal target is calculated respectively. The skeletal target with the smallest detection box distance corresponding to the head target is determined, and when the detection box distance is less than or equal to a fourth preset threshold, the head target is configured to match the skeletal target, and this is used as the sixth screening result.

9. The pedestrian detection method according to claim 1, characterized in that, The method further includes: Repeat the above pedestrian detection method n times to obtain n sets of target comparison results; Statistically analyze the comparison results of n groups of targets. If a target is marked as a pedestrian detection result more than or equal to m times, then the target is marked as a pedestrian target; where m and n are positive integers.

10. A pedestrian detection device, characterized in that, The device includes: The first camera coordinate calculation module is used to perform head recognition on the first image captured by the first camera device in the first shooting direction using a head detection model, so as to obtain the first head coordinates of each head target in the first image. The second camera coordinate calculation module uses a skeleton point detection model and a pedestrian detection model to identify the second image acquired by the second camera device in the second shooting direction, so as to obtain the second skeleton coordinates of the skeleton target and the third pedestrian coordinates of the pedestrian target in the second image. The coordinate comparison module is used to calculate the corresponding IOU value based on the skeletal target and pedestrian target corresponding to the second image, and use the IOU value to filter the pedestrian targets in the second image to obtain a first filtering result of target mismatch and a second filtering result of target match; it performs distance comparison using the third pedestrian coordinates of the marked pedestrian targets in the first filtering result and the first head coordinates of the head target corresponding to the first image to obtain a third filtering result based on the mismatch comparison result and a fourth filtering result based on the successful matching comparison result; it performs distance comparison using the third pedestrian coordinates of the marked pedestrian targets in the second filtering result and the first head coordinates of the marked head targets in the third filtering result to generate a fifth filtering result based on the mismatch comparison result; it performs distance comparison using the first head coordinates of the head targets in the fifth filtering result and the second skeletal coordinates of the marked skeletal targets in the first filtering result to obtain a sixth filtering result of successful matching; and it determines the pedestrian detection result based on at least one of the second filtering result, the fourth filtering result, and the sixth filtering result.

11. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the pedestrian detection method as described in any one of claims 1 to 9.

12. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the pedestrian detection method of any one of claims 1 to 9 by executing the executable instructions.

Images