Multi-target tracking and early warning method, device and terminal equipment based on visual recognition

Abstract

This application relates to the field of image processing technology and provides a method, apparatus, and terminal device for multi-target tracking and early warning based on visual recognition. The method includes: acquiring a first image to be identified, which contains multiple targets to be identified; identifying the image to be identified and assigning a unique identification marker to each target based on its body characteristics; predicting the next target region of the first target to be identified based on its motion trajectory; acquiring a second image to be identified using an image acquisition device of the next target region; and tracking and identifying the first target based on the second image. This application eliminates the need for manual visual monitoring of multiple targets, thereby improving the efficiency and accuracy of multi-target tracking.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to a method, apparatus and terminal equipment for multi-target tracking and early warning based on visual recognition. Background Technology

[0002] With the rapid development of technology and the economy, large areas (such as shopping malls) are increasing in number and size, leading to higher and higher requirements for monitoring in these areas. Taking shopping malls as an example, current monitoring systems are relatively simple, primarily relying on manual visual inspection of the monitoring screens to assess security. However, with the increasing number of shoppers, traditional manual visual monitoring methods have security limitations, and inadequate oversight can easily lead to security incidents. Therefore, traditional manual visual monitoring is no longer sufficient to meet the monitoring needs of large areas, and there is an urgent need to improve the intelligence level of video surveillance systems in these areas. Summary of the Invention

[0003] To overcome the problems existing in related technologies, embodiments of this application provide a multi-target tracking and early warning method, device and terminal equipment based on visual recognition.

[0004] This application is achieved through the following technical solution:

[0005] In a first aspect, embodiments of this application provide a multi-target tracking and early warning method based on visual recognition, comprising: acquiring a first image to be identified, wherein the first image to be identified is an image acquired in a first target region, and the first image to be identified contains multiple targets to be identified; recognizing the image to be identified, setting a unique identification identifier for each target to be identified based on body characteristics, and storing the body characteristics corresponding to each identification identifier, wherein the body characteristics include at least one of facial features, limb features, and action features; predicting the next target region of the first target to be identified as a second target region based on the movement trajectory of the first target to be identified, wherein the first target to be identified is any one of the multiple targets to be identified; retrieving an image acquisition device of the second target region to acquire a second image to be identified, and tracking and identifying the first target to be identified based on the second image to be identified, wherein the second image to be identified contains the first target to be identified.

[0006] The aforementioned vision-based multi-target tracking and early warning method acquires a first image of a target region to be identified, and identifies the body features of each target in the first image. Then, based on the movement trajectory of the first target, it predicts the next target region. Next, based on the body features of the target, it identifies each target from a second image, thus achieving multi-target tracking, identification, and early warning. This eliminates the need for manual visual monitoring of multiple targets, improving the efficiency and accuracy of multi-target tracking.

[0007] In conjunction with the first aspect, in some embodiments, setting a unique identification identifier for each target to be identified based on body features includes: obtaining a first position of each head region and a second position of each limb region in the image to be identified; determining the positional relationship between each head region and each limb region based on the first position and the second position; determining the association relationship between each head region and each limb region based on the positional relationship between the head regions and each limb regions; determining multiple pairs of corresponding head regions and limb regions based on the association relationship; extracting head features of each pair of corresponding head regions and limb features of each pair of corresponding limb regions, and setting a unique identification identifier for the head features and limb features.

[0008] In conjunction with the first aspect, in some embodiments, the first position is a first region position of the head region, and the second position is a second region position of the limb region or a second position of multiple feature points of the limb region, wherein the limb region includes an upper limb region and a lower limb region; determining the association relationship between each head region and each limb region based on the positional relationship between each head region and each limb region includes: calculating the distance between the head region and the limb region, wherein the distance includes the distance from the center of the head region to the center of the limb region, and the maximum and minimum distances from the edge of the head region to the edge of the limb region; if the distance between the first head region and multiple second limb regions is less than a threshold, then calculating the angle between the first head region and each second limb region, wherein the angle is the angle between a first vertical line passing through the center of the head region and a second vertical line passing through the center of the limb region, wherein the first head region is any head region; determining the association relationship between the first head region and each second limb region based on the angle between the first head region and each second limb region.

[0009] In conjunction with the first aspect, in some embodiments, the first position is the position of a first feature point of a plurality of first feature points in the head region, and the second position is the position of a second feature point of a plurality of second feature points in the limb region; determining the association relationship between each head region and each limb region based on the positional relationship between each head region and each limb region includes: calculating the distance between the head region and the limb region, the distance including the distance between each first feature point and each second feature point; if the distance between the first head region and the plurality of second limb regions is less than a threshold, then calculating the angle between the first head region and each second limb region, the angle being the angle between a preset line connecting the first feature point and the second feature point and a vertical line, the first head region being any head region, the center of the head region; determining the association relationship between the first head region and each second limb region based on the angle between the first head region and each second limb region.

[0010] In conjunction with the first aspect, in some embodiments, before recognizing the image to be recognized, the method further includes a step of binarizing the image to be recognized; the binarization of the image to be recognized includes: for a head region in the image to be recognized, scanning the head region using a scanning window of a preset size, the head region including a face; if the brightness value of a pixel in the head region corresponding to the center of the scanning window is less than the brightness value of surrounding pixels, then setting the grayscale value of the pixel corresponding to the center of the scanning window to a first grayscale value; if the brightness value of a pixel in the head region corresponding to the center of the scanning window is greater than the brightness value of surrounding pixels, then setting the grayscale value of the pixel corresponding to the center of the scanning window to a second grayscale value.

[0011] In conjunction with the first aspect, in some embodiments, the binarization processing of the image to be identified further includes: for the outermost pixel in the head region, comparing the gray value of the pixel with a gray value threshold; if the gray value of the pixel is greater than or equal to the gray value threshold, then setting the gray value of the pixel to a second gray value; if the gray value of the pixel is less than the gray value threshold, then setting the gray value of the pixel to a first gray value.

[0012] In conjunction with the first aspect, in some embodiments, the image is converted into a grayscale image; pixels corresponding to preset rows and preset columns in the grayscale image are extracted; and the extracted pixels are combined to generate a new image to be recognized according to their positional relationship in the original image.

[0013] In conjunction with the first aspect, in some embodiments, each image acquisition device corresponds to a database for storing the acquired images, and the method further includes: if the target to be identified is a second target to be identified, setting a target storage space for the second target to be identified in the database, storing the image to be identified containing the second target to be identified into the target storage space; responding to a search instruction, the search instruction containing a target identification identifier corresponding to the second target to be identified; determining the target storage space in the database based on the target identification identifier; retrieving the image in the target storage space, and playing the image.

[0014] Secondly, embodiments of this application provide a multi-target tracking and early warning device based on visual recognition, comprising: an image acquisition module for acquiring a first image to be identified, wherein the first image to be identified is an image acquired in a first target region, and the first image to be identified contains multiple targets to be identified; an identification module for identifying the image to be identified, setting a unique identification identifier for each target to be identified based on body characteristics, and storing the body characteristics corresponding to each identification identifier, wherein the body characteristics include at least one of facial features, limb features, and action features; a prediction module for predicting the next target region of the first target to be identified as a second target region based on the movement trajectory of the first target to be identified, wherein the first target to be identified is any one of the multiple targets to be identified; and a tracking module for retrieving a second image to be identified from the image acquisition device of the second target region, and tracking and identifying the first target to be identified based on the second image to be identified, wherein the second image to be identified contains the first target to be identified.

[0015] Thirdly, embodiments of this application provide a terminal device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the multi-target tracking and early warning method based on visual recognition as described in any of the first aspects.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the multi-target tracking and early warning method based on visual recognition as described in any of the first aspects.

[0017] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute the multi-target tracking and early warning method based on visual recognition as described in any of the first aspects.

[0018] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0022] Figure 2 This is a schematic flowchart of a vision-based multi-target tracking and early warning method provided in an embodiment of this application;

[0023] Figure 3 yes Figure 2 A flowchart illustrating step 202 in the middle section;

[0024] Figure 4 This is a schematic diagram of a scene where a scanning window scans an image to be recognized, according to an embodiment of this application.

[0025] Figure 5 This is a schematic diagram of the pixels corresponding to the scanning window provided in one embodiment of this application;

[0026] Figures 6(a) and 6(b) are schematic diagrams of an image scaling scenario provided in an embodiment of this application;

[0027] Figure 7 This is a schematic diagram of the structure of the vision-based multi-target tracking and early warning device provided in the embodiments of this application;

[0028] Figure 8 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. Detailed Implementation

[0029] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0030] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0031] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0032] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0033] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0034] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0035] For example, embodiments of this application can be applied to, for example... Figure 1 The exemplary application scenario shown includes a first region, a second region, and a third region, each region being equipped with at least one image acquisition device. For example, the first region is equipped with a first camera 101, the second region with a second camera 102, and the third region with a third camera 103, each camera capable of capturing full-area images of its corresponding region.

[0036] The following explanation uses the example of users A, B, and C moving through the three areas mentioned above. Initially, users A, B, and C are located in the first area, and the image captured by the first camera 101 includes users A, B, and C. Then, users A and C move to the second area, and the image captured by the second camera 102 includes users A and C. Next, user C moves to the third area, and the image captured by the third camera 103 includes users C.

[0037] The camera captures images containing users and sends them to a server (not shown in the diagram). Based on these images, the server can perform multi-target tracking and monitoring of users A, B, and C. Additionally, the server can generate trajectories for users A, B, and C, and can lock onto the trajectory of a specific user.

[0038] The three areas mentioned above can be indoor areas such as shopping malls or outdoor areas such as plazas; this application embodiment does not limit this.

[0039] The following combination Figure 1 This application provides a detailed description of the multi-target tracking and early warning method based on visual recognition.

[0040] Figure 2 This is a schematic flowchart of a multi-target tracking and early warning method based on visual recognition provided in an embodiment of this application, with reference to... Figure 2 The following is a detailed description of the visual recognition-based multi-target tracking and early warning method:

[0041] Step 201: Acquire the first image to be identified.

[0042] The first image to be identified is an image captured in the first target area, and the first image to be identified contains multiple targets to be identified.

[0043] For example, an image acquisition device positioned within the target area can acquire images containing multiple targets to be identified. For instance, the image acquisition device can acquire images of the target area in real time, or acquire images of the target area at preset time intervals, or acquire images of the target area under certain conditions, or acquire images of the target area according to other rules; the embodiments described herein do not limit this.

[0044] Step 202: Recognize the image to be recognized, set a unique identification mark for each target to be recognized based on body characteristics, and store the body characteristics corresponding to each identification mark.

[0045] The physical features may include at least one of facial features, limb features, and movement features.

[0046] In some embodiments, see Figure 3 The implementation process of step 202 may include steps 2021 to 2025, as detailed below:

[0047] Step 2021: Obtain the first position of each head region and the second position of each limb region in the image to be identified.

[0048] Step 2022: Based on the first position and the second position, determine the positional relationship between each head region and each limb region.

[0049] The aforementioned positional relationships can include information such as distance, orientation, and angle between each head region and each limb region. The orientation between the head region and limb regions can be the orientation of the head region relative to the limb region, or the orientation of the limb region relative to the head region, in a global coordinate system. The angle between the head region and limb regions can be the included angle between the head region and the limb region in a global coordinate system.

[0050] Step 2023: Based on the positional relationship between each head region and each limb region, determine the association relationship between each head region and each limb region.

[0051] The relationship between the head region and the limb region can be either associated or unassociated. For example, if head region A is associated with limb region A, it means that head region A and limb region A correspond to the same human head and limbs. Conversely, if head region A and limb region B are unassociated, it means that head region A and limb region B do not correspond to the same human head and limbs.

[0052] Step 2024: Based on the aforementioned relationship, determine multiple pairs of corresponding head regions and limb regions.

[0053] Step 2025: Extract head features from each pair of corresponding head regions and limb features from each pair of limb regions, and set a unique identification identifier for each head feature and limb feature.

[0054] For example, head features may include multiple head feature points, and limb features may include multiple limb feature points. Therefore, a unique identification identifier can be set for each of the corresponding multiple head feature points and multiple limb feature points. In a new image to be identified, the identification identifier corresponding to each human body in the image is identified based on the aforementioned multiple head feature points and / or multiple limb feature points.

[0055] In one scenario, the first position is the first region position of the head region, and the second position is the second region position of the limb region or the second position of multiple feature points of the limb region, wherein the limb region includes the upper limb region and the lower limb region.

[0056] Correspondingly, the implementation process of step 2023 above may include: calculating the distance between the head region and the limb region, the distance including the distance from the center of the head region to the center of the limb region, and the maximum and minimum distances from the edge of the head region to the edge of the limb region; if the distance between the first head region and the multiple second limb regions is less than a first threshold, then calculating the angle between the first head region and each of the second limb regions, the angle being the angle between a first vertical line passing through the center of the head region and a second vertical line passing through the center of the limb region, the first head region being any head region; and determining the association relationship between the first head region and each of the second limb regions based on the angle between the first head region and each of the second limb regions.

[0057] The first threshold can be determined based on multiple samples, which are obtained by labeling the distance between the head region and the limb region in the image samples. The angle between the human head and limbs is usually within a certain range. If the angle between the first head region and a certain second limb region is within this range, it can be determined that the first head region and the second limb region are associated, that is, the first head region and the second limb region are the head and limb of the same human body.

[0058] In another scenario, the first position is the position of the first feature point of multiple first feature points in the head region, and the second position is the position of the second feature point of multiple second feature points in the limb region.

[0059] Correspondingly, the implementation process of step 1023 above may include: calculating the distance between the head region and the limb region, the distance including the distance between each first feature point and each second feature point; if the distance between the first head region and the multiple second limb regions is less than a second threshold, then calculating the angle between the first head region and each second limb region, the angle being the angle between the line connecting the first feature point and the second feature point and the vertical line, the first head region being any head region, the center of the head region; and determining the association relationship between the first head region and each second limb region based on the angle between the first head region and each second limb region.

[0060] The distance between the head region and the limb region can be the average or weighted average of the distances between each first feature point and each second feature point. The second threshold can be determined based on multiple samples, which are obtained by labeling the distances between the head region and the limb region in the image samples. The first threshold and the second threshold can be the same or different, and can be set according to the actual situation, which is not limited here.

[0061] For example, the line connecting the preset first feature point and the second feature point can be a line connecting one preset first feature point and one preset second feature point. Alternatively, the line connecting the preset first feature point and the second feature point can be a line connecting multiple first feature points and multiple second feature points. For example, weighted calculations can be performed on the multiple first feature points and the multiple second feature points to obtain two points, and then a line can be drawn connecting these two points to obtain the line connecting the preset first feature point and the second feature point. The vertical line can be a line perpendicular to the ground.

[0062] Step 203: Based on the motion trajectory of the first target to be identified, predict the next target area of ​​the first target to be identified as the second target area.

[0063] The first generation of identification target is any one of the plurality of targets to be identified.

[0064] In some embodiments, multiple walking positions of the first target to be identified within the target area can be collected, and the movement trajectory of the first target to be identified can be obtained based on these multiple walking positions. Then, based on the movement trajectory of the first target to be identified, the walking position of the first target to be identified is determined, thereby determining the next target area of ​​the first target to be identified.

[0065] In this embodiment, the motion trajectory of the first target to be identified obtained based on the multiple walking positions can be implemented based on existing related algorithms, which will not be elaborated here.

[0066] Step 204: The image acquisition device of the second target area is used to acquire the second image to be identified, and the first target to be identified is tracked and identified based on the second image to be identified.

[0067] The second image to be identified contains the first target to be identified.

[0068] In this step, after predicting that the first target to be identified has moved to the second target area, the second image to be identified acquired by the image acquisition device in the second target area is identified to determine the first generation of target to be identified, and the first target to be identified is continued to be tracked and monitored.

[0069] For example, the first target can be identified from the second image based on its body characteristics. The specific process will not be detailed here; please refer to the aforementioned related content. Additionally, the first target can be marked in real-time on a display device for easy viewing by staff.

[0070] In some embodiments, based on Figure 2 In the embodiment shown, before step 202, the above-mentioned multi-target tracking and early warning method based on visual recognition may further include a step of binarizing the image to be recognized.

[0071] For example, the process of binarizing the image to be identified includes: scanning the head region in the image to be identified using a scanning window of a preset size, the head region including the face; if the brightness value of the pixel corresponding to the center of the scanning window in the head region is less than the brightness value of the surrounding pixels, then the gray value of the pixel corresponding to the center of the scanning window is set to a first gray value; if the brightness value of the pixel corresponding to the center of the scanning window in the head is greater than the brightness value of the surrounding pixels, then the gray value of the pixel corresponding to the center of the scanning window is set to a second gray value.

[0072] The size of the scanning window can also be the size of n*m pixels, where n is an integer greater than or equal to 3 and m is an integer greater than or equal to 3. n and m can be the same or different.

[0073] See Figure 4 For example, the scanning window can be 3*3 pixels in size. The image to be recognized is scanned using a scanning window of size 3*3 pixels. The scanning process can be as follows: the scanning window is divided into three parts: top, middle, and bottom. The middle part of the scanning window is aligned with the first row of pixels in the image to be recognized. Then, starting from the first pixel, each pixel in the first row is scanned one by one, and then the pixels in the next row are scanned.

[0074] See Figure 5 Taking a 3x3 pixel scanning window as an example, the thick outline represents the scanning window. Pixels 1 to 9 within the scanning window are the nine pixels covered by the window, meaning the scanning window is currently scanning pixel 9 in the image to be recognized. Assuming the brightness values ​​of pixels 1 to 9 are l1 to l9 respectively, if the brightness value l5 of pixel 5 is less than or equal to the grayscale values ​​of the other eight pixels, then the grayscale value of pixel 5 is set to the first grayscale value, for example, 255. If the brightness value l5 of pixel 5 is less than or equal to the brightness values ​​of the other eight pixels, then the grayscale value of pixel 5 is set to the e-th grayscale value, for example, 0.

[0075] Considering that the face of the target to be identified may be under varying degrees of light, the acquired image is likely to be affected by this uneven illumination, potentially severely impacting the binarization process. If a single grayscale threshold is used to binarize all pixels, the resulting image is likely to be distorted and unable to accurately identify the face. To address this issue, the inventors employ a scanning window that scans and compares each pixel in the image to be identified, setting the grayscale value of each pixel to either a first or second grayscale value, thereby eliminating the influence of uneven lighting.

[0076] For example, if the target area is located outdoors, the location of the target area and the acquisition time can be used to determine whether there is uneven lighting in the image to be identified.

[0077] For example, if the target area is located outdoors where sunlight can directly reach it, and the collection time is during a period of good sunlight, there is a high probability of the above-mentioned uneven lighting. If the collection time is in the morning or evening, when the sunlight is weaker, there is basically no such uneven lighting.

[0078] For example, if the target area is located outdoors where sunlight can directly reach it, and the data collection time is during a period of good sunlight and it is a sunny day, then there is a high probability that the above-mentioned uneven lighting will exist; if the weather is cloudy, overcast, rainy, or snowy, then the above-mentioned uneven lighting will basically not exist.

[0079] For example, if the target area is located in a place where sunlight cannot directly reach it, then the above-mentioned uneven lighting situation will not exist.

[0080] If the aforementioned uneven lighting condition is confirmed, the binarization process of the image to be recognized described above can be performed. If the aforementioned uneven lighting condition is confirmed not to exist, a grayscale threshold can be used to binarize each pixel of the image to be recognized.

[0081] Optionally, the binarization processing of the image to be identified may further include: for the outermost pixels in the head region, comparing the gray value of the pixel with a gray value threshold; if the gray value of the pixel is greater than or equal to the gray value threshold, then setting the gray value of the pixel to a second gray value; if the gray value of the pixel is less than the gray value threshold, then setting the gray value of the pixel to a first gray value.

[0082] For example, when the target area is located outdoors, the aforementioned grayscale threshold can be determined based on the time, cloud cover, and location of the image to be identified, with each pair of time, cloud cover, and location corresponding to a grayscale threshold. Specifically, the stronger the illumination corresponding to each pair of time, cloud cover, and location, the smaller the grayscale threshold; conversely, the weaker the illumination corresponding to each pair of time, cloud cover, and location, the larger the grayscale threshold. Determining the grayscale threshold based on the time and cloud cover of the image to be identified can further reduce the impact of strong light irradiation.

[0083] In some embodiments, based on Figure 2 In the embodiment shown, before step 202, the above-mentioned visual recognition-based multi-target tracking and early warning method may further include a step of compressing the image to be recognized.

[0084] For example, the above-described process of compressing the image to be recognized may include: converting the image to be recognized into a grayscale image; extracting the pixels corresponding to preset rows and preset columns in the grayscale image; and combining the extracted pixels according to their positional relationship in the original image to generate a new image to be recognized.

[0085] The preset rows can be odd-numbered, even-numbered, or rows corresponding to other rules; the preset columns can be odd-numbered, even-numbered, or columns corresponding to other rules. For example, for rows near the edge of the image to be recognized, a row of pixels can be extracted every two rows; for rows near the center of the image to be recognized, a row of pixels can be extracted every other row. Similarly, for columns near the edge of the image to be recognized, a column of pixels can be extracted every two columns; for columns near the center of the image to be recognized, a column of pixels can be extracted every other column.

[0086] Referring to Figure 6(a), the pixels corresponding to even rows and even columns in the grayscale image are extracted; according to the positional relationship of each extracted pixel in the original image, the extracted pixels are combined to generate a new image to be recognized, as shown in Figure 6(b).

[0087] In some embodiments, each image acquisition device may correspond to a database for storing the acquired images, through which the required images and videos can be quickly searched and retrieved.

[0088] based on Figure 2In the embodiment shown, the above-mentioned multi-target tracking and early warning method based on visual recognition may further include: if the target to be identified is a second target to be identified, setting a target storage space for the second target to be identified in the database, and storing the image to be identified containing the second target to be identified into the target storage space; responding to a search instruction, the search instruction containing a target identification identifier corresponding to the second target to be identified; determining the target storage space in the database based on the target identification identifier; retrieving the image in the target storage space, and playing the image.

[0089] The second target to be identified can be a target that requires special attention. The image of the second target to be identified is stored in the target storage space of the database. Once it is necessary to view the historical image and video of the second target to be identified, the server can quickly determine the storage location of the historical image and video of the target based on the target identification identifier corresponding to the target, thereby quickly retrieving the image.

[0090] The aforementioned vision-based multi-target tracking and early warning method acquires a first image of a target region to be identified, and identifies the body features of each target in the first image. Then, based on the movement trajectory of the first target, it predicts the next target region. Next, based on the body features of the target, it identifies each target from a second image, thus achieving multi-target tracking, identification, and early warning. This eliminates the need for manual visual monitoring of multiple targets, improving the efficiency and accuracy of multi-target tracking.

[0091] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0092] Corresponding to the visual recognition-based multi-target tracking and early warning method described in the above embodiments, Figure 7 The diagram shows a structural block diagram of a multi-target tracking and early warning device based on visual recognition provided in an embodiment of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0093] See Figure 7 The multi-target tracking and early warning device based on visual recognition in this application embodiment may include an image acquisition module 301, a recognition module 302, a prediction module 303, and a tracking module 304.

[0094] The image acquisition module 301 is used to acquire a first image to be identified, which is an image acquired in a first target region and contains multiple targets to be identified. The identification module 302 is used to identify the image to be identified, assigning a unique identification identifier to each target based on body characteristics, and storing the body characteristics corresponding to each identifier. These body characteristics include at least one of facial features, limb features, and motion features. The prediction module 303 is used to predict the next target region of the first target to be identified as a second target region based on the movement trajectory of the first target. The first target to be identified is any one of the multiple targets to be identified. The tracking module 304 is used to retrieve a second image to be identified from the image acquisition device of the second target region, and to track and identify the first target based on the second image. The second image contains the first target to be identified.

[0095] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0096] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0097] This application also provides a terminal device, see [link to relevant documentation] Figure 8 The terminal device 400 may include at least one processor 410 and a memory 420, wherein the memory 420 stores a computer program that can run on the at least one processor 410, and the processor 410 executes the computer program to implement the steps in any of the above method embodiments, for example... Figure 2Steps 201 to 204 in the illustrated embodiment. Alternatively, when the processor 410 executes the computer program, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 7 The functions of modules 301 to 304 are shown.

[0098] For example, a computer program may be divided into one or more modules / units, one or more of which are stored in memory 420 and executed by processor 410 to complete this application. The one or more modules / units may be a series of computer program segments capable of performing a specific function, which are used to describe the execution process of the computer program in terminal device 400.

[0099] Those skilled in the art will understand that Figure 8 This is merely an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, buses, etc.

[0100] The processor 410 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0101] The memory 420 can be an internal storage unit of the terminal device or an external storage device, such as a plug-in hard drive, a smart media card (SMC), a secure digital card (SD), or a flash card. The memory 420 is used to store the computer program and other programs and data required by the terminal device. The memory 420 can also be used to temporarily store data that has been output or will be output.

[0102] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0103] The multi-target tracking and early warning method based on visual recognition provided in this application can be applied to terminal devices such as servers, computers, in-vehicle devices, tablets, laptops, netbooks, personal digital assistants (PDAs), augmented reality (AR) / virtual reality (VR) devices, and mobile phones. This application does not impose any restrictions on the specific type of terminal device.

[0104] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various embodiments of the vision-based multi-target tracking and early warning method.

[0105] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the various embodiments of the vision-based multi-target tracking and early warning method.

[0106] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0107] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0108] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0109] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0110] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0111] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A multi-target tracking and early warning method based on visual recognition, characterized in that, include: A first image to be identified is acquired, wherein the first image to be identified is an image acquired in a first target area, and the first image to be identified contains multiple targets to be identified; The image to be identified is identified, and a unique identification identifier is set for each target to be identified based on its body characteristics. The body characteristics corresponding to each identification identifier are stored. The body characteristics include at least one of facial features, limb features and action features. Based on the motion trajectory of the first target to be identified, the next target region of the first target to be identified is predicted to be the second target region, and the first target to be identified is any one of the plurality of targets to be identified; The image acquisition device of the second target area is used to acquire a second image to be identified, and the first target to be identified is tracked and identified based on the second image to be identified, wherein the second image to be identified contains the first target to be identified; The step of setting a unique identification identifier for each target based on its physical characteristics includes: Obtain the first position of each head region and the second position of each limb region in the image to be identified; Based on the first position and the second position, the positional relationship between each head region and each limb region is determined; Based on the positional relationship between each head region and each limb region, the association relationship between each head region and each limb region is determined; Based on the aforementioned relationship, multiple pairs of corresponding head regions and limb regions are identified; Extract head features from each pair of corresponding head regions and limb features from each pair of corresponding limb regions, and assign unique identification identifiers to the head features and limb features; The first position is the position of the first feature point of multiple first feature points in the head region, and the second position is the position of the second feature point of multiple second feature points in the limb region; The step of determining the association between each head region and each limb region based on the positional relationship between each head region and each limb region includes: Calculate the distance between the head region and the limb region, wherein the distance includes the distance between each first feature point and each second feature point; If the distance between the first head region and the multiple second limb regions is less than a threshold, then the angle between the first head region and each of the second limb regions is calculated. The angle is the angle between the line connecting the first feature point and the second feature point and the vertical line. The first head region is any head region. The relationship between the first head region and each of the second limb regions is determined based on the angle between the first head region and each of the second limb regions. The preset lines connecting the first feature points and the second feature points include lines connecting multiple first feature points and multiple second feature points; the process of determining the lines connecting the multiple first feature points and the multiple second feature points includes: Weighted calculations are performed on multiple first feature points and multiple second feature points respectively to obtain first weighted feature points and second weighted feature points; The line connecting the first weighted feature point and the second weighted feature point is defined as the line connecting multiple first feature points and multiple second feature points.

2. The multi-target tracking and early warning method based on visual recognition as described in claim 1, characterized in that, Before recognizing the image to be recognized, the method further includes a step of binarizing the image to be recognized; The binarization process of the image to be identified includes: For the head region in the image to be identified, a scanning window of a preset size is used to scan the head region, which includes the face. If the brightness value of the pixel in the head region corresponding to the center of the scanning window is less than the brightness value of the surrounding pixels, then the gray value of the pixel corresponding to the center of the scanning window is set to the first gray value. If the brightness value of the pixel corresponding to the center of the scanning window in the head is greater than the brightness value of the surrounding pixels, then the grayscale value of the pixel corresponding to the center of the scanning window is set to the second grayscale value.

3. The multi-target tracking and early warning method based on visual recognition as described in claim 2, characterized in that, The binarization process of the image to be identified further includes: For the outermost pixels in the head region, the gray value of the pixel is compared with a gray value threshold. If the gray value of the pixel is greater than or equal to the gray value threshold, the gray value of the pixel is set to a second gray value; if the gray value of the pixel is less than the gray value threshold, the gray value of the pixel is set to a first gray value.

4. The multi-target tracking and early warning method based on visual recognition as described in claim 1, characterized in that, Before recognizing the image to be recognized, the method further includes: Convert the image to be identified into a grayscale image; Extract the pixels corresponding to the preset rows and preset columns in the grayscale image; Based on the positional relationship of each extracted pixel in the original image, the extracted pixels are combined to generate a new image to be recognized.

5. The multi-target tracking and early warning method based on visual recognition as described in claim 1, characterized in that, Each image acquisition device corresponds to a database for storing the acquired images, and the method further includes: If the target to be identified is a second target to be identified, a target storage space is set up for the second target to be identified in the database, and the image to be identified containing the second target to be identified is stored in the target storage space; In response to a search instruction, the search instruction includes a target identification identifier corresponding to the second target to be identified; Based on the target identification identifier, the target storage space in the database is determined; Retrieve the image from the target storage space and play the image.

6. A terminal device, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 5.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 5.

Images