Target object tracking method and device, electronic equipment and readable storage medium

Abstract

The invention discloses a target object tracking method, a target object tracking device, electronic equipment and a readable storage medium, and belongs to the technical field of computers. The target object tracking method comprises the steps that at least two target objects are provided, each target object comprises a background part and a mark part which have different colors, each background part and each mark part have a single color, and the target part on each target object is set as one of the background part and the mark part, the non-target part on each target object is the other one of the background part and the mark part, the colors of the target parts are different, and the colors of the non-target parts of the target objects are the same; determining an HSV image of the target object based on the RGB image of the target object; determining the color of the target part of the target object based on the HSV image of the target object; and determining the region of the target part of which the color is determined as a target object. According to the embodiment of the invention, a plurality of target objects can be identified in the same image at the same time.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a target tracking method, a target tracking device, an electronic device and a readable storage device. Background Art

[0002] With the development of computer image processing technology, calibration plates play an increasingly important role in applications such as machine vision, image measurement, photogrammetry, and 3D reconstruction. For example, they can correct lens distortion, determine the conversion relationship between physical size and pixels, determine the relationship between the 3D geometric position of a point on the surface of a spatial object and its corresponding point in the image, and establish a geometric model of camera imaging.

[0003] However, currently most applications based on calibration plates are for the recognition of calibration patterns, such as squares, circles, or specified graphics. It is not possible to meet the scenario of simultaneously recognizing multiple calibration plates in the same image. For example, when the camera needs to track multiple similar objects at the same time or distinguish different surfaces of the same object (such as the surfaces in the front, back, left, right, up, and down directions), if the same calibration plate is used, the tracked objects or different surfaces of the same object cannot be distinguished; if different types of calibration plates are used, the maximum number of objects that can be tracked is limited. Summary of the invention

[0004] In order to solve at least one aspect of the above-mentioned problems and defects existing in the prior art, an embodiment of the present invention provides a target tracking method, a target tracking device and a cloud server to solve the problem of how to simultaneously determine or identify multiple calibration plates.

[0005] An object of the present invention is to provide a target tracking method.

[0006] Another object of the present invention is to provide a target tracking device.

[0007] Another object of the present invention is to provide an electronic device.

[0008] Another object of the present invention is to provide a readable storage medium.

[0009] According to one aspect of the present invention, a target tracking method is provided, comprising:

[0010] Providing at least two targets, each target comprising a background portion and a marking portion with different colors, the background portion and the marking portion both having a single color, the target portion on each target being set to be one of the background portion and the marking portion, and the non-target portion on each target being the other of the background portion and the marking portion, the target portion of each target having a different color, and the non-target portion of each target having the same color;

[0011] Determine the HSV image of the target object based on the RGB image of the target object;

[0012] Determine the color of the target part of the target object based on the HSV image of the target object;

[0013] The area of ​​the target portion whose color is determined is determined as the target object.

[0014] In some embodiments, determining the HSV image of the target object based on the RGB image of the target object includes: determining the HSV data of the target object based on the RGB data of the target object,

[0015] Determining the color of the target part of the target object based on the HSV image of the target object includes:

[0016] Determine a first correspondence between the HSV data of the target object and the HSV threshold ranges of different color series in a preset HSV threshold table;

[0017] The color of the target part of the target object is determined based on the first corresponding relationship.

[0018] In some embodiments, determining the color of the target portion of the target object based on the first corresponding relationship includes:

[0019] Determine whether the HSV data of all targets are within the HSV thresholds of different color series:

[0020] When the HSV data of all the target objects are within the HSV threshold range of different color series, the color corresponding to the HSV threshold range of the different color series is determined as the color of the target part of the target object;

[0021] When the HSV data of all target objects are not within the HSV threshold range of different color series, the first grayscale data is determined based on the HSV data of at least two target objects in the same color series, and the respective colors are determined by comparing the first grayscale data with the second grayscale data determined based on the respective corresponding RGB data.

[0022] In some embodiments, determining the area of ​​the target portion having the determined color as the target object comprises:

[0023] Determine a binary image of each color based on the image of the target portion whose color is determined;

[0024] The initial image of the target object is segmented using the binary image of each color, and the segmented area is determined as the target object.

[0025] In some embodiments, in the binary image, the value of the area with the determined color is 1, and the value of the remaining area is 0.

[0026] The binary image of each color is respectively calculated with the initial image of the target object, and the area with a true value is determined as the area of ​​the target part, and the area of ​​the target part is determined as the target object.

[0027] In some embodiments, the target object tracking method further includes:

[0028] determining a first number of target objects in an initial image of the target objects;

[0029] Determining a second number of the target object in a next frame image of the target object;

[0030] A relationship between the first number and the second number is determined, and the target object is tracked based on the relationship.

[0031] In some embodiments, tracking the target object according to the relationship includes:

[0032] When the second number is less than or equal to the first number,

[0033] Determine the position data of each target object in the initial image and the position data of each target object in the next frame image;

[0034] Determine the distance between the position data of the target object in the initial image and the position data of the target object in the next frame image, and use the position data with the closest distance as the pairing data;

[0035] The position data of the target object in the next frame image is used to update the position data of the target object in the initial image that is paired with the position data of the target object in the next frame image, and the position data of the unpaired target object is deleted.

[0036] In some embodiments, the position data of the target object includes coordinate data of a center point of a target portion of the target object in an image coordinate system.

[0037] In some embodiments, tracking the target object according to the relationship includes:

[0038] When the second number is greater than the first number,

[0039] Determine the color of the target part of the target object based on the next frame image, and determine the area where the target part is determined as the target object;

[0040] Determine the position data of all targets and the number of targets in the next frame image.

[0041] In some embodiments, the marking portion of the target object includes a regular pattern or an irregular pattern.

[0042] The regular graphics include at least one of a circle, an ellipse, a rectangle, a square, a polygon, a triangle, and ChArUco arranged in a regular pattern;

[0043] The irregular graphics include logos arranged in a regular pattern.

[0044] According to another aspect of the present invention, a target tracking device is provided, the target tracking device is suitable for tracking at least two targets, each target includes a background part and a marking part with different colors, the background part and the marking part each have a single color, the target part on each target is set to be one of the background part and the marking part, and the non-target part on each target is the other of the background part and the marking part, the target part of each target has a different color, and the non-target part of each target has the same color, the target tracking device includes:

[0045] a first determination module, the first determination module being configured to determine an HSV image of the target object based on an RGB image of the target object;

[0046] a second determination module connected to the first determination module, the second determination module being configured to determine a color of a target portion of the target object based on the HSV image of the target object;

[0047] The tracking module is connected to the second determination module, and is configured to determine the area of ​​the target part with the determined color as the target object.

[0048] In some embodiments, the tracking module is further configured to determine a first number of targets in an initial image of the target and a second number of targets in a next frame of image, determine a relationship between the first number and the second number, and track the target according to the relationship.

[0049] According to another aspect of the present invention, an electronic device is provided, comprising a memory and a processor, wherein the memory stores a program, wherein the processor implements the target tracking method described in any of the aforementioned embodiments when executing the program on the memory.

[0050] According to another aspect of the present invention, a readable storage medium is provided, in which a computer-readable program or instruction is stored. When the computer-readable program or instruction is executed by a processor, the target object tracking method described in any of the aforementioned embodiments is implemented.

[0051] The target tracking method, target tracking device, electronic device and readable storage medium according to the present invention have at least one of the following advantages:

[0052] (1) The target tracking method, target tracking device, electronic device, and readable storage medium of the present invention can simultaneously determine or identify targets that need to be distinguished by determining or identifying targets with target parts of different colors (such as calibration plates or tracked objects with calibration plates), thereby meeting the scenario of simultaneously identifying multiple targets (such as calibration plates) in the same image;

[0053] (2) The target tracking method, target tracking device, electronic device and readable storage medium of the present invention can meet the requirement for identifying multiple calibration plates by setting the number of different colors, eliminating the requirement for the maximum number of trackable objects;

[0054] (3) The target tracking method, target tracking device, electronic device and readable storage medium of the present invention can distinguish targets with similar colors to accurately identify or determine multiple targets;

[0055] (4) The target tracking method, target tracking device, electronic device and readable storage medium of the present invention can realize dynamic tracking of multiple targets;

[0056] (5) The target tracking method, target tracking device, electronic device and readable storage medium of the present invention are real-time and robust in the determination or identification process of multiple calibration plates. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] These and / or other aspects and advantages of the present invention will become apparent and readily understood from the following description of the preferred embodiments in conjunction with the accompanying drawings, in which:

[0058] Figure 1 A flow chart of a target object tracking method according to an embodiment of the present invention is shown;

[0059] Figure 2 A flow chart of a target tracking method according to another embodiment of the present invention is shown;

[0060] Figure 3-6 Shows the application Figure 1 The target object of the target object tracking method shown;

[0061] Figure 7 A schematic diagram of dynamic tracking according to an embodiment of the present invention is shown;

[0062] Figure 8 A target tracking device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0063] The technical solution of the present invention is further specifically described below by examples and in conjunction with the accompanying drawings. In the specification, the same or similar reference numerals indicate the same or similar components. The following description of the embodiments of the present invention with reference to the accompanying drawings is intended to explain the overall inventive concept of the present invention and should not be construed as a limitation of the present invention.

[0064] In an embodiment of the present invention, a target object tracking method is provided, which can simultaneously identify or distinguish multiple targets to achieve tracking of the targets.

[0065] The target object in the embodiment of the present invention includes an object that is expected to be tracked, such as a calibration plate, an object provided with a calibration plate, or different surfaces of the same object provided with a calibration plate.

[0066] like Figure 1 and Figure 2 As shown, the target tracking method includes:

[0067] Providing at least two targets, each target comprising a background portion and a marking portion with different colors, the background portion and the marking portion both having a single color, the target portion on each target being set to be one of the background portion and the marking portion, and the non-target portion on each target being the other of the background portion and the marking portion, the target portion on each target having a different color, and the non-target portion of each target having the same color;

[0068] Determine the HSV image of the target object based on the RGB image of the target object;

[0069] Determine the color of the target part of the target object based on the HSV image of the target object;

[0070] The area of ​​the target portion whose color is determined is determined as the target object.

[0071] The embodiments of the present invention can simultaneously determine or identify targets that need to be distinguished by determining or identifying targets with target parts of different colors (such as a calibration plate or a tracked object with a calibration plate), thereby meeting the scenario of simultaneously identifying multiple targets (such as a calibration plate) in the same image.

[0072] The embodiment of the present invention can meet the requirement for identifying multiple calibration plates by setting the number of different colors, thereby eliminating the requirement for the maximum number of trackable objects.

[0073] The embodiment of the present invention has real-time and robustness for the determination or identification process of multiple calibration plates.

[0074] The embodiment of the present invention needs to provide at least two targets so that different targets can be distinguished by setting different colors. Of course, the embodiment of the present invention does not limit the specific number of targets, and those skilled in the art can set it as needed.

[0075] Specifically, each target (e.g. Figure 3 The reference numeral 1 in the figure includes a background portion having a different color (eg Figure 3 11 in the figure) and the marked parts (e.g. Figure 3 As shown in the figure mark 12), the background part and the mark part each have a single color.

[0076] The pattern of the marking portion includes a regular pattern or an irregular pattern.

[0077] The regular graphics include at least one of a circle, an ellipse, a rectangle, a square, a polygon, a triangle, and ChArUco arranged in a regular pattern, such as Figure 3-6 shown.

[0078] Figure 3 An embodiment of the present invention is a checkerboard pattern. Figure 3 The pattern of the embodiment has the characteristics of high precision in the post-processing process, but it is necessary to ensure that the camera obtains a complete pattern image. Moreover, as the distance from the camera increases, the stability during the processing will decrease.

[0079] Figure 4 and Figure 5 The embodiment of the present invention is a circular pattern, and the difference between the two is that the arrangement rules of the circles are different. Figure 3 Compared to the example using Figure 4 and Figure 5 The pattern in the embodiment can improve the stability during the processing. Figure 4 and Figure 5 When you are viewing a pattern in the image, you also need to ensure that the camera obtains a complete image of the pattern.

[0080] Figure 6 An embodiment of is ChArUco. Figure 3-5 Compared with the pattern in the embodiment, Figure 6 The patterns of the embodiments allow processing of portions of an image based on the pattern.

[0081] The irregular graphics include asymmetrical polygons or asymmetrical curved edges arranged in a regular pattern. For example, the irregular graphics include logos arranged in a regular pattern. For patterns using irregular graphics, the freedom and recognition of product design can be increased.

[0082] The embodiments of the present invention do not limit the specific pattern of the marking part, and those skilled in the art can set it as needed.

[0083] The target part on each target object is set to one of the background part and the marking part, and the non-target part of each target object is the other of the background part and the marking part. The color of the target part on each target object is different, and the color of the non-target part of each target object is the same. That is, the target part and the non-target part correspond to the background part and the marking part of the target object respectively, and the target part corresponds to the background part or the marking part of these targets that are set with different colors, which is the focus of attention in the process of identifying the target object. When the background part of all targets is set with the same single color, the marking part of each target object will be set with a different single color, and in this case the marking part is the target part, and the background part is the non-target part. When the marking part of all targets is set with the same single color, the background part of each target object will be set with a different single color, and in this case the background part is the target part, and the marking part is the non-target part.

[0084] The target part should be made of different colors with large color differences, such as red, orange, yellow, green, cyan, blue, and purple, so as to distinguish and identify the preset colors in the subsequent steps. The greater the color difference, the more beneficial it is for the subsequent color distinction, which can improve the accuracy of the distinction.

[0085] The following uses Figure 3 All targets are described using Figure 3 Each target object 1 comprises a background portion 11 and a marking portion 12 .

[0086] In one example, the background parts 11 of all the targets 1 are set to the same single color, and the marking parts 12 of all the targets 1 are set to different single colors. For example, this example includes three targets, the background parts 11 of the three targets 1 are all set to white, and the marking parts 12 of the three targets are set to blue, orange, and green, respectively.

[0087] In another example, the marking parts 12 of all the targets 1 are set to the same single color, and the background parts 11 of all the targets 1 are set to different single colors. For example, this example includes three targets, the marking parts 12 of the three targets 1 are all set to black, and the background parts 11 of the three targets 1 are set to blue, orange, and green, respectively.

[0088] Furthermore, after the colors of the target part and the non-target part are pre-designed, the RGB values ​​of the selected or designed colors need to be recorded for subsequent identification.

[0089] After the design of the target object is determined (i.e., the design of the target object is completed), the HSV image of the target object is determined based on the RGB image of the target object. The HSV color space has more advantages than the RGB color space in the color segmentation of the image. However, in practice, it is more convenient to obtain the RGB image of the target object than the HSV image. Therefore, in order to accurately identify the marked parts of different colors, it is necessary to convert the RGB color space of the obtained RGB image into the HSV color space.

[0090] In one example, a designed target object is placed in a space, and a color camera is used to collect image information (e.g., an RGB image) in the space, i.e., an initial image. The image at this time can be an image acquired in real time, or a pre-stored image. The image can be in a picture format or a video format. The process of the camera collecting image information can be manually triggered or automatically performed.

[0091] Specifically, determining the HSV image of the target object based on the RGB image of the target object includes: determining the HSV data of the target object based on the RGB data of the target object.

[0092] The RGB data of the target object can be determined based on the RGB image of the target object. The data in the RGB image has three channels, namely, data information of R, G and B. The RGB data can be normalized by the following equations (1)-(3) to obtain R', G' and B'.

[0093] R'=R / 255 Formula (1)

[0094] G'=G / 255 Formula (2)

[0095] B'=B / 255 Formula (3)

[0096] Based on equations (4)-(5), the maximum value Cmax and the minimum value Cmin of R', G' and B' are determined.

[0097] Cmax=max(R',G',B') Formula (4)

[0098] Cmin=min(R',G',B') Formula (5)

[0099] Based on equation (6), the difference Δ between the maximum value Cmax and the minimum value Cmin is determined.

[0100] Δ=Cmax-Cmin Formula (6)

[0101] Determine the HSV data of the target object based on equations (7)-(9).

[0102]

[0103]

[0104] V=Cmax Formula (9)

[0105] After the HSV data of the target object is determined, the color of the target part of the target object can be determined based on the HSV image of the target object. Specifically, the process includes:

[0106] Determine a first correspondence between the HSV data of the target object and the HSV threshold ranges of different color series in a preset HSV threshold table;

[0107] The color of the target part of the target object is determined based on the first corresponding relationship.

[0108] In one example, a preset HSV threshold table may be provided. Table 1 gives an example of a preset HSV threshold table. In Table 1, HSV color space ranges of ten color series are provided, where h, s, and v represent hue, saturation, and tone, respectively, and the subscripts min and max represent the minimum and maximum values ​​corresponding to the color series, respectively. The HSV threshold range of each color series is determined by its hmin and hmax ranges, its smin and smax ranges, and its vmin and vmax ranges.

[0109] Table 1

[0110]

[0111] Of course, those skilled in the art can adjust the value ranges of different HSV color series as needed.

[0112] Determining the first corresponding relationship includes determining within which color series the HSV data of each target object corresponds. Specifically, it can be determined within which color series the HSV data of the target part of each target object corresponds. Because the target part of each target object is known before tracking, that is, whether the target part is a background part or a marking part is known, in the later tracking process, only the image data of the target part can be concerned with or determined. Only when the h of a target object is within the hmin and hmax range of a certain color series, s is within the smin and smax range of the certain color series, and v is within the vmin and vmax range of the certain color series, will the target object (specifically, the target part) be determined to be the corresponding color in Table 1. For example, for HSV (0, 100, 100), the HSV value corresponds to the red series in Table 1.

[0113] In order to complete the recognition of different targets in the same image, it is necessary to traverse the HSV data of all pixel points of the target in the same image, record and save the pixel points whose values ​​in the image are within the HSV threshold of a certain color series. That is, determine the first correspondence between the HSV data of all pixel points in the same image and the HSV threshold range of different color systems in the preset HSV threshold table.

[0114] Further, determining the color of the target part of the target object based on the first corresponding relationship includes: determining whether the HSV data of all the target objects are located within the HSV thresholds of different color series.

[0115] When the HSV data of all targets are within the HSV threshold range of different color series, the color corresponding to the HSV threshold range of the different color series is determined as the color of the target part of the target. That is, the colors of the target parts of all targets can be distinguished based on the first correspondence. For example, an example provides three targets, and the HSV data of the three targets correspond to the threshold range of the orange series, the threshold range of the blue series, and the threshold range of the green series (i.e., the first correspondence), so the HSV data of the three targets (specifically, the target parts) are all within the HSV threshold range of different color series, so that the colors of the target parts of the three targets can be determined as orange, blue, and green, respectively.

[0116] When the HSV data of all the targets are not within the HSV threshold range of different color series, the color of the target part of the target cannot be determined based on the current first correspondence (the colors are similar and cannot be distinguished), but it is necessary to determine the first grayscale data based on the HSV data of at least two targets in the same color series, and determine the respective colors by comparing the first grayscale data with the second grayscale data determined based on the respective corresponding RGB data. For all the targets in the HSV threshold range of different color series, the color corresponding to the HSV threshold range corresponding to the HSV data is determined as the color of the target part of the target.

[0117] For example, an example provides three targets, and it is known that the colors of the target parts of the three targets are dark red, light red, and blue respectively; the HSV data of one target (specifically, the target part) is within the threshold range of the blue series, and the color of the target part of the target is determined to be blue; the HSV data of the remaining two targets (specifically, the target parts) are both within the threshold range of the red series. Therefore, in this case, the HSV data of all targets are not all within the HSV threshold range of different color series, because the HSV data of two targets are both within the HSV threshold range of the same color series.

[0118] In this case, in order to distinguish or identify the target objects in the HSV threshold range of the same color series, as described above, the respective colors are determined by the first grayscale data and the second grayscale data.

[0119] Specifically, the HSV data of at least two targets in the same color series are converted into RGB data. The specific conversion process can adopt the existing method, which is not described in detail here. The RGB data is then converted into grayscale data, for example, by a weighted average method (such as formula (10)). Thus, the first grayscale data is obtained.

[0120] Gray value gray=0.299*R+0.578*G+0.114*B Formula (10)

[0121] As can be seen from the foregoing, the colors and RGB values ​​of the target parts and non-target parts of all targets are known. For example, for the aforementioned embodiment, it is known that the target parts of all targets are dark red, light red and blue. Based on the first corresponding relationship, the color of the target object that has been able to be determined or identified can be determined. For example, the aforementioned embodiment has identified that the color of the target part of a target object is blue. Then it can be determined that the colors of the targets in the same HSV threshold range are dark red and light red. Therefore, the second grayscale data can be determined based on the respective corresponding RGB data, for example, the corresponding grayscale data is determined based on the RGB data of dark red and light red. The method for determining the second grayscale data based on RGB data is the same as the method for determining the first grayscale data, and will not be repeated here. The second grayscale data is determined based on the RGB data for determining the color, so the second grayscale data is the grayscale data for determining the color. For example, in the aforementioned embodiment, the second grayscale data is the grayscale data of dark red and light red.

[0122] After determining the first grayscale data and the second grayscale data, the first grayscale data and the second grayscale data can be compared and analyzed to determine the color of the target part of the target object. For example, when comparing the first grayscale data with the second grayscale data, if a grayscale value in the first grayscale data is equal to or close to a grayscale value in the second grayscale data, the corresponding target part in the first grayscale data is determined to be the color corresponding to the grayscale value of the second grayscale data. In this way, the recognition of targets with similar colors can be achieved.

[0123] After the color of the target part is determined, the area of ​​the target part with the determined color is determined as the target object. Specifically, the process includes the following steps:

[0124] Determine a binary image of each color based on the image of the target portion whose color is determined;

[0125] The initial image of the target object is segmented using the binary image of each color, and the segmented area is determined as the target object.

[0126] In one example, the image of the target part can be an RGB image or an HSV image. In the aforementioned process of determining the color of the target part, the RGB image and the HSV image of the target part have been obtained. Therefore, the binary image of each color can be generated based on the RGB image or the HSV image.

[0127] Each color here refers to each color of all colors of all targets that have been determined.

[0128] In the image of the target part, a binary image of each color is determined for each color. For example, an example includes three targets, and the colors of the target parts of the three targets are blue, red, and green respectively; in the RGB image of the target part, all blue areas are assigned to 1 and the remaining areas are assigned to 0, thereby generating a blue binary image; then all red areas are assigned to 1 and the remaining areas are assigned to 0, thereby generating a red binary image; then all green areas are assigned to 1 and the remaining areas are assigned to 0, thereby generating a green binary image.

[0129] The binary image of each color is respectively compared with the initial image of the target object, and the area with true value (true if both values ​​are non-zero, false otherwise) is determined as the area of ​​the target part, and the area of ​​the target part is determined as the target object. In this way, the recognition of multiple targets in the same image is achieved.

[0130] Optionally, after the target object is identified, a detection method may be used to detect the identified area, and subsequent related applications may be performed after the detection is completed. For example, when the target object is a calibration plate, a known method may be used to detect the identified calibration plate.

[0131] In another example of the present invention, Figure 2 As shown, the target object tracking method also includes:

[0132] determining a first number of target objects in an initial image of the target objects;

[0133] Determining a second number of the target object in a next frame image of the target object;

[0134] A relationship between the first number and the second number is determined, and the target object is tracked based on the relationship.

[0135] In this way, the embodiment of the present invention realizes a dynamic tracking process by tracking the current target object according to the relationship between the first number and the second number.

[0136] After the objects in the initial image are determined or identified according to the aforementioned embodiments, the first number of the objects determined or identified can be known. For example, the first number of the objects can be determined by counting the number of colors of the identified target parts. The first number of the objects can be determined after the object detection passes. Of course, the first number of the objects can also be determined after the object is identified and without performing the object detection.

[0137] For the determination or recognition process of the target in the next frame of image, the target is not recognized based on the difference in color (such as the solution of the aforementioned embodiment), but is recognized based on the pattern or marked part of the target. For example, feature points can be set on the target, and the target is recognized based on the spatial data of the feature points in the pattern coordinate system, the intrinsic parameter data of the camera, the world coordinate system and the camera coordinate system. Of course, the embodiments of the present disclosure do not limit the specific recognition process, and those skilled in the art can also use other disclosed technologies to identify the number of targets in the next frame of image.

[0138] The target object is tracked by comparing the relationship (eg, size relationship) between the first number and the second number.

[0139] Specifically, when the second number is less than or equal to the first number, the target object determined in the next frame image has been determined or recognized in the initial image, and includes the following steps:

[0140] Determine the position data of each target object in the initial image and the position data of each target object in the next frame image;

[0141] Determine the distance between the position data of the target object in the initial image and the position data of the target object in the next frame image, and use the position data with the closest distance as the pairing data;

[0142] The position data of the target object in the next frame image is used to update the position data of the target object in the initial image that is paired with the position data of the target object in the next frame image, and the position data of the unpaired target object is deleted.

[0143] The position data of the target object includes the coordinate data of the center point of the target part of the target object in the image coordinate system. After determining the area of ​​the target part, the coordinate value of the area where the target part is located in the image coordinate system can be determined, such as the O-XY rectangular coordinate system; determine the coordinate value of the center point of the area where the target part is located, x mid =(x max +x min ) / 2,y mid =(y max +y min ) / 2,x max is the maximum coordinate value of the region on the x-axis of the image coordinate system, xmin is the minimum coordinate value of the region on the x-axis of the image coordinate system, and y max is the maximum coordinate value of the region on the y-axis of the image coordinate system, y min is the minimum coordinate value of the region on the y-axis of the image coordinate system; the coordinate value of the center point of the region is determined as the position data of the target object. In this way, the position data of each target object in the initial image and the position data of each target object in the next frame of image can be determined.

[0144] After determining the position data of the target object in the initial image and the position data of the target object in the next frame of image, determine the distance (e.g., Euclidean distance) between the position data in the two images. The Euclidean distance between the center points of the target areas in the two images can be calculated respectively. The data of the position with the closest distance (i.e., the smallest Euclidean distance value) is determined as the paired data.

[0145] For the paired data, the position data of the target object in the next frame image in the pairing is used to update the position data of the target object in the initial image in the pairing, and the position data of the unpaired target object is deleted, thereby realizing dynamic tracking of the target object.

[0146] by Figure 7 For example, three targets A, B and C are identified in the initial image, and the first number of targets in the initial image is three; three targets A', B' and C' are identified in the next frame image, and the second number of targets in the next frame image is three, and the first number is equal to the second number; the Euclidean distance between the center point of A and the center point of each target (A', B' and C') in the next frame image is determined, which can be expressed as AA', AB' and AC' respectively; the Euclidean distance between the center point of B and the center point of each target in the next frame image is determined, which can be expressed as BA', BB' and BC' respectively; the Euclidean distance between the center point of C and the center point of each target in the next frame image is determined, which can be expressed as CA', CB' and CC' respectively. Figure 7 In the example, AA' is the shortest among AA', AB' and AC', so the target A and A' are paired data, and the position data of A is updated by using the position data of A; BB' is the shortest among BA', BB' and BC', so the target B and B' are paired data, and the position data of B is updated by using the position data of B; CC' is the shortest among CA', CB' and CC', so the target C and C' are paired data, and the position data of C is updated by using the position data of C', thereby completing the tracking of the target.

[0147] When the second number is greater than the first number, a target object that is not identified or recognized in the initial image appears in the next frame of image, so it is necessary to identify the target object based on the color of the target part according to the method of the above embodiment. Specifically, the following process is included:

[0148] Determine the color of the target part of the target object based on the next frame image, and determine the area where the target part is determined as the target object. This process is referred to the above content and will not be repeated here;

[0149] Determine the position data of all targets and the number of targets in the next frame image.

[0150] After all the targets in the next frame image are identified based on color, it is necessary to determine the position data of all the targets in the next frame image and the number of the targets for use in subsequent dynamic tracking.

[0151] In this way, the image updated later can use the above method to update the position data of the target object, thereby completing the dynamic tracking of the target object.

[0152] In an embodiment of the present invention, a target tracking device is provided, which is suitable for tracking at least two targets. Each target includes a background part and a marking part with different colors, and the background part and the marking part each have a single color. The target part on each target is set to be one of the background part and the marking part, and the non-target part of each target is the other of the background part and the marking part. The color of the target part on each target is different, and the color of the non-target part of each target is the same. The target tracking device can execute the target tracking method described in any of the above embodiments.

[0153] like Figure 8 As shown, the target tracking device 100 includes a first determination module 10 , a second determination module 20 and a tracking module 30 .

[0154] The first determination module 10 is configured to determine the HSV image of the target object based on the RGB image of the target object. The process of determining the HSV image of the target object based on the RGB image of the target object can be referred to in the above embodiment, and will not be described in detail here.

[0155] The second determination module 20 is connected to the first determination module 10. The second determination module 20 is configured to determine the color of the target part of the target object based on the HSV image of the target object. The specific process of determining the color of the target part can refer to the above embodiment, and will not be repeated here.

[0156] The tracking module 30 is connected to the second determination module 20. The tracking module 30 is configured to determine the area of ​​the target part of the color determined as the target object. The specific process of determining as the target object can refer to the above embodiment, and will not be repeated here.

[0157] In one example, the tracking module 30 is further configured to determine a first number of targets in an initial image of the target and a second number of targets in a next frame of image, determine a relationship between the first number and the second number, and track the target according to the relationship. The specific process of tracking the target can be referred to the above embodiment, and will not be repeated here.

[0158] In an embodiment of the present invention, a readable storage medium is further provided, wherein the readable storage medium stores a program or an instruction, and when the program or the instruction is executed by a processor, the target object tracking method described in any of the above embodiments is implemented.

[0159] The "readable storage medium" of an embodiment of the present invention refers to any medium that participates in providing a program or instruction to a processor for execution. The medium can take a variety of forms, including but not limited to non-volatile media, volatile media and transmission media. Non-volatile media include, for example, optical disks or disks, such as storage devices. Volatile media include dynamic memory, such as main memory. Transmission media include coaxial cables, copper wires and optical fibers, including wires containing buses. Transmission media can also take the form of sound waves or light waves, such as sound waves or light waves generated during radio frequency (RF) and infrared (IR) data communications. Common forms of readable storage media include, for example, floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, DVDs, any other optical media, punch cards, paper tapes, any other physical media with hole patterns, RAMs, PROMs and EPROMs, FLASH-EPROMs, any other memory chips or boxes, carriers as described below, or any other media from which a computer can read.

[0160] In an embodiment of the present invention, a cloud server is also provided. The cloud server includes a memory and a processor. The processor may be a central processing unit (CPU). The memory stores programs or instructions, and when the programs or instructions are executed by the processor, the above-mentioned target tracking method is executed. In one example, the cloud server may be a virtual server formed by mapping a physical server through virtualization technology. Among them, there may be one or more physical servers. When there are multiple physical servers, the cloud server may be a virtual server formed by mapping a server cluster through virtualization technology. The virtual server may also be one or more. In one example, the cloud server can be provided to users through a cloud platform. In one example, the memory may be a readable storage medium.

[0161] In an embodiment of the present invention, an electronic device is also provided. The electronic device (not shown) includes a processor (not shown) and a memory (not shown). The memory stores a program, and when the program is executed by the processor, any of the target tracking methods in the above examples can be implemented.

[0162] In one example, the processor may be a microprocessor, such as a general-purpose processor such as a graphics processing unit (GPU), a central processing unit (CPU), a digital signal processor (DSP), etc. In one example, the processor may also be a microprocessor core implemented by a hardware circuit, such as a microprocessor core implemented in a hardware logic component by a reconfigurable logic, and the hardware logic component includes a field programmable gate array (FPGA), a complex programmable logic device (CPLD), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system on a chip (SOC), etc.

[0163] In one example, the processor may also be a virtual processor, which may be a virtual processor with the characteristics of an Intel x86 processor, or a virtual processor with the characteristics of a PowerPC processor. Preferably, the processor is a graphics processor. In one example, the processor may be a single-core processor or a multi-core processor.

[0164] In one example, the memory includes a volatile memory (i.e., a random access memory) and a non-volatile memory. The volatile memory includes a main memory, a cache, etc., and the non-volatile memory includes an auxiliary memory, etc. In one example, the memory can be set as a remote memory, and the remote memory can be connected to the processor via a network (wired network or wireless network). The network includes, but is not limited to, a wide area network, a local area network, a metropolitan area network, a personal area network, the Internet, a satellite communication network, and any combination thereof.

[0165] In one example, the processor executes a program based on the program obtained from the memory to create a corresponding task thread and execute the thread. In one example, the processor obtains a program from the external memory based on the read instruction in the memory to create a corresponding task thread and execute the thread. The above program is used to implement a control method for tracking a target object.

[0166] Although the subject matter described herein is provided in the general context of being executed in conjunction with the execution of an operating system and an application program on a computer system, it will be appreciated by those skilled in the art that it can also be implemented in conjunction with other types of program modules. Generally speaking, program modules include routines, programs, components, data structures, and other types of structures that perform specific tasks or implement specific abstract data types. It will be appreciated by those skilled in the art that the method steps described in conjunction with any one of the examples herein can all be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether to perform in hardware or software mode depends mainly on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0167] When the method steps are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Therefore, the technical solution of the present invention, or the part that contributes to the original technology, or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each example of the present invention.

[0168] The target tracking method, target tracking device, electronic device and readable storage medium according to the present invention have at least one of the following advantages:

[0169] (1) The target tracking method, target tracking device, electronic device, and readable storage medium of the present invention can simultaneously determine or identify targets that need to be distinguished by determining or identifying targets with target parts of different colors (such as calibration plates or tracked objects with calibration plates), thereby meeting the scenario of simultaneously identifying multiple calibration plates in the same image;

[0170] (2) The target tracking method, target tracking device, electronic device and readable storage medium of the present invention can meet the requirement for identifying multiple calibration plates by setting the number of different colors, eliminating the requirement for the maximum number of trackable objects;

[0171] (3) The target tracking method, target tracking device, electronic device and readable storage medium of the present invention can distinguish targets with similar colors to accurately identify or determine multiple targets;

[0172] (4) The target tracking method, target tracking device, electronic device and readable storage medium of the present invention can realize dynamic tracking of multiple targets;

[0173] (5) The target tracking method, target tracking device, electronic device and readable storage medium of the present invention are real-time and robust in the determination or identification process of multiple calibration plates.

[0174] Although some embodiments of the present general inventive concept have been shown and described, it will be appreciated by those skilled in the art that changes may be made to these embodiments without departing from the principles and spirit of the present general inventive concept, the scope of which is defined by the claims and their equivalents.

Claims

1. A target tracking method, comprising: Providing at least two targets, each target comprising a background portion and a marking portion with different colors, the background portion and the marking portion both having a single color, the target portion on each target being set to be one of the background portion and the marking portion, and the non-target portion on each target being the other of the background portion and the marking portion, the target portion of each target having a different color, and the non-target portion of each target having the same color; Determine the HSV image of the target object based on the RGB image of the target object; Determine the color of the target part of the target object based on the HSV image of the target object; The area of ​​the target portion whose color is determined is determined as the target object.

2. The target tracking method according to claim 1, wherein: Determining the HSV image of the target object based on the RGB image of the target object includes: determining the HSV data of the target object based on the RGB data of the target object, Determining the color of the target part of the target object based on the HSV image of the target object includes: Determine a first correspondence between the HSV data of the target object and the HSV threshold ranges of different color series in a preset HSV threshold table; The color of the target part of the target object is determined based on the first corresponding relationship.

3. The target tracking method according to claim 2, wherein: Determining the color of the target part of the target object based on the first corresponding relationship includes: Determine whether the HSV data of all targets are within the HSV thresholds of different color series: When the HSV data of all the target objects are within the HSV threshold range of different color series, the color corresponding to the HSV threshold range of the different color series is determined as the color of the target part of the target object; When the HSV data of all target objects are not within the HSV threshold range of different color series, the first grayscale data is determined based on the HSV data of at least two target objects in the same color series, and the respective colors are determined by comparing the first grayscale data with the second grayscale data determined based on the respective corresponding RGB data.

4. The target tracking method according to any one of claims 1 to 3, wherein: Determining the region of the target portion whose color is determined as the target object includes: Determine a binary image of each color based on the image of the target portion whose color is determined; The initial image of the target object is segmented using the binary image of each color, and the segmented area is determined as the target object.

5. The target tracking method according to claim 4, wherein: In the binary image, the value of the area with the determined color is 1, and the value of the rest of the area is 0. The binary image of each color is respectively calculated with the initial image of the target object, and the area with a true value is determined as the area of ​​the target part, and the area of ​​the target part is determined as the target object.

6. The target tracking method according to claim 5, wherein: The target object tracking method further includes: determining a first number of target objects in an initial image of the target objects; Determining a second number of the target object in a next frame image of the target object; A relationship between the first number and the second number is determined, and the target object is tracked based on the relationship.

7. The target tracking method according to claim 6, wherein: Tracking targets according to the relationship includes: When the second number is less than or equal to the first number, Determine the position data of each target object in the initial image and the position data of each target object in the next frame image; Determine the distance between the position data of the target object in the initial image and the position data of the target object in the next frame image, and use the position data with the closest distance as the pairing data; The position data of the target object in the next frame image is used to update the position data of the target object in the initial image that is paired with the position data of the target object in the next frame image, and the position data of the unpaired target object is deleted.

8. The target tracking method according to claim 7, wherein: The position data of the target object includes coordinate data of a center point of a target part of the target object in the image coordinate system.

9. The target tracking method according to claim 8, wherein: Tracking targets according to the relationship includes: When the second number is greater than the first number, Determine the color of the target part of the target object based on the next frame image, and determine the area where the target part is determined as the target object; Determine the position data of all targets and the number of targets in the next frame image.

10. The target tracking method according to any one of claims 1 to 3, wherein: The marked part of the target object includes regular patterns or irregular patterns. The regular graphics include at least one of a circle, an ellipse, a rectangle, a square, a polygon, a triangle, and ChArUco arranged in a regular pattern; The irregular graphics include logos arranged in a regular pattern.

11. A target tracking device, characterized in that: The target tracking device is suitable for tracking at least two targets, each target includes a background part and a marking part with different colors, the background part and the marking part each have a single color, the target part on each target is set to be one of the background part and the marking part, and the non-target part on each target is the other of the background part and the marking part, the color of the target part on each target is different, and the color of the non-target part of each target is the same, and the target tracking device includes: a first determination module, the first determination module being configured to determine an HSV image of the target object based on an RGB image of the target object; a second determination module connected to the first determination module, the second determination module being configured to determine a color of a target portion of the target object based on the HSV image of the target object; The tracking module is connected to the second determination module, and is configured to determine the area of ​​the target part with the determined color as the target object.

12. The target tracking device according to claim 11, characterized in that: The tracking module is also configured to determine a first number of targets in an initial image of the target and a second number of targets in a next frame of image, determine a relationship between the first number and the second number, and track the target according to the relationship.

13. An electronic device, characterized in that: The electronic device comprises: A memory and a processor, wherein the memory stores a program, wherein the processor implements the target tracking method according to any one of claims 1-10 when executing the program on the memory.

14. A readable storage medium, characterized in that: The readable storage medium stores a computer-readable program or instruction, and when the computer-readable program or instruction is executed by a processor, the target tracking method according to any one of claims 1-10 is implemented.

Images