Space cooperation target visual detection and identification method, system and medium

Abstract

The present disclosure provides a space cooperative target visual detection and recognition method, system and medium, wherein the space cooperative target visual detection and recognition method comprises: constructing a plurality of scale image pyramids and scale images thereof for an image containing a cooperative target; determining a correlation coefficient of a matching correlation degree between each scale image and a preset target matching template according to each scale image and the preset target matching template; determining a center coordinate of a cooperative target on the image containing the cooperative target according to the correlation coefficient; determining a cross ratio feature vector of a preset number of targets in the cooperative target on the image containing the cooperative target; and comparing the cross ratio feature vector of the preset number of targets with a cross ratio feature vector in a preset cooperative target library to determine an optimal matching cooperative target in the cooperative target library. The present disclosure realizes detection and recognition of a cooperative target in an environment with poor imaging conditions.

Description

Technical Field

[0001] This disclosure relates to the field of relative target recognition and attitude measurement technology, specifically to a method, system, and medium for visual detection and recognition of spatial cooperative targets. Background Technology

[0002] As humanity's exploration of space deepens, space missions are becoming increasingly diversified, including autonomous rendezvous and docking, clearing space debris and other space junk, and on-orbit maintenance and servicing. These missions require a certain level of operational and control capabilities over space targets. Space targets are categorized into cooperative and non-cooperative targets. Cooperative targets are those whose structural dimensions are fully known and for which targets have been pre-installed on the target spacecraft. Non-cooperative targets lack fixed structural features and whose dimensions and motion information are not (or are not fully) known. To ensure the reliable execution of space missions, important space missions such as rendezvous and docking often employ cooperative targets. Cooperative targets are pre-installed on the target spacecraft, target detection equipment on the spacecraft tracks the targets, and the relative attitude between the two spacecraft is obtained through autonomous target identification and matching.

[0003] Among them, optical cameras and lidar are currently the mainstream detection equipment. Optical cameras have become one of the mainstream active detection devices due to their high imaging accuracy, but their imaging effect largely depends on the optical environment conditions. Spacecraft are usually covered with thermal insulation materials, which have strong light reflection capabilities, affecting the imaging of optical cameras. Therefore, how to accurately detect and identify targets under poor imaging conditions is one of the key challenges that needs to be overcome to ensure the successful completion of space missions. Summary of the Invention

[0004] In view of the deficiencies in the prior art, the purpose of this disclosure is to provide a method, system and medium for visual detection and recognition of spatial cooperative targets.

[0005] To achieve the above objectives, according to one aspect of this disclosure, a method for visual detection and recognition of spatial cooperative targets is provided, comprising:

[0006] Based on the preset cooperative target distance information and the preset cooperative target size information, construct image pyramids of multiple scales for images containing cooperative targets, and determine the scale image corresponding to each scale image pyramid;

[0007] Based on the scale image corresponding to each scale of the image pyramid and the preset target matching template, determine the correlation coefficient of the matching correlation between the scale image corresponding to each scale of the image pyramid and the preset target matching template.

[0008] Based on the correlation coefficient of the matching correlation between the scale image corresponding to each scale image pyramid and the preset target matching template, the center point coordinates of the cooperative target on the image containing the cooperative target are determined.

[0009] A predetermined number of targets are randomly selected from the cooperative targets on the image containing the cooperative targets, and the cross-ratio feature vectors of the predetermined number of targets are determined;

[0010] The cross-ratio feature vectors of the preset number of targets are compared with the cross-ratio feature vectors in the preset cooperative target library to determine the optimal matching cooperative target in the cooperative target library.

[0011] Optionally, the step of constructing image pyramids of multiple scales for images containing cooperative targets based on preset cooperative target distance information and preset cooperative target size information, and determining the scale image corresponding to each scale image pyramid, includes:

[0012] Based on the preset cooperative target distance information and the preset cooperative target size information, the upper and lower limits of the image pyramids at multiple scales are determined;

[0013] Based on the upper and lower limits of the image pyramids at multiple scales, a Gaussian kernel is used to perform convolutional sampling on the image containing the cooperative target to construct image pyramids at multiple scales, and the scale image corresponding to each scale image pyramid is determined.

[0014] Optionally, determining the correlation coefficient between the scale image corresponding to each scale of the image pyramid and the preset target matching template based on the scale image corresponding to each scale of the image pyramid and the preset target matching template includes:

[0015] Using each pixel in the scale image corresponding to each scale of the image pyramid as a reference, a sub-image of the scale image corresponding to each scale of the image pyramid is determined, and the sub-image has the same size as the preset target matching template;

[0016] The sub-image of the scale image corresponding to each scale of the image pyramid and the preset target matching template are subjected to normalized product processing to determine the correlation coefficient of the matching correlation between the scale image corresponding to each scale of the image pyramid and the preset target matching template.

[0017] Optionally, determining the center point coordinates of the cooperative target on the image containing the cooperative target based on the correlation coefficient of the matching correlation between the scale image corresponding to each scale image pyramid and the preset target matching template includes:

[0018] Points whose correlation coefficient between the scale image corresponding to each scale of the image pyramid and the preset target matching template is greater than the preset correlation threshold are subjected to four-connected component determination processing.

[0019] The correlation coefficients of the matching correlations that are greater than the preset correlation threshold after the four-connected component determination process are subjected to correlation weighting calculation to determine the center point coordinates of the cooperative target on the image containing the cooperative target.

[0020] Optionally, determining the cross-ratio feature vectors of the predetermined number of targets selected from the cooperative targets on the image containing the cooperative targets includes:

[0021] Each of the predetermined number of targets is used as the center point in turn, and each target is connected to the remaining target to form a straight line, generating a predetermined number of common point lines;

[0022] Determine the intersection ratio of each set of concurrent lines based on the slope of each straight line in each set of concurrent lines.

[0023] Based on the cross ratio of the common lines of the preset number of groups, the cross ratio feature vector corresponding to the preset number of targets is determined.

[0024] Optionally, determining the intersection ratio of each set of concurrent lines based on the slope of each straight line in each set of concurrent lines includes:

[0025]

[0026] Among them, c A k represents the intersection ratio of concurrent lines centered at target A. AB k represents the slope of the straight line passing through targets A and B. AC k represents the slope of the straight line passing through targets A and C. AD k represents the slope of the straight line passing through targets A and D. AE This represents the slope of the straight line passing through targets A and E.

[0027] Optionally, the method for determining the preset target matching template includes:

[0028] Based on the grayscale characteristics of the preset cooperative target, determine a grayscale template with a size of N×M;

[0029] The grayscale template with a size of N×M is normalized to confirm the preset target matching template.

[0030] Optionally, the method for determining the preset cooperative target library includes:

[0031] Based on the preset number and location information of cooperative targets, determine the cross ratio feature vector corresponding to each cooperative target;

[0032] The cross-ratio feature vector corresponding to each cooperative target is stored and processed to determine the preset cooperative target library.

[0033] According to a second aspect of this disclosure, a spatial cooperative target visual detection and recognition system is provided, comprising:

[0034] The image pyramid construction module is used to construct image pyramids of multiple scales for images containing cooperative targets based on preset cooperative target distance information and preset cooperative target size information, and to determine the scale image corresponding to each scale image pyramid.

[0035] The correlation coefficient determination module is used to determine the correlation coefficient between the scale image corresponding to each scale of the image pyramid and the preset target matching template based on the scale image corresponding to each scale of the image pyramid and the preset target matching template.

[0036] The target determination module is used to determine the center coordinates of the cooperative target on the image containing the cooperative target based on the correlation coefficient between the scale image corresponding to each scale image pyramid and the preset target matching template.

[0037] The cross-ratio feature vector determination module is used to select a preset number of targets from the cooperative targets on the image containing cooperative targets, and determine the cross-ratio feature vectors of the preset number of targets;

[0038] The matching module is used to compare the cross-ratio feature vectors of the preset number of targets with the cross-ratio feature vectors in the preset cooperative target library to determine the optimal matching cooperative target in the cooperative target library.

[0039] According to a third aspect of this disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, characterized in that, when executed by a processor, the program implements the steps of the method provided in the first aspect of this disclosure.

[0040] Compared with the prior art, the embodiments disclosed herein have at least one of the following beneficial effects:

[0041] The above technical solution constructs image pyramids at multiple scales for images containing cooperative targets and determines the scale image corresponding to each scale image pyramid. Then, combined with a preset target matching template, the correlation coefficient between each scale image and the preset target matching template is determined. Points with correlation coefficients greater than a preset correlation threshold undergo correlation weighting to determine the center coordinates of the cooperative targets in the images containing cooperative targets. The cross-ratio invariance of concurrent lines is used to determine the cross-ratio feature vectors of a preset number of targets randomly selected from the cooperative targets in the images containing cooperative targets. Finally, these cross-ratio feature vectors are compared with the cross-ratio feature vectors in a preset cooperative target library to determine the optimal matching cooperative target in the cooperative target library. This enables the identification of cooperative targets in environments with poor imaging conditions and reduces the impact of illumination on the identification and extraction of cooperative targets. Attached Figure Description

[0042] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0043] Figure 1 This is a flowchart illustrating a spatial cooperative target visual detection and recognition method according to an exemplary embodiment.

[0044] Figure 2 This is a schematic diagram illustrating the grayscale feature distribution of a cooperative target according to an exemplary embodiment.

[0045] Figure 3 This is a grayscale distribution map of a preset target matching template according to an exemplary embodiment.

[0046] Figure 4 This is a correlation coefficient distribution diagram illustrating matching relevance according to an exemplary embodiment.

[0047] Figure 5 This is a schematic diagram illustrating the construction of a set of concurrent lines according to an exemplary embodiment.

[0048] Figure 6 This is a schematic diagram illustrating the construction of five sets of concurrent lines according to an exemplary embodiment.

[0049] Figure 7 This is a block diagram illustrating a spatial cooperative target visual detection and recognition system according to an exemplary embodiment. Detailed Implementation

[0050] The present disclosure will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present disclosure, but do not limit the present disclosure in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present disclosure. These all fall within the protection scope of the present disclosure.

[0051] Figure 1 This is a flowchart illustrating a spatial cooperative target visual detection and recognition method according to an exemplary embodiment. Figure 1 As shown, a spatial cooperative target visual detection and recognition method includes S11 to S15.

[0052] S11. Based on the preset cooperative target distance information and the preset cooperative target size information, construct image pyramids of multiple scales for the image containing the cooperative target, and determine the scale image corresponding to each scale image pyramid.

[0053] The preset cooperative target distance information refers to the estimated distance between the tracking target and the cooperative target; the preset cooperative target size information refers to the estimated size of the cooperative target mounted on the cooperative target. The image containing the cooperative target is an image of the cooperative target taken using an optical camera.

[0054] Figure 2 This is a schematic diagram of the grayscale feature distribution of a cooperative target according to an exemplary embodiment.

[0055] In this disclosure, the cooperative objective includes several circular targets, and the grayscale feature distribution of each circular target is as follows: Figure 2 As shown.

[0056] In one possible embodiment, the Gaussian pyramid algorithm can be used to construct image pyramids at multiple scales for images containing cooperative targets. The Gaussian pyramid algorithm is used to preprocess the images containing cooperative targets to construct image pyramids at multiple scales and the scale images corresponding to each scale image pyramid.

[0057] S12, based on the scale image corresponding to each scale of the image pyramid and the preset target matching template, determine the correlation coefficient of the matching correlation between the scale image corresponding to each scale of the image pyramid and the preset target matching template.

[0058] Figure 3 This is a grayscale distribution image of a preset target matching template according to an exemplary embodiment. In this disclosure, the grayscale distribution of the determined preset target matching template is as follows: Figure 3 As shown.

[0059] In some possible embodiments, the method for determining a preset target matching template includes:

[0060] Based on the grayscale characteristics of the preset cooperative target, determine a grayscale template with a size of N×M;

[0061] Normalize the grayscale template with size N×M and confirm the preset target matching template.

[0062] Figure 4 This is a correlation coefficient distribution diagram illustrating matching relevance according to an exemplary embodiment.

[0063] For example, Figure 4 The value in the middle represents the correlation coefficient between the region with the horizontal coordinate of 300-306 and the vertical coordinate of 171-183 in the scale image and the preset target matching template.

[0064] In some possible embodiments, S12 may include S21 to S22.

[0065] S21, using each pixel in the scale image corresponding to each scale image pyramid as a reference, determine the sub-image of the scale image corresponding to each scale image pyramid, and the sub-image has the same size as the preset target matching template.

[0066] Each pixel corresponds to a sub-image of the same size as the preset target matching template.

[0067] S22, perform normalized product processing on the sub-image of the scale image corresponding to each scale image pyramid and the preset target matching template to determine the matching correlation between the scale image corresponding to each scale image pyramid and the preset target matching template.

[0068] The normalized product processing can be performed using the NCC (Normalized Cross Correlation) algorithm metric, which is as follows:

[0069]

[0070] Where (i,j) represents the pixel coordinates of the scaled image, R(i,j) represents the correlation coefficient of the matching relevance corresponding to pixel (i,j), M represents the length of the preset target matching template, N represents the width of the preset target matching template, and S i,j (s, t) represents the subgraph corresponding to pixel (i, j), S i,j (s,t) represents the gray value of the subimage at coordinates (s,t), E(S i,j ) represents subgraph S i,jThe average gray value, T(s,t) represents the gray value of the preset target matching template at coordinate (s,t), and E(T) represents the average gray value of the preset target matching template.

[0071] The preset target matching template has a size of M×N, S i,j (s,t) indicates that the size of the sub-image corresponding to pixel (i,j) is also M×N.

[0072] S13, based on the correlation coefficient of the matching correlation between the scale image corresponding to each scale image pyramid and the preset target matching template, determine the center coordinates of the cooperative target on the image containing the cooperative target.

[0073] Specifically, a correlation threshold can be preset to determine the correlation coefficient between the scale image corresponding to each scale of the image pyramid and the preset target matching template. The center coordinates of the cooperative target on the image containing the cooperative target are determined based on the points whose correlation coefficient is greater than the preset correlation threshold.

[0074] S14, Select a preset number of targets from the cooperative targets on the image containing the cooperative targets, and determine the cross ratio feature vector of the preset number of targets.

[0075] Specifically, a common line can be constructed for any preset number of targets, and the cross ratio value can be determined by the slope of the common line. Then, the cross ratio values ​​with each target as the center point are combined to determine the cross ratio feature vector of the preset number of targets.

[0076] S15, compare the cross-ratio feature vectors of a preset number of targets with the cross-ratio feature vectors in a preset cooperative target library to determine the optimal matching cooperative target in the cooperative target library.

[0077] Based on the preset number and location information of cooperative targets, a preset cooperative target library is constructed. This library stores all possible cross-ratio feature vectors corresponding to known cooperative targets.

[0078] The above technical solution constructs image pyramids at multiple scales for images containing cooperative targets and determines the scale image corresponding to each scale image pyramid. Then, combined with a preset target matching template, the matching correlation between each scale image and the preset target matching template is determined. Points with matching correlation greater than a preset correlation threshold are subjected to correlation weighting calculation to determine the center coordinates of cooperative targets on the images containing cooperative targets. The cross-ratio invariance of concurrent lines is used to determine the cross-ratio feature vectors of a preset number of targets randomly selected from the cooperative targets on the images containing cooperative targets. Finally, the cross-ratio feature vectors are compared with the cross-ratio feature vectors in the preset cooperative target library to determine the optimal matching cooperative target in the cooperative target library. This enables the recognition of cooperative targets in poor imaging environments and reduces the impact of illumination on the recognition and extraction of cooperative targets.

[0079] In some possible embodiments, based on preset cooperative target distance information and preset cooperative target size information, an image pyramid of multiple scales is constructed for the image containing the cooperative target, and the scale image corresponding to each scale image pyramid is determined, which may include S31 to S32.

[0080] S31, based on the preset cooperative target distance information and the preset cooperative target size information, determine the upper and lower limits of the image pyramid at multiple scales.

[0081] S32, based on the upper and lower limits of the image pyramids at multiple scales, a Gaussian kernel is used to perform convolutional sampling on the image containing the cooperative target to construct image pyramids at multiple scales, and the scale image corresponding to each scale image pyramid is determined.

[0082] The Gaussian kernel can be described as follows:

[0083]

[0084] Where G(x,y) represents the value of the kernel function, σ represents the standard deviation, and (x0,y0) represents the coordinates of the center point of the two-dimensional kernel function.

[0085] The value of the standard deviation affects the smoothing effect of image processing.

[0086] When using a Gaussian kernel to perform convolutional sampling on an image containing cooperative targets:

[0087] As an example, if it is necessary to downsample an image containing cooperative targets, the image of the current layer is sampled by Gaussian kernel convolution and then downsampled according to the size of the image containing cooperative targets.

[0088] As another example, if it is necessary to upsample an image containing cooperative targets, the image of the current layer is sampled by Gaussian kernel convolution and then interpolated according to the size of the image containing cooperative targets.

[0089] In some possible embodiments, the center point coordinates of the cooperative target on the image containing the cooperative target are determined based on the matching correlation between the scale image corresponding to each scale image pyramid and the preset target matching template, including S41 to S42.

[0090] S41, perform four-connected component determination on the points where the correlation coefficient between the scale image corresponding to each scale image pyramid and the preset target matching template is greater than the preset correlation threshold.

[0091] The correlation coefficient of the matching relevance can range from -1 to 1. When the correlation coefficient is between 0.8 and 1.0, it indicates that the matching relevance between the scale image corresponding to each scale of the image pyramid and the preset target matching template is extremely strong; when the correlation coefficient is between 0.6 and 0.8, it indicates that the matching relevance between the scale image corresponding to each scale of the image pyramid and the preset target matching template is relatively strong.

[0092] In this disclosure, a preset correlation threshold can be set according to the imaging effect of the optical camera and the correlation requirements with the cooperative target. The preset correlation threshold can be in the range of [0.6, 0.9].

[0093] In one possible embodiment, the four-connected domain determination process involves determining the neighboring pixels in the four directions (up, down, left, and right) of a pixel whose correlation coefficient is greater than a preset correlation threshold.

[0094] S42, perform correlation weighting on the correlation coefficients of the matching correlations that are greater than the preset correlation threshold after the four-connected domain judgment process, and determine the center point coordinates of the cooperative target on the image containing the cooperative target.

[0095] This includes the coordinates of the center point of the cooperative target on the image containing the cooperative target, including:

[0096]

[0097] Where R(x,y) represents the correlation coefficient between the coordinate point (x,y) and the preset target matching template, x0 represents the abscissa of the center point of the cooperative target, and y0 represents the ordinate of the center point of the cooperative target.

[0098] As an example, the preset relevance threshold can be set to 0.75, such as... Figure 4 As shown, the coordinates of the pixels with a relevance greater than the preset relevance threshold are (303, 177), (304, 177), and (303, 178). After weighting the relevance coefficients of their corresponding matching relevance scores, the coordinates of the center point of the cooperative target in the image containing the cooperative target are:

[0099]

[0100] In some possible embodiments, a preset number of targets are randomly selected from the cooperative targets on the image containing the cooperative targets, and the cross-ratio feature vectors of the preset number of targets are determined, including S51 to S53.

[0101] S51, take each of the preset number of targets as the center point in turn, and connect it with each of the remaining targets to form a straight line, generating a preset number of common point lines.

[0102] The preset quantity can be 5, and the preset number of groups is 5, meaning the preset quantity equals the preset number of groups.

[0103] Figure 5 This is a schematic diagram illustrating the construction of a set of concurrent lines according to an exemplary embodiment.

[0104] like Figure 5 As shown, as an example, five targets are randomly selected from the cooperative targets detected on an image containing cooperative targets, namely target A, target B, target C, target D, and target E, with target A as the center point:

[0105] Using target A as the center point, connect it to targets B, C, D, and E respectively to form straight lines, namely lines AB, AC, AD, and AE, which together form a set of lines with common points.

[0106] S52, determine the cross ratio of each set of concurrent lines based on the slope of each straight line in each set of concurrent lines.

[0107] Following the example above, the intersection ratio of the concurrent lines centered on target A is:

[0108]

[0109] Among them, c A k represents the intersection ratio of concurrent lines centered at target A. AB k represents the slope of the straight line passing through targets A and B. AC k represents the slope of the straight line passing through targets A and C. AD k represents the slope of the straight line passing through targets A and D. AE This represents the slope of the straight line passing through targets A and E.

[0110] The intersection ratios of the lines centered on targets A, B, C, D, and E are respectively c. A c B c C c D c E .

[0111] S53, determine the cross ratio feature vector corresponding to a preset number of targets based on the cross ratio value of the concurrent lines of the preset number of groups.

[0112] Figure 6 This is a schematic diagram illustrating the construction of five sets of concurrent lines according to an exemplary embodiment.

[0113] like Figure 6 As shown, targets A, B, C, D, and E are used as center points in turn, and are connected to the remaining targets to form straight lines, forming 5 sets of lines with the same point.

[0114] Following the example above, determine the intersection ratio values ​​of the five sets of concurrent lines, and based on these values, determine the intersection ratio eigenvectors corresponding to targets A, B, C, D, and E as C = [c A c B c C c D c E ].

[0115] In some possible embodiments, the method for determining a preset cooperative target library includes S61 to S62.

[0116] S61, based on the preset number and location information of cooperative targets, determine the cross ratio feature vector corresponding to each cooperative target.

[0117] S62, store and process the cross-ratio feature vector corresponding to each cooperative target to determine the preset cooperative target library.

[0118] In one possible embodiment, based on the number and location information of cooperative targets on the known cooperative target, and according to the method of determining the cross-ratio feature vector of an arbitrarily preset number of targets in S51 to S53 of this disclosure, all possible combinations of arbitrarily preset number of cooperative targets on the known cooperative target are obtained, and the cross-ratio feature vector corresponding to each target combination is determined as the cross-ratio feature vector corresponding to each cooperative target. The cross-ratio feature vector corresponding to each cooperative target is stored in the cooperative target library to generate a preset cooperative target library.

[0119] The above technical solution enables the identification of cooperative targets in environments with poor imaging capabilities, reducing the impact of lighting on the identification and extraction of cooperative targets.

[0120] Figure 7 This is a block diagram illustrating a spatial cooperative target visual detection and recognition system according to an exemplary embodiment.

[0121] Based on the same concept, this disclosure also provides a spatial cooperative target visual detection and recognition system, referring to... Figure 7 The spatial cooperative target visual detection and recognition system 100 includes: an image pyramid construction module 110, a correlation coefficient determination module 120, a target determination module 130, a cross-ratio feature vector determination module 140, and a matching module 150.

[0122] The image pyramid construction module 110 is used to construct image pyramids of multiple scales for an image containing cooperative targets based on preset cooperative target distance information and preset cooperative target size information, and to determine the scale image corresponding to each scale image pyramid.

[0123] The correlation coefficient determination module 120 is used to determine the correlation coefficient between the scale image corresponding to each scale image pyramid and the preset target matching template based on the scale image corresponding to each scale image pyramid and the preset target matching template.

[0124] The target determination module 130 is used to determine the center coordinates of the cooperative target on the image containing the cooperative target based on the correlation coefficient between the scale image corresponding to each scale image pyramid and the preset target matching template.

[0125] The cross-ratio feature vector determination module 140 is used to select a preset number of targets from the cooperative targets on the image containing cooperative targets, and determine the cross-ratio feature vector of the preset number of targets;

[0126] The matching module 150 is used to compare the cross-ratio feature vectors of the preset number of targets with the cross-ratio feature vectors in the preset cooperative target library to determine the optimal matching cooperative target in the cooperative target library.

[0127] The above technical solution constructs image pyramids at multiple scales for images containing cooperative targets and determines the scale image corresponding to each scale image pyramid. Then, combined with a preset target matching template, the matching correlation between each scale image and the preset target matching template is determined. Points with matching correlation greater than a preset correlation threshold are subjected to correlation weighting calculation to determine the center coordinates of cooperative targets on the images containing cooperative targets. The cross-ratio invariance of concurrent lines is used to determine the cross-ratio feature vectors of a preset number of targets randomly selected from the cooperative targets on the images containing cooperative targets. Finally, the cross-ratio feature vectors are compared with the cross-ratio feature vectors in the preset cooperative target library to determine the optimal matching cooperative target in the cooperative target library. This enables the recognition of cooperative targets in poor imaging environments and reduces the impact of illumination on the recognition and extraction of cooperative targets.

[0128] Optionally, the image pyramid building module 110 includes:

[0129] The first determining submodule is used to determine the upper and lower limits of the image pyramid at multiple scales based on the preset cooperative target distance information and the preset cooperative target size information.

[0130] The second determining submodule is used to perform convolution sampling processing on the image containing the cooperative target using a Gaussian kernel based on the upper and lower limits of the image pyramids at multiple scales, to construct image pyramids at multiple scales, and to determine the scale image corresponding to each scale image pyramid.

[0131] Optionally, the correlation coefficient determination module 120 includes:

[0132] The third determining submodule is used to determine a sub-image of the scale image corresponding to each scale image pyramid, using each pixel in the scale image corresponding to each scale image pyramid as a reference, wherein the sub-image has the same size as the preset target matching template.

[0133] The fourth determination submodule is used to perform normalized product processing on the sub-image of the scale image corresponding to each scale image pyramid and the preset target matching template to determine the correlation coefficient of the matching correlation between the scale image corresponding to each scale image pyramid and the preset target matching template.

[0134] Optionally, the target determination module 130 includes:

[0135] The determination submodule is used to perform four-connected component determination processing on points where the correlation coefficient between the scale image corresponding to each scale image pyramid and the preset target matching template is greater than a preset correlation threshold.

[0136] The fifth determining submodule is used to perform correlation weighting operations on the correlation coefficients of the matching correlations that are greater than the preset correlation threshold after the four-connected component determination process, and to determine the center point coordinates of the cooperative target on the image containing the cooperative target.

[0137] Optionally, the cross-ratio eigenvector determination module 140 includes:

[0138] The concurrent line generation submodule is used to take each of the preset number of targets as the center point in turn and connect it with each of the remaining targets to form a straight line, thereby generating a preset number of concurrent lines.

[0139] The sixth determination submodule is used to determine the intersection ratio of each set of concurrent lines based on the slope of each straight line in each set of concurrent lines.

[0140] The seventh determination submodule is used to determine the cross ratio feature vector corresponding to the preset number of targets based on the cross ratio value of the common lines of the preset number of groups.

[0141] Optionally, the sixth determination submodule includes:

[0142]

[0143] Among them, c A k represents the intersection ratio of concurrent lines centered at target A. AB k represents the slope of the straight line passing through targets A and B. AC k represents the slope of the straight line passing through targets A and C. AD k represents the slope of the straight line passing through targets A and D. AE This represents the slope of the straight line passing through targets A and E.

[0144] Optionally, the method for determining the preset target matching template includes:

[0145] Based on the grayscale characteristics of the preset cooperative target, determine a grayscale template with a size of N×M;

[0146] The grayscale template with a size of N×M is normalized to confirm the preset target matching template.

[0147] Optionally, the method for determining the preset cooperative target library includes:

[0148] Based on the preset number and location information of cooperative targets, determine the cross ratio feature vector corresponding to each cooperative target;

[0149] The cross-ratio feature vector corresponding to each cooperative target is stored and processed to determine the preset cooperative target library.

[0150] Regarding the embodiments of the above system, the specific ways in which each module performs operations have been described in detail in the embodiments of the method, and will not be elaborated here.

[0151] In this embodiment of the invention, a non-transitory computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of a spatial cooperative target visual detection and recognition method in any of the above embodiments.

[0152] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0153] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0154] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0155] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0156] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0157] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for visual detection and recognition of spatial cooperative targets, characterized in that, include: Based on the preset cooperative target distance information and the preset cooperative target size information, construct image pyramids of multiple scales for images containing cooperative targets, and determine the scale image corresponding to each scale image pyramid; Based on the scale image corresponding to each scale of the image pyramid and the preset target matching template, determine the correlation coefficient of the matching correlation between the scale image corresponding to each scale of the image pyramid and the preset target matching template. Based on the correlation coefficient of the matching correlation between the scale image corresponding to each scale image pyramid and the preset target matching template, the center point coordinates of the cooperative target on the image containing the cooperative target are determined. A predetermined number of targets are randomly selected from the cooperative targets on the image containing the cooperative targets, and the cross-ratio feature vectors of the predetermined number of targets are determined; The cross-ratio feature vectors of the preset number of targets are compared with the cross-ratio feature vectors in the preset cooperative target library to determine the optimal matching cooperative target in the cooperative target library; The step of randomly selecting a preset number of targets from the cooperative targets on the image containing the cooperative targets, and determining the cross-ratio feature vectors of the preset number of targets, includes: Each of the predetermined number of targets is used as the center point in turn, and each target is connected to the remaining target to form a straight line, generating a predetermined number of common point lines; Determine the intersection ratio of each set of concurrent lines based on the slope of each straight line in each set of concurrent lines. Based on the cross ratio value of the concurrent lines of the preset number of groups, determine the cross ratio feature vector corresponding to the preset number of targets; The step of determining the intersection ratio of each set of concurrent lines based on the slope of each straight line in each set includes: ； in, This represents the intersection ratio of the lines that converge to the target A. This represents the slope of the straight line passing through targets A and B. This represents the slope of the straight line passing through targets A and C. This represents the slope of the straight line passing through targets A and D. This represents the slope of the straight line passing through targets A and E.

2. The method according to claim 1, characterized in that, The step of constructing image pyramids of multiple scales for images containing cooperative targets based on preset cooperative target distance information and preset cooperative target size information, and determining the scale image corresponding to each scale image pyramid, includes: Based on the preset cooperative target distance information and the preset cooperative target size information, the upper and lower limits of the image pyramids at multiple scales are determined; Based on the upper and lower limits of the image pyramids at multiple scales, a Gaussian kernel is used to perform convolutional sampling on the image containing the cooperative target to construct image pyramids at multiple scales, and the scale image corresponding to each scale image pyramid is determined.

3. The method according to claim 1, characterized in that, The step of determining the correlation coefficient between the scale image corresponding to each scale of the image pyramid and the preset target matching template, based on the scale image corresponding to each scale of the image pyramid and the preset target matching template, includes: Using each pixel in the scale image corresponding to each scale of the image pyramid as a reference, a sub-image of the scale image corresponding to each scale of the image pyramid is determined, and the sub-image has the same size as the preset target matching template; The sub-image of the scale image corresponding to each scale of the image pyramid and the preset target matching template are subjected to normalized product processing to determine the correlation coefficient of the matching correlation between the scale image corresponding to each scale of the image pyramid and the preset target matching template.

4. The method according to claim 1, characterized in that, The step of determining the center point coordinates of the cooperative target on the image containing the cooperative target based on the correlation coefficient between the scale image corresponding to each scale image pyramid and the preset target matching template includes: Points whose correlation coefficient between the scale image corresponding to each scale of the image pyramid and the preset target matching template is greater than the preset correlation threshold are subjected to four-connected component determination processing. The correlation coefficients of the matching correlations that are greater than the preset correlation threshold after the four-connected component determination process are subjected to correlation weighting calculation to determine the center point coordinates of the cooperative target on the image containing the cooperative target.

5. The method according to claim 1, characterized in that, The method for determining the preset target matching template includes: Based on the grayscale characteristics of the preset cooperative target, determine a grayscale template with a size of N×M; The grayscale template with a size of N×M is normalized to confirm the preset target matching template.

6. The method according to claim 1, characterized in that, The method for determining the preset target library of cooperation includes: Based on the preset number and location information of cooperative targets, determine the cross ratio feature vector corresponding to each cooperative target; The cross-ratio feature vector corresponding to each cooperative target is stored and processed to determine the preset cooperative target library.

7. A spatial cooperative target visual detection and recognition system, characterized in that, include: The image pyramid construction module is used to construct image pyramids of multiple scales for images containing cooperative targets based on preset cooperative target distance information and preset cooperative target size information, and to determine the scale image corresponding to each scale image pyramid. The correlation coefficient determination module is used to determine the correlation coefficient between the scale image corresponding to each scale of the image pyramid and the preset target matching template based on the scale image corresponding to each scale of the image pyramid and the preset target matching template. The target determination module is used to determine the center coordinates of the cooperative target on the image containing the cooperative target based on the correlation coefficient between the scale image corresponding to each scale image pyramid and the preset target matching template. The cross-ratio feature vector determination module is used to select a preset number of targets from the cooperative targets on the image containing cooperative targets, and determine the cross-ratio feature vectors of the preset number of targets; The matching module is used to compare the cross-ratio feature vectors of the preset number of targets with the cross-ratio feature vectors in the preset cooperative target library to determine the optimal matching cooperative target in the cooperative target library; The cross-ratio feature vector determination module includes: The concurrent line generation submodule is used to take each of the preset number of targets as the center point in turn and connect it with each of the remaining targets to form a straight line, thereby generating a preset number of concurrent lines. The sixth determination submodule is used to determine the intersection ratio of each set of concurrent lines based on the slope of each straight line in each set of concurrent lines. The seventh determining submodule is used to determine the cross ratio feature vector corresponding to the preset number of targets based on the cross ratio value of the concurrent lines of the preset number of groups; The sixth determining submodule includes: ； in, This represents the intersection ratio of the lines that converge to the target A. This represents the slope of the straight line passing through targets A and B. This represents the slope of the straight line passing through targets A and C. This represents the slope of the straight line passing through targets A and D. This represents the slope of the straight line passing through targets A and E.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-6.

Images