Real-time positioning and classification method and device based on complementary single pixel detection, electronic equipment and storage medium

Abstract

The invention relates to a real-time positioning and classification method and device based on complementary single-pixel detection, electronic equipment and a storage medium, and the method comprises the steps: controlling a DMD of a complementary single-pixel detection device to simulate four target matrixes in sequence, and when the DMD simulates any target matrix, carrying out the real-time positioning and classification of the DMD; respectively determining a first light intensity signal of a first single-pixel detector and a second light intensity signal of a second single-pixel detector of the complementary single-pixel detection device; determining a plurality of target geometric moments of the target object according to a first light intensity signal of the first single-pixel detector when the DMD simulates each target matrix and a second light intensity signal of the second single-pixel detector when the DMD simulates each target matrix; and according to the plurality of target geometric moments, determining a centroid coordinate and roundness of the target object, and further determining a real-time positioning result and a classification result of the target object. According to the embodiment of the invention, complex data processing is not needed, and the real-time performance of real-time positioning and classification, the positioning precision and the classification accuracy can be improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of target positioning and classification, and in particular to a real-time positioning and classification method, device, electronic device and storage medium based on complementary single-pixel detection. Background Art

[0002] With the rapid development of optoelectronic imaging technology and computer vision technology, the technology of real-time positioning and classification of fast-moving objects has been widely used in many fields. However, common positioning and classification methods based on traditional imaging technology usually require a large amount of data throughput, resulting in complex target positioning and classification processes and poor real-time performance. On the other hand, in special application scenarios such as high-speed movement of target objects or limited imaging of non-visible light wavelengths, the detection precision and accuracy of traditional imaging technology will be affected, thereby affecting the accuracy of real-time positioning and classification of target objects. Summary of the invention

[0003] In view of this, the present disclosure proposes a technical solution of a real-time positioning and classification method, device, electronic device and storage medium based on complementary single-pixel detection.

[0004] According to one aspect of the present disclosure, a real-time positioning and classification method based on complementary single-pixel detection is provided, comprising: controlling a digital micromirror device (DMD) of a complementary single-pixel detection device to simulate four target matrices in sequence, and respectively determining a first light intensity signal of a first single-pixel detector and a second light intensity signal of a second single-pixel detector of the complementary single-pixel detection device when the DMD simulates any one of the target matrices, wherein the complementary single-pixel detection device is used to perform single-pixel imaging on a target object, any one of the target matrices is used to indicate a deflection state of each micromirror in the DMD, and the installation positions of the first single-pixel detector and the second single-pixel detector satisfy optical path complementarity; determining a plurality of target geometric moments corresponding to the target object according to a first light intensity signal of the first single-pixel detector when the DMD simulates each of the target matrices, and a second light intensity signal of the second single-pixel detector when the DMD simulates each of the target matrices, wherein any one of the target geometric moments corresponding to the target object is used to describe the position and shape characteristics of the target object; determining the centroid coordinates and roundness corresponding to the target object according to the plurality of target geometric moments corresponding to the target object; and determining a real-time positioning result and classification result corresponding to the target object according to the centroid coordinates and roundness corresponding to the target object.

[0005] In one possible implementation, the digital micromirror device DMD that controls the complementary single-pixel detection apparatus sequentially simulates four target matrices, including: for any one of the target matrices, using an error diffusion algorithm to control the DMD to load a binary modulation mask corresponding to the target matrix, wherein any one of the binary modulation masks represents a binary template formed by all micromirrors in the DMD.

[0006] In a possible implementation, the error diffusion algorithm is a Floyd-Stuhberg dithering algorithm.

[0007] In a possible implementation, the four target matrices include:

[0008] First target matrix Wherein, M represents the number of micromirror columns corresponding to the DMD; N represents the number of micromirror rows corresponding to the DMD;

[0009] Second target matrix

[0010] The third target matrix

[0011] Fourth target matrix

[0012] In a possible implementation, the multiple target geometric moments include: 0th order geometric moment m 0,0 (S), 1st order geometric moment m 1,0 (S), 1st order geometric moment m 0,1 (S), second-order geometric moment m 2,0 (S) and the second-order geometric moment m 0,2 (S), where S represents the target object;

[0013] The method comprises: determining the 0th order geometric moment m according to the first light intensity signal of the first single-pixel detector when the DMD simulates each target matrix, and the second light intensity signal of the second single-pixel detector when the DMD simulates each target matrix, and the second light intensity signal of the second single-pixel detector when the DMD simulates the target matrix. 0,0 (S); According to the first single pixel detector, the first target matrix M is simulated in the DMD (1) 10 The first light intensity signal at the time of determination of the first-order geometric moment m 1,0 (S); According to the first single pixel detector, the second target matrix M is simulated in the DMD (1) 01The first light intensity signal at the time of determination of the first-order geometric moment m 0,1 (S); According to the first single pixel detector, the third target matrix M is simulated in the DMD (1) 20 The first light intensity signal at the time of determination of the second-order geometric moment m 2,0 (S); According to the first single pixel detector, the fourth target matrix M is simulated in the DMD (1) 02 The first light intensity signal at the time of determination of the second-order geometric moment m 0,2 (S).

[0014] In a possible implementation manner, determining the centroid coordinates and roundness corresponding to the target object according to the multiple target geometric moments corresponding to the target object includes:

[0015] According to the 0th order geometric moment, the 1st order geometric moment m 1,0 (S) and the first-order geometric moment m 0,1 (S), determine the centroid coordinates corresponding to the target object; according to the 0th order geometric moment, the 1st order geometric moment m 1,0 (S), the first-order geometric moment m 0,1 (S), the second-order geometric moment m 2,0 (S) and the second-order geometric moment m 0,2 (S), determining the roundness corresponding to the target object.

[0016] In a possible implementation, the complementary single-pixel detection device further includes a front-end lens group, a triple-connected total internal reflection prism and a rear-end lens group.

[0017] According to another aspect of the present disclosure, a real-time positioning and classification device based on complementary single-pixel detection is provided, comprising: a light intensity signal determination module, which is used to control a digital micromirror device (DMD) of a complementary single-pixel detection device to sequentially simulate four target matrices, and when the DMD simulates any one of the target matrices, respectively determine a first light intensity signal of a first single-pixel detector of the complementary single-pixel detection device, and a second light intensity signal of a second single-pixel detector, wherein the complementary single-pixel detection device is used to perform single-pixel imaging on a target object, any one of the target matrices is used to indicate a deflection state of each micromirror in the DMD, and the installation positions of the first single-pixel detector and the second single-pixel detector satisfy optical path complementarity; A geometric moment determination module is used to determine multiple target geometric moments corresponding to the target object based on a first light intensity signal of the first single-pixel detector when the DMD simulates each target matrix, and a second light intensity signal of the second single-pixel detector when the DMD simulates each target matrix, wherein any one target geometric moment corresponding to the target object is used to describe the position and shape characteristics of the target object; a center of mass and roundness determination module is used to determine the center of mass coordinates and roundness corresponding to the target object based on the multiple target geometric moments corresponding to the target object; a positioning and classification module is used to determine a real-time positioning result and classification result corresponding to the target object based on the center of mass coordinates and roundness corresponding to the target object.

[0018] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0019] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0020] In the disclosed embodiments, the high flip frequency characteristic of the digital micromirror device can be fully utilized to control the digital micromirror device of the complementary single-pixel detection device to simulate four target matrices in sequence, and perform high-speed, dynamic and precise modulation on the light field corresponding to the target object, wherein any one of the target matrices is used to indicate the deflection state of each micromirror in the DMD; and when the digital micromirror device simulates any one of the target matrices, the first light intensity signal of the first single-pixel detector whose installation position satisfies the optical path complementarity in the complementary single-pixel detection device and the second light intensity signal of the second single-pixel detector are respectively determined, and the data required for real-time positioning and classification are acquired with only four projections, thereby improving the real-time performance of real-time positioning and classification; and, since the data received by the first single-pixel detector and the second single-pixel detector are light intensity signals, compared with the image-based positioning and classification methods in the prior art, the data throughput can be reduced and the data update rate can be increased. According to the first light intensity signal of the first single-pixel detector when the DMD simulates each target matrix, and the second light intensity signal of the second single-pixel detector when the DMD simulates each target matrix, multiple target geometric moments corresponding to the target object can be determined to describe the position and shape characteristics of the target object, and then the center of mass coordinates and circularity corresponding to the target object can be directly determined according to the multiple target geometric moments corresponding to the target object, without the need for complex image processing, with high data processing efficiency, and can further improve the real-time positioning and classification. According to the center of mass coordinates and circularity corresponding to the target object, the real-time positioning result and classification result corresponding to the target object can be determined, with high positioning accuracy and classification accuracy.

[0021] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0023] Figure 1 A flow chart showing a real-time positioning and classification method based on complementary single pixel detection according to an embodiment of the present disclosure is shown;

[0024] Figure 2 A schematic structural diagram of a complementary single-pixel detection device according to an embodiment of the present disclosure is shown;

[0025] Figure 3 A schematic diagram showing error diffusion between adjacent pixels according to an embodiment of the present disclosure;

[0026] Figure 4 A schematic diagram showing a spatial jitter averaging effect according to an embodiment of the present disclosure;

[0027] Figure 5 A schematic diagram showing a binary modulation mask according to an embodiment of the present disclosure;

[0028] Figure 6 A schematic diagram showing a binary modulation mask according to an embodiment of the present disclosure;

[0029] Figure 7 A schematic diagram of a real-time positioning and classification process according to an embodiment of the present disclosure is shown;

[0030] Figure 8 A block diagram of a real-time positioning and classification device based on complementary single-pixel detection according to an embodiment of the present disclosure is shown;

[0031] Fig. 9 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0032] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0033] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0034] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0035] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0036] With the rapid development of optoelectronic imaging technology and computer vision technology, the technology of real-time positioning and classification of fast-moving objects has been widely used in many fields, such as autonomous driving, industrial automation, and space exploration.

[0037] However, common positioning and classification methods based on traditional imaging technology require positioning and classification based on images, and rely on imaging systems with high spatial and temporal resolution. Such imaging systems usually require a large amount of data throughput, resulting in complex structures and poor real-time performance, and limited application scenarios. Especially in the application scenarios of high-speed moving objects, the accuracy and reliability of positioning and classification based on traditional imaging technology are easily affected by the motion blur of the objects. In addition, in some specific application scenarios, imaging of non-visible light wavelengths will be subject to greater restrictions, which will also affect the accuracy of real-time positioning and classification.

[0038] In response to the above problems, an alternative solution based on single-pixel imaging technology has been proposed in the prior art. A single detector can be used to sample a series of spatially resolved light patterns and then reconstruct images for real-time positioning and classification. Although this method can reduce data throughput and hardware requirements for imaging systems, the single-pixel imaging technology commonly used in the prior art is limited by the modulation frequency and the number of light field coding patterns, and its real-time performance is also poor.

[0039] In view of this, the embodiments of the present disclosure provide a real-time positioning and classification method based on complementary single-pixel detection, which can make full use of the high flip frequency characteristics of the digital micromirror device, perform high-speed, dynamic and precise modulation of the light field corresponding to the target object, and use the light intensity signal received by the complementary single-pixel detection device to quickly determine the center of mass coordinates and roundness corresponding to the target object, without the need for complex image processing, with high data processing efficiency, improve the real-time performance of real-time positioning and classification, and have high positioning accuracy and classification accuracy. The real-time positioning and classification method based on complementary single-pixel detection provided by the present disclosure is described in detail below.

[0040] Figure 1 A flow chart of a real-time positioning and classification method based on complementary single-pixel detection according to an embodiment of the present disclosure is shown. The real-time positioning and classification method based on complementary single-pixel detection can be executed by an electronic device such as a terminal device or a server. The terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The real-time positioning and classification method based on complementary single-pixel detection can be implemented by a processor calling computer-readable instructions stored in a memory. Alternatively, the real-time positioning and classification method based on complementary single-pixel detection can be executed by a server. Figure 1 As shown, the real-time positioning and classification method based on complementary single pixel detection includes:

[0041] In step S11, the digital micromirror device DMD of the complementary single-pixel detection device is controlled to simulate four target matrices in sequence, and when the DMD simulates any target matrix, the first light intensity signal of the first single-pixel detector of the complementary single-pixel detection device and the second light intensity signal of the second single-pixel detector are respectively determined, wherein the complementary single-pixel detection device is used to perform single-pixel imaging of the target object, any target matrix is ​​used to indicate the deflection state of each micromirror in the DMD, and the installation positions of the first single-pixel detector and the second single-pixel detector satisfy the optical path complementarity.

[0042] The target object here can represent any object that needs to be located and classified in real time. Its specific form can be flexibly set according to actual usage requirements, and this disclosure does not make specific limitations on this.

[0043] The complementary single-pixel detection device can be used to perform single-pixel imaging of the target object. Its specific structure can be flexibly set according to actual usage requirements. It should include a digital micromirror device DMD, a first single-pixel detector and a second single-pixel detector whose installation positions satisfy optical path complementarity. The present disclosure does not make specific limitations on this.

[0044] Among them, DMD is a reflective spatial light modulator with high-speed modulation capability (maximum modulation frequency is about 22.2kHz), including multiple independently controllable micromirrors (about millions), which can load a variety of different binary templates to simulate different target matrices in real time. Its specific form can be based on the implementation methods in the reference related technology, and the present disclosure does not make specific limitations on this.

[0045] The first single-pixel detector and the second single-pixel detector are usually the same single-pixel detectors, which are respectively used to receive the first light intensity signal and the second light intensity signal after DMD modulation; the specific forms of the first single-pixel detector and the second single-pixel detector can refer to the implementation methods in the relevant technology, and the present disclosure does not make specific limitations on this.

[0046] Normally, any one micromirror in the DMD can only be deflected to a first preset deflection angle and a second preset deflection angle, and the first preset deflection angle is equal to the absolute value of the first preset deflection angle. The specific values ​​of the first preset deflection angle and the second preset deflection angle can refer to the commonly used settings of the micromirrors of the DMD in the related art. For example, the first preset deflection angle can be set to +12° and the second preset deflection angle can be set to -12°, or the first preset deflection angle can be set to +24° and the second preset deflection angle can be set to -24°, etc. The embodiments of the present disclosure do not specifically limit this.

[0047] Generally, for any micromirror in the DMD, when the micromirror is deflected to a first preset deflection angle, the micromirror can be considered to be in an "on" state, at which time the micromirror can reflect the light incident to the DMD to the first single-pixel detector; when the micromirror is deflected to a second preset deflection angle, the micromirror can be considered to be in an "off" state, at which time the micromirror can reflect the light incident to the DMD to the second single-pixel detector. Accordingly, when the DMD simulates any target matrix, the complementary first light intensity signal and the second light intensity signal can be detected by using the first single-pixel detector and the second single-pixel detector with complementary optical paths.

[0048] In a possible implementation, the complementary single-pixel detection device further includes a front-end lens group, a triple-connected total internal reflection prism, and a rear-end lens group.

[0049] Figure 2 FIG. 2 is a schematic diagram showing the structure of a complementary single pixel detection device according to an embodiment of the present disclosure. Figure 2 As shown, the complementary single-pixel detection device 200 includes a front-end lens group 201, a triple-connected total internal reflection prism 202, a digital micromirror device 203, a first rear-end lens group 204, a second rear-end lens group 205, a first single-pixel detector 206 and a second single-pixel detector 207.

[0050] Among them, the front lens group 201 can focus the light from the target object onto the surface of the digital micromirror device 203. Its specific form can refer to the implementation method in the relevant technology, and the present disclosure does not make any specific limitation on this.

[0051] The triple-link total internal reflection lens 202 can be used to reduce the probability of optical path conflict in the complementary single-pixel detection device 200, and reduce the interference of stray light. Specifically, since the deflection angle of each micromirror in the digital micromirror device 203 is usually small (for example, usually ±12° or ±24°, etc.), the incident light beam and the reflected light beam corresponding to each micromirror are very close, which can easily cause optical path conflict, which may affect the accuracy of the first light intensity signal and the second light intensity signal obtained subsequently, and thus affect the accuracy of real-time positioning and classification of the target object. By using the triple-link total internal reflection lens, the optical path can be separated and folded, thereby reducing the probability of optical path interference, while maintaining the compactness of the structure of the complementary single-pixel detection device, aperture sharing is achieved, and the accuracy and reliability of the complementary single-pixel detection device 200 are improved. The specific form of the triple-link total internal reflection lens can refer to the implementation in the relevant technology, and the present disclosure does not make specific limitations on this.

[0052] The rear lens group may include: a first rear lens group corresponding to the first single-pixel detector, used to converge the light emitted from the two sides of the triple-connected total internal reflection prism to the first single-pixel detector; and a second rear lens group corresponding to the second single-pixel detector, used to converge the light emitted from the two sides of the triple-connected total internal reflection prism to the second single-pixel detector. The specific forms of the first rear lens group and the second rear lens group can refer to the implementation in the related art, and the present disclosure does not specifically limit this.

[0053] By controlling the DMD of the complementary single-pixel detection device to simulate four target matrices in sequence, the incident light from the target object can be modulated multiple times, and when the DMD simulates any target matrix, the first light intensity signal of the first single-pixel detector of the complementary single-pixel detection device and the second light intensity signal of the second single-pixel detector are determined respectively.

[0054] Among them, any one of the target matrices can be used to indicate the deflection state of each micromirror in the DMD, and its specific form and content can be flexibly set according to actual use requirements, and the present disclosure does not specifically limit this. The deflection state of any one of the micromirrors here can be flexibly set according to actual use requirements, for example, it can include the deflection direction and deflection angle of the micromirror, etc., and the present disclosure does not specifically limit this.

[0055] The process of controlling the digital micromirror device DMD of the complementary single-pixel detection device to sequentially simulate four target matrices will be described in detail later in conjunction with possible implementation methods of the present disclosure, and will not be elaborated here.

[0056] In step S12, multiple target geometric moments corresponding to the target object are determined based on the first light intensity signal of the first single-pixel detector when the DMD simulates each target matrix, and the second light intensity signal of the second single-pixel detector when the DMD simulates each target matrix, wherein any target geometric moment corresponding to the target object is used to describe the position and shape characteristics of the target object.

[0057] The center of mass and geometric shape of any object can be represented by the geometric moments corresponding to the object. Therefore, by determining multiple target geometric moments corresponding to the target object, the position and shape characteristics of the target object can be described, so that the target object can be located and classified in real time using multiple target geometric moments.

[0058] In an example, for a target object S in any two-dimensional image I(x, y), its corresponding p+q-order geometric moment can be expressed as formula (1):

[0059] m p，q (S) = ∫∫(x, y)x p y qdxdy (1)

[0060] Among them, m p,q (S) represents the p+q order geometric moment corresponding to the target object S; x represents the horizontal coordinate corresponding to any pixel in the two-dimensional image I(x,y); y represents the vertical coordinate corresponding to any pixel in the two-dimensional image I(x,y).

[0061] By using the first light intensity signal of the first single-pixel detector when the DMD simulates each target matrix and the second light intensity signal of the second single-pixel detector when the DMD simulates each target matrix, multiple target geometric moments corresponding to the target object can be determined to describe the position and shape characteristics of the target object. The multiple target geometric moments here can include multiple p+q order geometric moments corresponding to the target object.

[0062] Among them, the specific method of determining multiple target geometric moments corresponding to the target object based on the first light intensity signal of the first single-pixel detector when the DMD simulates each target matrix, and the second light intensity signal of the second single-pixel detector when the DMD simulates each target matrix, can refer to the implementation methods in the relevant technology. For example, it can be implemented by using methods such as light field feature dimensionality reduction, and the present disclosure does not make specific limitations on this.

[0063] The following text will describe in detail the process of determining multiple target geometric moments corresponding to the target object based on the first light intensity signal of the first single-pixel detector when the DMD simulates each target matrix, and the second light intensity signal of the second single-pixel detector when the DMD simulates each target matrix, in combination with possible implementation methods of the present disclosure. No further details will be given here.

[0064] In step S13, the centroid coordinates and circularity corresponding to the target object are determined according to a plurality of target geometric moments corresponding to the target object.

[0065] For any two-dimensional image including a target object, the position of the target object in the two-dimensional image can be directly reflected by the centroid coordinates corresponding to the target object, which can be used to provide a basis for locating the target object. Similarly, the roundness corresponding to the target object can indicate the similarity of the target object to the theoretical circle, which is widely used in object classification tasks. Therefore, the centroid coordinates and roundness corresponding to the target object can be determined based on the multiple target geometric moments corresponding to the target object, which can serve as the basis for real-time positioning and classification of the target object.

[0066] Among them, the specific method of determining the center of mass coordinates and roundness corresponding to the target object according to the multiple target geometric moments corresponding to the target object can refer to the implementation method in the relevant technology, and the present disclosure does not make specific limitations on this.

[0067] In an example, for a target object S in any two-dimensional image I(x, y), the relationship between the centroid coordinates corresponding to the target object S and its geometric moment can be expressed as formula (2):

[0068]

[0069] Among them, C x (S) represents the horizontal coordinate of the center of mass of the target object S in the two-dimensional image I (x, y); C y (S) represents the vertical coordinate of the center of mass of the target object S in the two-dimensional image I (x, y); m 0,0 (S) represents the 0th-order geometric moment corresponding to the target object S; m 1,0 (S) represents the first-order geometric moment corresponding to the target object S; m 0,1 (S) represents the first-order geometric moment corresponding to the target object S.

[0070] The relationship between the roundness of the target object S and its geometric moment can be expressed as formula (3):

[0071]

[0072] Among them, ξ C Indicates the roundness of the target object S; m 2,0 (S) represents the second-order geometric moment corresponding to the target object S; m 0,2 (S) represents the second-order geometric moment corresponding to the target object S.

[0073] In step S14, the real-time positioning result and classification result corresponding to the target object are determined according to the centroid coordinates and circularity corresponding to the target object.

[0074] According to the center of mass coordinates corresponding to the target object, the real-time positioning result corresponding to the target object can be determined according to the actual positioning requirements and the relevant equipment parameters of the complementary single-pixel detection device. The specific method can refer to the implementation method in the relevant technology, and the present disclosure does not make specific limitations on this.

[0075] According to the roundness corresponding to the target object, the classification result corresponding to the target object can be determined according to the actual positioning requirements and the relevant equipment parameters of the complementary single-pixel detection device. The specific method can refer to the implementation method in the relevant technology, and the present disclosure does not make specific limitations on this.

[0076] According to the relevant experimental verification results, the real-time positioning and classification method based on complementary single-pixel detection provided by the present disclosure is used to perform real-time positioning of any target object, and the root mean square error (RMSE) of the real-time positioning result is less than 0.5 pixels, which has high positioning accuracy. Thirty different objects are classified, and the accuracy of the classification results is as high as 93.3%, which has high classification accuracy and reliability.

[0077] In the disclosed embodiments, the high flip frequency characteristic of the digital micromirror device can be fully utilized to control the digital micromirror device of the complementary single-pixel detection device to simulate four target matrices in sequence, and perform high-speed, dynamic and precise modulation on the light field corresponding to the target object, wherein any one of the target matrices is used to indicate the deflection state of each micromirror in the DMD; and when the digital micromirror device simulates any one of the target matrices, the first light intensity signal of the first single-pixel detector whose installation position satisfies the optical path complementarity in the complementary single-pixel detection device and the second light intensity signal of the second single-pixel detector are respectively determined, and the data required for real-time positioning and classification are acquired with only four projections, thereby improving the real-time performance of real-time positioning and classification; and, since the data received by the first single-pixel detector and the second single-pixel detector are light intensity signals, compared with the image-based positioning and classification methods in the prior art, the data throughput can be reduced and the data update rate can be increased. According to the first light intensity signal of the first single-pixel detector when the DMD simulates each target matrix, and the second light intensity signal of the second single-pixel detector when the DMD simulates each target matrix, multiple target geometric moments corresponding to the target object can be determined to describe the position and shape characteristics of the target object, and then the center of mass coordinates and circularity corresponding to the target object can be directly determined according to the multiple target geometric moments corresponding to the target object, without the need for complex image processing, with high data processing efficiency, and can further improve the real-time positioning and classification. According to the center of mass coordinates and circularity corresponding to the target object, the real-time positioning result and classification result corresponding to the target object can be determined, with high positioning accuracy and classification accuracy.

[0078] In one possible implementation, a digital micromirror device (DMD) of a complementary single-pixel detection apparatus is controlled to simulate four target matrices in sequence, including: for any target matrix, using an error diffusion algorithm, controlling the DMD to load a binary modulation mask corresponding to the target matrix, wherein any binary modulation mask represents a binary template composed of all micromirrors in the DMD.

[0079] Since any micromirror in a DMD has only two states, "on" and "off", the DMD can usually only perform high-speed binary modulation to generate a corresponding binary template. However, the matrix elements of any target matrix are not binary elements of 0 or 1. Therefore, for any target matrix, an error diffusion algorithm is required to convert the target matrix into a corresponding binary modulation mask to control the micromirrors of the DMD to form a corresponding binary template, thereby simulating the target matrix.

[0080] Among them, the specific form of the error diffusion algorithm can refer to the implementation methods in the relevant technology, such as the Floyd-Steinberg (FS) algorithm, the Jarvis-Judice-Ninke algorithm, the Stucki algorithm, etc., and the present disclosure does not make specific limitations on this.

[0081] In one possible implementation, the error diffusion algorithm is a Floyd-Stuhberg dithering algorithm.

[0082] For any target pixel in the image, the FS dithering algorithm can be used to determine the quantization error corresponding to the target pixel based on the pixel value corresponding to the target pixel and the preset error threshold, and the quantization error can be applied to the adjacent pixels corresponding to the target pixel according to the preset error weight distribution to perform error diffusion. This can convert a low-bit-depth image into a high-bit-depth image, reduce color distortion in image transformation, and improve the visual effect of the image.

[0083] Figure 3 FIG. 2 is a schematic diagram showing an error diffusion between adjacent pixels according to an embodiment of the present disclosure. Figure 3 As shown, each square represents a pixel. In the case of binary jitter, for any pixel, when the pixel value of the pixel changes, the quantization error corresponding to the pixel is distributed to the four pixels on the right, lower right, below and lower left of the pixel according to the preset weight error. Among them, the weight error corresponding to the pixel on the right side of the pixel is 7 / 16, the weight error corresponding to the pixel on the lower right side of the pixel is 1 / 16, the weight error corresponding to the pixel below the pixel is 5 / 16, and the weight error corresponding to the pixel on the lower left side of the pixel is 3 / 16.

[0084] Based on the above principle, multiple adjacent micromirrors in the DMD can be regarded as a pixel. When each micromirror in the pixel switches between the "on" state and the "off" state, the pixel value of the pixel can be regarded as changing according to the spatial jitter averaging effect. The number of micromirrors required to construct a pixel can be flexibly set according to actual usage requirements, for example, it can be set to four adjacent micromirrors, etc., and the present disclosure does not make specific limitations on this.

[0085] Figure 4 FIG. 2 is a schematic diagram showing a spatial jitter averaging effect according to an embodiment of the present disclosure. Figure 4 As shown, the white square represents a micromirror with a deflection angle of +12°, that is, in the "open" state, and the black square represents a micromirror with a deflection angle of -12°, that is, in the "closed" state. When the deflection angles of the four adjacent micromirrors are all +12°, the pixel value corresponding to the pixel composed of these four micromirrors is 1; when the deflection angles of the two micromirrors on the left and the two micromirrors on the right of the four adjacent micromirrors are +12°, the pixel value corresponding to the pixel composed of these four micromirrors is 0.5; when the deflection angles of the four adjacent micromirrors are all -12°, the pixel value corresponding to the pixel composed of these four micromirrors is 0.

[0086] Through these pixels including multiple adjacent micromirrors, a mask image with a lower bit depth can be determined; using the FS dithering algorithm, the pixel value corresponding to each pixel in the mask image can be determined according to any target matrix; and for any pixel, the deflection angle of each micromirror in the pixel can be determined according to its pixel value, and then the binary modulation mask corresponding to the target matrix can be determined.

[0087] Figure 5 FIG. 1 is a schematic diagram showing a binary modulation mask according to an embodiment of the present disclosure. Figure 5 As shown in (a), the mask image at the top is determined based on the four target matrices, and the binary modulation mask corresponding to each target matrix is ​​determined using the FS dithering algorithm. Figure 5 As shown in (b), the left side is a part of the mask image determined according to an arbitrary target matrix, and its corresponding bit depth is 8 bits. The right side is the same part of the binary modulation mask corresponding to the target matrix determined by the FS dithering algorithm, and its corresponding bit depth is 1 bit.

[0088] In one possible implementation, the four target matrices include:

[0089] First target matrix Wherein, M represents the number of micromirror columns corresponding to DMD; N represents the number of micromirror rows corresponding to DMD;

[0090] Second target matrix

[0091] The third target matrix

[0092] Fourth target matrix

[0093] The p+q-order geometric moment corresponding to any target image can be simulated by the micromirror in the "on" state in the DMD, which can be specifically expressed as formula (4):

[0094]

[0095] Among them, M (1) pq Represents the matrix of micromirror simulation in the "on" state in the DMD; M represents the number of micromirror columns corresponding to the DMD; N represents the number of micromirror rows corresponding to the DMD.

[0096] Combining the above formulas (2) and (3), it can be seen that to determine the centroid coordinates and circularity corresponding to the target object, only the corresponding 0th order geometric moment, two 1st order geometric moments and two 2nd order geometric moments are needed. For the complementary single-pixel detection device, the 0th order geometric moment corresponding to the target object can be directly obtained by summing and transforming the first light intensity signal corresponding to the first single-pixel detector and the second light intensity signal corresponding to the second single-pixel detector at the same time. Therefore, the DMD only needs to simulate four different target matrices to obtain the two 1st order geometric moments and two 2nd order geometric moments corresponding to the target object.

[0097] Therefore, combined with formula (3), we can determine that the four target matrices are M (1) 10 、M (1) 01 、M (1) 20 and M (1) 02 , using the FS dithering algorithm, the DMD can be controlled to sequentially load the binary modulation masks corresponding to the four target matrices. The specific order of loading the binary modulation masks corresponding to the four target matrices can be flexibly set according to actual usage requirements, and the present disclosure does not make specific limitations on this.

[0098] Figure 6 FIG. 1 is a schematic diagram showing a binary modulation mask according to an embodiment of the present disclosure. Figure 6 As shown, the FS dithering algorithm can be used to control the digital micromirror device in the complementary single pixel detection device, and the target matrices are loaded in sequence, namely M (1) 10 The corresponding binary modulation mask 601, M (1) 01The corresponding binary modulation mask 602, M (1) 20 The corresponding binary modulation mask 603, and M (1) 02 The corresponding binary modulation mask 604.

[0099] In a possible implementation, the multiple target geometric moments include: 0th order geometric moment m 0,0 (S), 1st order geometric moment m 1,0 (S), 1st order geometric moment m 0,1 (S), second-order geometric moment m 2,0 (S) and the second-order geometric moment m 0,2 (S), where S represents the target object; according to the first light intensity signal of the first single-pixel detector when the DMD simulates each target matrix, and the second light intensity signal of the second single-pixel detector when the DMD simulates each target matrix, determining multiple target geometric moments corresponding to the target object, including: according to the first light intensity signal of the first single-pixel detector when the DMD simulates any target matrix, and the second light intensity signal of the second single-pixel detector when the DMD simulates the target matrix, determining the 0th order geometric moment m 0,0 (S); According to the first single pixel detector in DMD simulation first target matrix M (1) 10 The first light intensity signal at time , determines the first-order geometric moment m 1,0 (S); According to the first single pixel detector in DMD simulation second target matrix M (1) 01 The first light intensity signal at time , determines the first-order geometric moment m 0,1 (S); According to the first single pixel detector in the DMD simulation third target matrix M (1) 20 The first light intensity signal at time , determines the second-order geometric moment m 2,0 (S); According to the first single pixel detector in the DMD simulation fourth target matrix M (1) 02 The first light intensity signal at time , determines the second-order geometric moment m 0,2 (S).

[0100] Specifically, for the complementary single-pixel detection device, when the DMD simulates each target matrix, the first light intensity signal corresponding to the first single-pixel detector is complementary to the second light intensity signal corresponding to the second single-pixel detector. Therefore, the 0th-order geometric moment m can be determined according to the first light intensity signal of the first single-pixel detector when the DMD simulates any target matrix and the second light intensity signal of the second single-pixel detector when the DMD simulates the target matrix. 0,0 (S).

[0101] When the DMD sequentially simulates the above four target matrices, the first light intensity signal corresponding to the first single-pixel detector can be acquired sequentially, thereby sequentially obtaining two first-order geometric moments and two second-order geometric moments corresponding to the target object.

[0102] Specifically, the first single pixel detector can be used to simulate the first target matrix M in the DMD (1) 10 The first light intensity signal at time , determines the first-order geometric moment m 1,0 (S); According to the first single pixel detector in DMD simulation second target matrix M (1) 01 The first light intensity signal at time , determines the first-order geometric moment m 0,1 (S); According to the first single pixel detector in the DMD simulation third target matrix M (1) 20 The first light intensity signal at time , determines the second-order geometric moment m 2,0 (S); According to the first single pixel detector in the DMD simulation fourth target matrix M (1) 02 The first light intensity signal at time , determines the second-order geometric moment m 0,2 (S).

[0103] Among them, the specific method of determining each geometric moment according to the first light intensity signal can refer to the implementation method in the relevant technology, and the present disclosure does not make any specific limitation on this.

[0104] The first single-pixel detector and the second single-pixel detector with complementary optical paths are used to receive the light intensity signals when the DMD simulates four target matrices in sequence. Compared with the method of realizing real-time positioning and classification based on image sensors in the prior art, only four modulation projections are required, and the amount of light intensity signal data obtained is smaller and the update rate is faster. The five target geometric moments corresponding to the target object can be determined quickly and simply to determine the center of mass position and roundness corresponding to the target object, thereby improving the efficiency of real-time positioning and classification of the target object.

[0105] In a possible implementation, the center of mass coordinates and the roundness of the target object are determined according to the multiple target geometric moments corresponding to the target object, including: determining the center of mass coordinates and the roundness of the target object according to the 0th order geometric moment, the 1st order geometric moment m 1,0 (S) and the first-order geometric moment m 0,1 (S), determine the coordinates of the center of mass corresponding to the target object; according to the 0th order geometric moment and the 1st order geometric moment m 1,0 (S), 1st order geometric moment m 0,1 (S), second-order geometric moment m 2,0 (S) and the second-order geometric moment m 0,2 (S), determine the circularity corresponding to the target object.

[0106] The 0th-order geometric moment and 1st-order geometric moment m corresponding to the target object 1,0 (S) and the first-order geometric moment m 0,1 By inputting (S) into the above formula (2), the center of mass coordinates corresponding to the target object can be directly determined without complicated image processing, which can significantly improve the efficiency of real-time positioning of the target object and ensure that the real-time positioning has a high accuracy to meet the needs of high-speed positioning.

[0107] Similarly, the 0th-order geometric moment and 1st-order geometric moment m corresponding to the target object are 1,0 (S), 1st order geometric moment m 0,1 (S), second-order geometric moment m 2,0 (S) and the second-order geometric moment m 0,2 By inputting (S) into the above formula (3), the roundness corresponding to the target object can be directly determined for classification and identification without going through the complex process of target object extraction and identification, which can significantly improve the efficiency and accuracy of target object classification.

[0108] Figure 7 FIG. 2 is a schematic diagram showing a process of real-time positioning and classification according to an embodiment of the present disclosure. Figure 7 As shown, the first column is a first-angle light field image determined by taking the light path corresponding to the first single-pixel detector as the observation light path when the digital micromirror device sequentially loads the binary modulation masks corresponding to the four target matrices; the second column is a second-angle light field image determined by taking the light path corresponding to the second single-pixel detector as the observation light path when the digital micromirror device sequentially loads the binary modulation masks corresponding to the four target matrices; the second-angle light field image and the first-angle light field image at the same moment are complementary. Among them, the projection of the target object on the digital micromirror device is a five-pointed star.

[0109] The digital micromirror device can sequentially load the binary modulation masks corresponding to the four target matrices at four moments ΔT, 2ΔT, 3ΔT, and 4ΔT. The first single-pixel detector and the second single-pixel detector can determine the corresponding first light intensity signal and second light intensity signal at each moment, thereby realizing real-time positioning and classification of the target object based on the aforementioned process. At moment t, the position of the target object changes, and the digital micromirror device can sequentially load the binary modulation masks corresponding to the four target matrices at four moments t+ΔT, t+2ΔT, t+3ΔT, and t+4ΔT, and repeat the aforementioned process again. Among them, the specific value of ΔT can be flexibly set according to actual use requirements, depending on the performance of the digital micromirror device. For example, ΔT can be set to 45μs, etc., and the present disclosure does not make specific limitations on this.

[0110] In the disclosed embodiments, the high flip frequency characteristic of the digital micromirror device can be fully utilized to control the digital micromirror device of the complementary single-pixel detection device to simulate four target matrices in sequence, and perform high-speed, dynamic and precise modulation on the light field corresponding to the target object, wherein any one of the target matrices is used to indicate the deflection state of each micromirror in the DMD; and when the digital micromirror device simulates any one of the target matrices, the first light intensity signal of the first single-pixel detector whose installation position satisfies the optical path complementarity in the complementary single-pixel detection device and the second light intensity signal of the second single-pixel detector are respectively determined, and the data required for real-time positioning and classification are acquired with only four projections, thereby improving the real-time performance of real-time positioning and classification; and, since the data received by the first single-pixel detector and the second single-pixel detector are light intensity signals, compared with the image-based positioning and classification methods in the prior art, the data throughput can be reduced and the data update rate can be increased. According to the first light intensity signal of the first single-pixel detector when the DMD simulates each target matrix, and the second light intensity signal of the second single-pixel detector when the DMD simulates each target matrix, multiple target geometric moments corresponding to the target object can be determined to describe the position and shape characteristics of the target object, and then the center of mass coordinates and circularity corresponding to the target object can be directly determined according to the multiple target geometric moments corresponding to the target object, without the need for complex image processing, with high data processing efficiency, and can further improve the real-time positioning and classification. According to the center of mass coordinates and circularity corresponding to the target object, the real-time positioning result and classification result corresponding to the target object can be determined, with high positioning accuracy and classification accuracy.

[0111] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0112] In addition, the present disclosure also provides a real-time positioning and classification device based on complementary single-pixel detection, an electronic device and a storage medium, all of which can be used to implement any real-time positioning and classification method based on complementary single-pixel detection provided by the present disclosure. The corresponding technical solutions and descriptions can be found in the corresponding records in the method part and will not be repeated here.

[0113] Figure 8 FIG. 2 is a block diagram of a real-time positioning and classification device based on complementary single pixel detection according to an embodiment of the present disclosure. Figure 8 As shown, the device 800 includes:

[0114] A light intensity signal determination module 801 is used to control the digital micromirror device DMD of the complementary single-pixel detection device to simulate four target matrices in sequence, and when the DMD simulates any one of the target matrices, respectively determine a first light intensity signal of a first single-pixel detector of the complementary single-pixel detection device and a second light intensity signal of a second single-pixel detector, wherein the complementary single-pixel detection device is used to perform single-pixel imaging of a target object, any one of the target matrices is used to indicate the deflection state of each micromirror in the DMD, and the installation positions of the first single-pixel detector and the second single-pixel detector satisfy optical path complementarity;

[0115] The geometric moment determination module 802 is used to determine a plurality of target geometric moments corresponding to the target object according to a first light intensity signal of the first single-pixel detector when the DMD simulates each target matrix, and a second light intensity signal of the second single-pixel detector when the DMD simulates each target matrix, wherein any one of the target geometric moments corresponding to the target object is used to describe the position and shape characteristics of the target object;

[0116] A centroid and roundness determination module 803 is used to determine the centroid coordinates and roundness corresponding to the target object according to a plurality of target geometric moments corresponding to the target object;

[0117] The positioning and classification module 804 is used to determine the real-time positioning result and classification result corresponding to the target object according to the centroid coordinates and roundness corresponding to the target object.

[0118] In one possible implementation, the light intensity signal determination module 801 is specifically used to: for any target matrix, use an error diffusion algorithm to control the DMD to load the binary modulation mask corresponding to the target matrix, wherein any binary modulation mask represents a binary template composed of all micromirrors in the DMD.

[0119] In one possible implementation, the error diffusion algorithm is a Floyd-Stuhberg dithering algorithm.

[0120] In one possible implementation, the four target matrices include:

[0121] First target matrix Wherein, M represents the number of micromirror columns corresponding to DMD; N represents the number of micromirror rows corresponding to DMD;

[0122] Second target matrix

[0123] The third target matrix

[0124] Fourth target matrix

[0125] In a possible implementation, the multiple target geometric moments include: 0th order geometric moment m0,0 (S), 1st order geometric moment m 1,0 (S), 1st order geometric moment m 0,1 (S), second-order geometric moment m 2,0 (S) and the second-order geometric moment m 0,2 (S), where S represents the target object; the geometric moment determination module 802 is specifically used to: determine the 0th order geometric moment m according to the first light intensity signal of the first single pixel detector when the DMD simulates any target matrix, and the second light intensity signal of the second single pixel detector when the DMD simulates the target matrix. 0,0 (S); According to the first single pixel detector in DMD simulation first target matrix M (1) 10 The first light intensity signal at time , determines the first-order geometric moment m 1,0 (S); According to the first single pixel detector in DMD simulation second target matrix M (1) 01 The first light intensity signal at time , determines the first-order geometric moment m 0,1 (S); According to the first single pixel detector in the DMD simulation third target matrix M (1) 20 The first light intensity signal at time , determines the second-order geometric moment m 2,0 (S); According to the first single pixel detector in the DMD simulation fourth target matrix M (1) 02 The first light intensity signal at time , determines the second-order geometric moment m 0,2 (S).

[0126] In a possible implementation, the centroid and roundness determination module 803 is specifically configured to: 1,0 (S) and the first-order geometric moment m 0,1 (S), determine the coordinates of the center of mass corresponding to the target object; according to the 0th order geometric moment and the 1st order geometric moment m 1,0 (S), 1st order geometric moment m 0,1 (S), second-order geometric moment m 2,0 (S) and the second-order geometric moment m 0,2 (S), determine the circularity corresponding to the target object.

[0127] In a possible implementation, the complementary single-pixel detection device further includes a front-end lens group, a triple-connected total internal reflection prism, and a rear-end lens group.

[0128] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0129] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0130] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0131] Fig. 9 1900 is a block diagram of an electronic device according to an embodiment of the present disclosure. For example, the apparatus 1900 may be provided as a server or a terminal device. Fig. 9 , the apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0132] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2000. TM , MacOS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.

[0133] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the device 1900 to perform the above method.

[0134] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0135] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0136] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0137] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0138] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0139] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0140] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0141] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0142] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A real-time positioning and classification method based on complementary single pixel detection, characterized in that: include: A digital micromirror device (DMD) of a complementary single-pixel detection device is controlled to simulate four target matrices in sequence, and when the DMD simulates any one of the target matrices, a first light intensity signal of a first single-pixel detector of the complementary single-pixel detection device and a second light intensity signal of a second single-pixel detector are respectively determined, wherein the complementary single-pixel detection device is used to perform single-pixel imaging of a target object, any one of the target matrices is used to indicate the deflection state of each micromirror in the DMD, and the installation positions of the first single-pixel detector and the second single-pixel detector satisfy optical path complementarity; Determine a plurality of target geometric moments corresponding to the target object according to a first light intensity signal of the first single-pixel detector when the DMD simulates each target matrix, and a second light intensity signal of the second single-pixel detector when the DMD simulates each target matrix, wherein any one of the target geometric moments corresponding to the target object is used to describe the position and shape characteristics of the target object; Determining the centroid coordinates and roundness corresponding to the target object according to a plurality of target geometric moments corresponding to the target object; According to the centroid coordinates and circularity corresponding to the target object, a real-time positioning result and a classification result corresponding to the target object are determined.

2. The method according to claim 1, characterized in that The digital micromirror device DMD controlling the complementary single pixel detection device sequentially simulates four target matrices, including: For any of the target matrices, an error diffusion algorithm is used to control the DMD to load a binary modulation mask corresponding to the target matrix, wherein any of the binary modulation masks represents a binary template composed of all micromirrors in the DMD.

3. The method according to claim 2, characterized in that The error diffusion algorithm is a Floyd-Sternberg dithering algorithm.

4. The method according to any one of claims 1 to 3, characterized in that The four target matrices include: First target matrix Wherein, M represents the number of micromirror columns corresponding to the DMD; N represents the number of micromirror rows corresponding to the DMD; Second target matrix The third target matrix Fourth target matrix 5. The method according to claim 4, characterized in that The multiple target geometric moments include: 0th order geometric moment m 0,0 (S), 1st order geometric moment m 1,0 (S), 1st order geometric moment m 0,1 (S), second-order geometric moment m 2,0 (S) and the second-order geometric moment m 0,2 (S), where S represents the target object; Determining a plurality of target geometric moments corresponding to the target object according to a first light intensity signal of the first single-pixel detector when the DMD simulates each target matrix, and a second light intensity signal of the second single-pixel detector when the DMD simulates each target matrix, comprises: The 0th-order geometric moment m is determined according to the first light intensity signal of the first single-pixel detector when the DMD simulates any target matrix and the second light intensity signal of the second single-pixel detector when the DMD simulates the target matrix. 0,0 (S); According to the first single pixel detector, the DMD simulates a first target matrix M (1) 10 The first light intensity signal at the time of determination of the first-order geometric moment m 1,0 (S); According to the first single pixel detector, the second target matrix M is simulated in the DMD (1) 01 The first light intensity signal at the time of determination of the first-order geometric moment m 0,1 (S); According to the first single pixel detector, the DMD simulates a third target matrix M (1) 20 The first light intensity signal at the time of determination of the second-order geometric moment m 2,0 (S); According to the first single pixel detector, the fourth target matrix M is simulated in the DMD (1) 02 The first light intensity signal at the time of determination of the second-order geometric moment m 0,2 (S).

6. The method according to claim 5, characterized in that Determining the centroid coordinates and roundness corresponding to the target object according to the multiple target geometric moments corresponding to the target object includes: According to the 0th order geometric moment, the 1st order geometric moment m 1,0 (S) and the first-order geometric moment m 0,1 (S), determining the centroid coordinates corresponding to the target object; According to the 0th order geometric moment, the 1st order geometric moment m 1,0 (S), the first-order geometric moment m 0,1 (S), the second-order geometric moment m 2,0 (S) and the second-order geometric moment m 0,2 (S), determining the roundness corresponding to the target object.

7. The method according to any one of claims 1 to 3, characterized in that The complementary single-pixel detection device also includes a front-end lens group, a triple-connected total internal reflection prism and a rear-end lens group.

8. A real-time positioning and classification device based on complementary single pixel detection, characterized in that: include: A light intensity signal determination module, used for controlling a digital micromirror device (DMD) of a complementary single-pixel detection device to sequentially simulate four target matrices, and respectively determining a first light intensity signal of a first single-pixel detector of the complementary single-pixel detection device and a second light intensity signal of a second single-pixel detector when the DMD simulates any one of the target matrices, wherein the complementary single-pixel detection device is used to perform single-pixel imaging of a target object, any one of the target matrices is used to indicate a deflection state of each micromirror in the DMD, and the installation positions of the first single-pixel detector and the second single-pixel detector satisfy optical path complementarity; a geometric moment determination module, used for determining a plurality of target geometric moments corresponding to the target object according to a first light intensity signal of the first single-pixel detector when the DMD simulates each target matrix, and a second light intensity signal of the second single-pixel detector when the DMD simulates each target matrix, wherein any one of the target geometric moments corresponding to the target object is used for describing the position and shape characteristics of the target object; A centroid and roundness determination module, used to determine the centroid coordinates and roundness corresponding to the target object according to a plurality of target geometric moments corresponding to the target object; The positioning and classification module is used to determine the real-time positioning result and classification result corresponding to the target object according to the centroid coordinates and roundness corresponding to the target object.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method described in any one of claims 1 to 7 when executing the instructions stored in the memory.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images