Multi-target tracking ranging system, method, device and equipment and storage medium

Abstract

The invention relates to the technical field of laser ranging, and discloses a multi-target tracking ranging system, method, device and equipment and a storage medium, the system comprises image acquisition equipment, ranging equipment and a servo turntable: the image acquisition equipment is used for performing image acquisition on a target object to obtain image frame data; the distance measurement equipment is used for measuring the distance between the target object and the distance measurement equipment at the same moment as the image acquisition equipment to obtain distance measurement data; the servo turntable is arranged on the driving platform and is used for supporting the image acquisition equipment and the distance measurement equipment; and in response to the control instruction, adjusting a horizontal angle and a pitching angle of the image acquisition equipment, so that the target object appears in an image optical center of the image acquisition equipment. Through combination of the monocular camera and the laser ranging module, the requirement of the driving platform for multi-target tracking is met, distance measurement is performed on an interested target, and the system complexity is remarkably reduced. And the long-distance multi-target distance measurement precision and efficiency are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of laser ranging, and particularly relates to a multi-target tracking and ranging system, method, device, equipment and storage medium. Background Art

[0002] When measuring the distance of a target on a moving platform by using an optical camera with a turntable in combination with a laser ranging module, the method of tracking the target by the image optical center is adopted for distance measurement.

[0003] In the related art, in the actual use process, when the stability of the moving platform is poor and the response speed of the turntable is not fast enough, it is very difficult to measure the distance of the target by tracking the image optical center, the complexity of the motion control of the turntable is relatively high, and the camera is occupied by a single task, making it difficult to realize the tracking and ranging of multiple targets. Summary of the Invention

[0004] In view of this, the present invention provides a multi-target tracking and ranging system, method, device, equipment and storage medium to solve the problem that it is difficult to realize the tracking and ranging of multiple targets.

[0005] In a first aspect, the present invention provides a multi-target tracking and ranging system, which includes an image acquisition device, a ranging device, and a servo turntable:

[0006] The image acquisition device is used to acquire an image of a target object to obtain image frame data, where the image frame data includes the target object;

[0007] The ranging device is used to measure the distance between the target object and the ranging device at the same moment as the image acquisition device to obtain ranging data;

[0008] The servo turntable is arranged on the moving platform and is used to support the image acquisition device and the ranging device; in response to a control instruction, it adjusts the horizontal angle and the pitch angle of the image acquisition device so that the target object appears at the image optical center of the image acquisition device.

[0009] In the present invention, by combining a monocular camera and a laser ranging module, an easy-to-implement method is provided, which can not only meet the requirements of the moving platform for multi-target tracking, but also measure the distance of the target of interest, and can significantly reduce the system complexity. Compared with the traditional tracking and ranging, it can effectively improve the ranging accuracy of multiple targets at a long distance, improve the ranging efficiency, and reduce the measurement difficulty. It can combine the movement of the carrier in a dynamic scene to provide a more rapid and accurate measurement of the multi-target tracking distance.

[0010] In an alternative embodiment, the system further includes an image processing module: The image processing module includes an image preprocessing unit and a feature extraction unit. The image preprocessing unit is configured to denoise, enhance, and segment the image frame data to obtain preprocessed image frame data; The feature extraction unit is configured to identify target feature points in the preprocessed image frame data.

[0011] In this way, by processing the image, the image preprocessing denoises, enhances, and segments the image to obtain clearer image data, which is convenient for subsequent image recognition; By identifying the feature points in the image through the feature extraction unit, it is convenient for subsequent target tracking using the image.

[0012] In an alternative embodiment, the system further includes a data storage device: The data storage device is configured to use a sliding window to cache image frame data and ranging data within a preset time period.

[0013] In this way, by using the data storage device to cache the image and ranging data within a preset time period through a sliding window, it is convenient to subsequently determine whether there is a target in the image and obtain the corresponding measurement data, which is convenient for determining the relative distance between the target object and the device.

[0014] In a second aspect, the present invention provides a multi-target tracking and ranging method applied to a multi-target tracking and ranging system according to any one of the first aspect. The system includes an image acquisition device, a ranging device, and a servo turntable. The method includes:

[0015] Obtain image frame data, where the image frame data is an image obtained by using the image acquisition device to acquire an image of a target object;

[0016] Based on the image frame data, obtain ranging data, where the ranging data is the distance measured between the ranging device and the image acquisition device at the same moment for the target object and the ranging device;

[0017] Based on the ranging data, determine the distance between the target object and the ranging device.

[0018] In the present invention, by using the methods of data sliding window caching, searching, and timestamp matching, compared with traditional tracking and ranging, it can effectively improve the ranging accuracy of multiple targets at long distances, improve the ranging efficiency, and reduce the measurement difficulty. It can provide a more rapid and accurate multi-target tracking distance measurement in combination with the movement of the carrier in a dynamic scene.

[0019] In an alternative embodiment, based on the image frame data, obtaining the ranging data includes:

[0020] Sequentially identify and locate each frame of image frame data, and determine whether the target object appears at the optical center of the current frame of image frame data;

[0021] When the target object appears at the optical center of the image frame data of the current frame, the ranging data corresponding to the time stamp is obtained by looking up according to the time stamp corresponding to the image frame data of the current frame.

[0022] In this method, the cached image frames are detected by polling the sliding window to determine whether the position of the target object can cover the optical center. For the image frame data at a certain moment that can cover the optical center, the laser ranging data at the corresponding moment is the relative distance between the current target object and the device.

[0023] In an alternative embodiment, when the target object appears in the image frame data of the current frame and the target object does not cover the optical center of the image frame data of the current frame, the servo turntable is controlled to move based on the position of the target object in the image frame data of the current frame until the target object appears at the optical center of the image frame data of the current frame.

[0024] In this method, when there is a target object in the image frame but there is no image frame in the cached image frames that can cover the optical center, the servo turntable is adjusted in terms of pitch according to the position of the target object on the horizontal line of the optical center, so as to accurately determine the distance between the target object and the system.

[0025] In a third aspect, the present invention provides a multi-target tracking and ranging device, which includes:

[0026] An image acquisition module, configured to acquire image frame data, where the image frame data is an image obtained by using an image acquisition device to perform image acquisition on a target object;

[0027] An image matching module, configured to obtain ranging data based on the image frame data, where the ranging data is the distance measured between the ranging device and the image acquisition device at the same moment for the distance between the target object and the ranging device;

[0028] A distance determination module, configured to determine the distance between the target object and the ranging device based on the ranging data.

[0029] In a fourth aspect, the present invention provides a computer device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to execute the multi-target tracking and ranging method according to the second aspect or any corresponding embodiment thereof.

[0030] In a fifth aspect, the present invention provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to cause a computer to execute the multi-target tracking and ranging method according to the second aspect or any corresponding embodiment thereof.

[0031] Sixth aspect, the present invention provides a computer program product, including computer instructions for causing a computer to execute the multi-target tracking and ranging method according to the second aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0033] Figure 1 FIG. is a schematic structural diagram of a multi-target tracking and ranging system according to an embodiment of the present invention.

[0034] Figure 2 FIG. is a schematic diagram of an application scenario of a multi-target tracking and ranging system device using an optical camera and a laser ranging module according to an embodiment of the present invention.

[0035] Figure 3 FIG. is a schematic flowchart of a multi-target tracking and ranging method according to an embodiment of the present invention.

[0036] Figure 4 FIG. is a working flowchart of a ranging method using an optical camera and a laser ranging module according to an embodiment of the present invention.

[0037] Figure 5 FIG. is a schematic diagram of multi-target tracking and ranging according to an embodiment of the present invention.

[0038] Figure 6 FIG. is a schematic diagram of the determination of the target rectangle covering the optical center and the screening of data measurement data of a ranging method using an optical camera and a laser ranging module according to an embodiment of the present invention.

[0039] Figure 7 FIG. is a block diagram of the structure of a multi-target tracking and ranging device according to an embodiment of the present invention.

[0040] Figure 8 FIG. is a schematic hardware structure diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present invention.

[0042] In the related art, when using an optical camera with a turntable combined with a laser ranging module to measure the distance of a target on a moving platform, the method of tracking the target with the image optical center is used for distance measurement. In actual use, when the stability of the moving platform is poor and the response speed of the turntable is not fast enough, it is very difficult to measure the distance of the target by tracking the image optical center, the complexity of the motion control of the turntable is relatively high, and the camera is occupied by a single task, making it difficult to achieve the tracking and ranging of multiple targets.

[0043] To solve the above problems, an embodiment of the present invention provides a multi-target tracking and ranging system. The computer device in this embodiment is applicable to the usage scenario of tracking and ranging multiple targets during the platform's movement. By providing the multi-target tracking and ranging system of the present invention, by combining a monocular camera and a laser ranging module, an easy-to-implement method is provided, which can not only meet the requirements of the moving platform for multi-target tracking, but also measure the distance of the target of interest, and can significantly reduce the system complexity. Compared with traditional tracking and ranging, it can effectively improve the ranging accuracy of multiple targets at a long distance, improve the ranging efficiency, and reduce the measurement difficulty. It can combine the movement of the carrier in a dynamic scene to provide a faster and more accurate measurement of the multi-target tracking distance.

[0044] According to an embodiment of the present invention, a multi-target tracking and ranging system is provided. Figure 1 The multi-target tracking and ranging system according to an embodiment of the present invention is as Figure 1 shown. The system includes an image acquisition device 201, a ranging device 202, and a servo turntable 203: The image acquisition device 201 is used to acquire an image of a target object to obtain image frame data, where the image frame data includes the target object; the ranging device 202 is used to measure the distance between the target object and the ranging device at the same moment as the image acquisition device 202 to obtain ranging data; the servo turntable 203 is disposed on the moving platform 1 and is used to support the image acquisition device 201 and the ranging device 202; in response to a control instruction, it adjusts the horizontal angle and pitch angle of the image acquisition device so that the target object appears at the image optical center of the image acquisition device 201.

[0045] In an alternative embodiment, the system further includes an image processing module: The image processing module includes an image preprocessing unit and a feature extraction unit. The image preprocessing unit is configured to denoise, enhance, and segment the image frame data to obtain preprocessed image frame data; the feature extraction unit is configured to identify target feature points in the preprocessed image frame data.

[0046] In this manner, by processing the image, the image preprocessing denoises, enhances, and segments the image to obtain clearer image data, which is convenient for subsequent image recognition; the feature extraction unit identifies the feature points in the image, which is convenient for subsequent target tracking using the image.

[0047] In an alternative embodiment, the system further includes a data storage device: The data storage device is configured to use a sliding window to cache image frame data and ranging data within a preset time period.

[0048] In this manner, the data storage device uses a sliding window to cache the image and ranging data within a preset time period, which is convenient for subsequent determination of whether there is a target in the image and obtaining corresponding measurement data, and is convenient for determining the relative distance between the target object and the device.

[0049] In one example, Figure 2 is a schematic diagram of an application scenario of a multi-target tracking and ranging system device using an optical camera and a laser ranging module according to an embodiment of the present invention. As Figure 2 shown, the multi-target tracking and ranging system using an optical camera and a laser ranging module includes: 1. An image acquisition device 201, which is a camera for acquiring a target image, and can be an ordinary digital camera or a high-precision camera with a specific resolution, which is not limited in the present invention. 2. An image processing module, which includes an image preprocessing unit and a feature extraction unit: The image preprocessing unit is configured to denoise, enhance, and segment the image, and the feature extraction unit is configured to identify target feature points in the image to provide a data basis for subsequent distance calculation. 3. A ranging device 202, which measures the distance between the target object and the ranging device 202 through the cooperation of a laser beam emitter and a receiver. 4. A data storage module, which is responsible for caching image frame data and distance measurement data acquired and target-recognized within a certain time period using a sliding window. 5. A servo turntable 203, which is a support module for supporting the camera module and the distance measurement module to achieve high-precision movement in different directions and angles. Among them, 1 is a driving platform, 2 is a combination of the image acquisition device 201, the laser ranging device 202, and the servo turntable 203 integrated with a pan-tilt head, 3, 4, and 5 are different target objects, and 6 is the viewing field angle of the observation range of the image acquisition device 201.

[0050] The multi-target tracking and ranging system provided in this embodiment combines a monocular camera and a laser ranging module to provide an easily implementable method that can not only meet the requirements of a moving platform for multi-target tracking, but also measure the distance to the target of interest, and can significantly reduce the system complexity. Compared with traditional tracking and ranging, it can effectively improve the ranging accuracy of multi-targets at long distances, enhance the ranging efficiency, and reduce the measurement difficulty. It can combine the movement of the carrier in a dynamic scenario to provide a more rapid and accurate measurement of the distance of multi-target tracking.

[0051] According to an embodiment of the present invention, there is provided an embodiment of a multi-target tracking and ranging method. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0052] In this embodiment, a multi-target tracking and ranging method is provided, which can be used in the above multi-target tracking and ranging system. Figure 3 It is a flowchart of the multi-target tracking and ranging method according to an embodiment of the present invention. As Figure 3 shown, the process includes the following steps:

[0053] Step S301, obtain image frame data.

[0054] In an embodiment of the present invention, the image frame data is an image obtained by using an image acquisition device to perform image acquisition on a target object.

[0055] In an example, during the platform driving process, the camera turret is controlled to swing back and forth to achieve large-range and multi-target tracking. At the same time, when a target object that needs to measure the distance appears in the picture, by controlling the horizontal angle and pitch angle of the turret, it is made to appear near the optical center horizontal line. In this way, during the repeated swinging of the camera, the target object will repeatedly pass through the image optical center; a certain angle of circumferential scanning is first performed in the horizontal direction, and then the pitch direction angle is adjusted according to the strategy.

[0056] Step S302, obtain ranging data based on the image frame data.

[0057] In an embodiment of the present invention, the ranging data is obtained by measuring the distance between the ranging device and the target object at the same moment as the image acquisition device.

[0058] In an alternative embodiment, step S302 includes: sequentially identifying and positioning each frame of image frame data, and determining whether the target object appears at the optical center of the current frame of image frame data; when the target object appears at the optical center of the current frame of image frame data, according to the time stamp corresponding to the current frame of image frame data, the ranging data corresponding to the time stamp is found.

[0059] When the target object appears in the image frame data of the current frame and the target object does not cover the optical center of the image frame data of the current frame, based on the position of the target object in the image frame data of the current frame, control the servo turntable to move until the target object appears at the optical center of the image frame data of the current frame.

[0060] In this method, the buffered image frames are detected by polling sliding windows to determine whether the position of the target object can cover the optical center. For the image frame data at a certain moment that can cover the optical center, the laser ranging data at the corresponding moment is the relative distance between the current target object and the device. There is a target object in the image frame, but there is no situation in the buffered image frame that can cover the optical center. According to the position of the target object on the horizontal line of the optical center, the corresponding pitch adjustment of the turntable is performed on the servo turntable to accurately determine the distance between the target object and the system.

[0061] In one example, during the process of using the image for tracking and recognition, the laser ranging module is turned on simultaneously for continuous readings; during the data reading process, the data information of the camera image and the laser ranging module at the same timestamp is numbered and marked and stored. The image frame numbers are F1, F2, F3, ……, F n The distance data measured by the laser ranging module is numbered D1, D2, D3,......, D n The image frame data and the distance measurement data with the same number represent the readings at the same timestamp.

[0062] Step S303, based on the ranging data, determine the distance between the target object and the ranging device.

[0063] In one example, each frame of the image is identified and located in sequence. When the target object appears at the optical center of the image, according to the number F of the current frame X (1 ≤ X ≤ n), find the corresponding laser ranging data D X The distance value reflected by this data is the actual distance of the target object.

[0064] In one embodiment, Figure 4 is a flowchart of a ranging method using an optical camera and a laser ranging module according to an embodiment of the present invention. As Figure 4 shown, the ranging method using an optical camera and a laser ranging module may include:

[0065] 1. Image acquisition during the movement process and acquisition of distance measurement data: By receiving a motion control instruction, high-precision rotation and positioning during the movement of the servo turntable are achieved. During the movement process, the target image is collected in real time by the camera, and at the same time, the distance measurement module is turned on to obtain the measurement result in real time.

[0066] 2. Image preprocessing: Preprocess the acquired images, such as denoising, enhancing, and segmenting them, and use feature extraction algorithms to identify the target boundaries and key points in the image frames, laying a foundation for multi-target tracking.

[0067] 3. Image caching: Allocate a section of dynamic memory to circularly cache the sequence of image data frames recognized during a certain period, and the sequence of distance data measured by the ranging module at the same moment.

[0068] 4. Image matching: Poll the cached image frames using a sliding window to determine whether the position of the target object can cover the optical center. For the image frame data at a certain moment that can cover the optical center, the laser ranging data at the corresponding moment is the relative distance between the current target object and the device. The above steps 1 - 4 are the loop execution processes during the working process.

[0069] Exemplarily, the method for determining whether the position of the target object can cover the optical center in step 4 above can be that when polling the cached image frames through a sliding window, if it is found that there is pixel content of the target object at the center of the image, it is determined that the position of the target object covers the optical center.

[0070] Figure 5 is a schematic diagram of multi-target tracking and ranging according to an embodiment of the present invention, as Figure 5 shown, 7 is the image optical center, that is, the target point of the laser rangefinder; F1 - F n are the continuously acquired image frames in the cache; D1 - D n are the continuously acquired laser rangefinder readings in the cache.

[0071] Specifically, provide an example of an application scenario. The implementation process in this application scenario may include:

[0072] (1) Set the servo turntable to support the circumferential scanning of the camera module and the distance measurement module within the horizontal range of [-60°, 60°] at a rotation speed of 30° per second.

[0073] (2) The camera module acquires target data at a resolution of 1920x1080 and a frame rate of 30 frames per second.

[0074] (3) After the image preprocessing module extracts frames from the video stream data of the camera module at a rate of one frame per 100 milliseconds, preprocesses the extracted image data and performs target recognition.

[0075] (4) Allocate a data cache in the dynamic memory that can accommodate the data for one circumferential scan. For the sequence of image data frames F1, F2, F3,..., F n recognized during one circumferential scan in a certain period, and the distance data numbers D1, D2, D3,..., D n measured by the laser ranging module at the same moment. In this scenario, take n = 40.

[0076] (5) Judging the image data frames F1, F2, F3, ……, F n in sequence through a sliding window to determine whether there is an image frame F X in which the position of the target object can cover the optical center. Here, 1 ≤ X ≤ n. If it exists, obtain the corresponding distance measurement data D X .

[0077] Exemplarily, after identifying the target image, obtain the target rectangle formed by the upper, lower, left, and right boundaries (i.e., y T , y B , x L , x R ) of the target object in the image data frame. If the optical center position (x C , y C ) satisfies the conditions x L < x C < x R and y T < y C < y B , it can be determined that there is a target rectangle of the target image covering the optical center.

[0078] Furthermore, considering the possibility that the target image itself is irregular in shape, even if the target rectangle can cover the optical center, there may still be a possibility that the target image itself does not cover the optical center. At this time, the continuity of sampling and the characteristics of laser ranging can be considered. That is, if the target rectangles in multiple consecutive frames of images all cover the optical center, take the minimum value of the corresponding distance values of the above-mentioned multiple consecutive frames as the distance measurement data D X .

[0079] (6) For the situation where there is a target object in the image frame in the previous step, but there is no target rectangle covering the optical center in the cached image frame, perform corresponding turntable pitch adjustment on the servo turntable according to the position of the optical center horizontal line of the target object, and repeat step (5).

[0080] Here, the multiple numerical values in the above-provided application scenarios are only one of the examples of this embodiment. The specific numerical values can be set according to actual requirements and device parameters, and no specific limitations are made here.

[0081] As an example, for the execution logic of step 5 above, please refer to Figure 6 . Figure 6 is a schematic diagram of determining whether the target rectangle covers the optical center and screening the data measurement data of a ranging method using an optical camera and a laser ranging module according to an embodiment of the present invention, as Figure 6 shown:[[]] Figure 6 It includes target rectangles F A-1 , F A and FA+1 , where the boundary of each target rectangle consists of a left boundary x L , a right boundary x R , an upper boundary y T and a lower boundary y B . The optical center position (x C , y C ) is located at the center of the three target rectangles. If the optical center position (x C , y C ) satisfies the condition x L < x C < x R and y T < y C < y B , it can be determined that there is a target rectangle of the target image covering the optical center.

[0082] Among them, three consecutive target images F A-1 , F A and F A+1 respectively correspond to the laser ranging data D A-1 , D A and D A+1 . If the target rectangles of the above three consecutive target images all cover the optical center, the minimum value of the corresponding distance values of the three frames can be taken as the distance measurement data of the target image.

[0083] The multi-target tracking and ranging method provided in this embodiment adopts the methods of data sliding window caching, searching and timestamp matching. Compared with the traditional tracking and ranging, it can effectively improve the ranging accuracy of multiple targets at long distances, improve the ranging efficiency, and reduce the measurement difficulty. It can combine the movement of the carrier in a dynamic scene to provide a faster and more accurate multi-target tracking distance measurement.

[0084] In this embodiment, a multi-target tracking and ranging device is also provided. This device is used to implement the above-mentioned embodiments and preferred implementation manners, and those that have been described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0085] This embodiment provides a multi-target tracking and ranging device, as Figure 7 shown, including:

[0086] An image acquisition module 701, configured to acquire image frame data, where the image frame data is an image obtained by using an image acquisition device to perform image acquisition on a target object. For details, please refer to Figure 1 the steps of the embodiment shown in

[0087] The image matching module 702 is configured to obtain ranging data based on image frame data, where the ranging data is the distance measured between the ranging device and the image acquisition device at the same moment for the target object and the ranging device. For details, please refer to Figure 3 step S302 of the illustrated embodiment, which will not be elaborated here.

[0088] The distance determination module 703 is configured to determine the distance between the target object and the ranging device based on the ranging data. For details, please refer to Figure 3 step S303 of the illustrated embodiment, which will not be elaborated here.

[0089] In some alternative embodiments, the image matching module 702 includes:

[0090] The image recognition and positioning unit is configured to sequentially recognize and position each frame of image frame data to determine whether the target object appears at the optical center of the current frame of image frame data.

[0091] The ranging data search unit is configured to, when the target object appears at the optical center of the current frame of image frame data, search for the ranging data corresponding to the time stamp according to the time stamp corresponding to the current frame of image frame data.

[0092] In some alternative embodiments, the image matching module 702 includes:

[0093] The servo turntable control unit is configured to, when the target object appears in the current frame of image frame data and the target object does not cover the optical center of the current frame of image frame data, control the servo turntable to move based on the position of the target object in the current frame of image frame data until the target object appears at the optical center of the current frame of image frame data.

[0094] The further function descriptions of the above-mentioned modules and units are the same as those in the corresponding embodiments above, which will not be elaborated here.

[0095] The multi-target tracking and ranging device in this embodiment is presented in the form of functional units. Here, the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and a memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0096] The embodiment of the present invention further provides a computer device having the above Figure 7 illustrated multi-target tracking and ranging device.

[0097] Please refer to Figure 8 , Figure 8 which is a schematic structural diagram of a computer device provided by an alternative embodiment of the present invention. As shown in Figure 8As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting the components, including a high-speed interface and a low-speed interface. Each component communicates with each other using different buses and can be installed on a common motherboard or in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in the memory or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some alternative embodiments, if needed, multiple processors and / or multiple buses can be used with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides part of the necessary operations (such as an array of servers, a set of blade servers, or a multi-processor system). Figure 8 Take one processor 10 as an example in Figure 8 .

[0098] The processor 10 can be a central processing unit, a network processor, or a combination thereof. Among them, the processor 10 can further include a hardware chip. The above hardware chip can be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The above programmable logic device can be a complex programmable logic device, a field programmable gate array, a generic array logic, or any combination thereof.

[0099] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiments.

[0100] The memory 20 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the computer device, etc. In addition, the memory 20 can include a high-speed random access memory and can also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some alternative embodiments, the memory 20 can optionally include a memory remotely set relative to the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0101] The memory 20 can include a volatile memory, such as a random access memory; the memory can also include a non-volatile memory, such as a flash memory, a hard disk, or a solid-state drive; the memory 20 can also include a combination of the above types of memories.

[0102] The computer device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30, and the output device 40 may be connected by a bus or other means. Figure 8 Taking the connection by bus as an example.

[0103] The input device 30 can receive input digital or character information, and generate key signal inputs related to the user settings and function controls of the computer device, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointing stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED), and a haptic feedback device (e.g., a vibration motor), etc. The above display device includes, but is not limited to, a liquid crystal display, a light-emitting diode, a display, and a plasma display. In some alternative embodiments, the display device may be a touch screen.

[0104] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or be implemented as computer code that can be recorded on a storage medium, or be implemented as computer code that is originally stored in a remote storage medium or a non-transitory machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored as such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid-state drive, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by the computer, the processor, or the hardware, the method shown in the above embodiment is implemented.

[0105] A part of the present invention can be applied as a computer program product, such as computer program instructions. When executed by a computer, through the operation of the computer, the methods and / or technical solutions according to the present invention can be called or provided. Those skilled in the art should be able to understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Herein, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible by the computer.

[0106] Although embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A multi-target tracking and ranging system, characterized in that The system includes an image acquisition device, a ranging device, and a servo turntable: The image acquisition device is configured to acquire image frame data of a target object, where the image frame data includes the target object; The ranging device is configured to measure the distance between the target object and the ranging device at the same time as the image acquisition device, to obtain ranging data; The servo turntable is disposed on a traveling platform and is configured to support the image acquisition device and the ranging device; in response to a control instruction, adjust the horizontal angle and the pitch angle of the image acquisition device so that the target object appears at the image optical center of the image acquisition device.

2. The system according to claim 1, wherein The system further includes an image processing module: The image processing module includes an image preprocessing unit and a feature extraction unit. The image preprocessing unit is configured to perform denoising, enhancement, and segmentation on the image frame data to obtain preprocessed image frame data; the feature extraction unit is configured to identify target feature points in the preprocessed image frame data.

3. The system according to claim 1, characterized in that, The system further includes a data storage device: The data storage device is configured to use a sliding window to cache the image frame data and the ranging data within a preset time period.

4. A multi-target tracking and ranging method, characterized in that, Applied to the multi-target tracking and ranging system according to any one of claims 1-3, the system includes an image acquisition device, a ranging device, and a servo turntable, and the method includes: Obtain image frame data, where the image frame data is an image obtained by using an image acquisition device to acquire an image of a target object; Based on the image frame data, obtain ranging data, where the ranging data is the distance measured between the target object and the ranging device by the ranging device and the image acquisition device at the same time; Based on the ranging data, determine the distance between the target object and the ranging device.

5. The method according to claim 4, characterized in that, The obtaining the ranging data based on the image frame data includes: Sequentially identify and locate each frame of image frame data, and determine whether the target object appears at the optical center of the current frame of image frame data; When the target object appears at the optical center of the current frame of image frame data, look up the ranging data corresponding to the time stamp according to the time stamp corresponding to the current frame of image frame data.

6. The method according to claim 5, wherein When the target object appears in the current frame of image frame data and the target object does not cover the optical center of the current frame of image frame data, control the servo turntable to move based on the position of the target object in the current frame of image frame data until the target object appears at the optical center of the current frame of image frame data.

7. A multi-target tracking and ranging device, characterized in that, The apparatus includes: An image acquisition module, configured to acquire image frame data, where the image frame data is an image obtained by using an image acquisition device to acquire an image of a target object; An image matching module, configured to obtain ranging data based on the image frame data, where the ranging data is the distance measured between the target object and the ranging device by the ranging device and the image acquisition device at the same time; A distance determination module, configured to determine the distance between the target object and the ranging device based on the ranging data.

8. A computer device, characterized in that, Including: A memory and a processor, which are communicatively connected to each other. Computer instructions are stored in the memory, and the processor executes the computer instructions to perform the multi-target tracking and ranging method according to any one of claims 4 to 6.

9. A computer-readable storage medium, characterized in that, Computer instructions are stored on the computer-readable storage medium, and the computer instructions are used to cause a computer to execute the multi-target tracking and ranging method according to any one of claims 4 to 6.

10. A computer program product, characterized in that, It includes computer instructions, and the computer instructions are used to cause a computer to execute the multi-target tracking and ranging method according to any one of claims 4 to 6.

Images