Image feature acquisition method and device, unmanned equipment and storage medium

Abstract

The invention discloses an image feature acquisition method and device, unmanned equipment and a storage medium, and relates to the technical field of unmanned equipment. The method is applied to the unmanned equipment, the unmanned equipment comprises image acquisition equipment, a digital signal processor and a central processor, the digital signal processor is connected with the image acquisition equipment and the central processor, and the method comprises the following steps: acquiring a first image through the image acquisition equipment, and transmitting the first image to the digital signal processor; performing block processing on the first image through a digital signal processor to obtain a plurality of image blocks, extracting feature information of each image block, generating feature information of the first image according to the feature information of each image block, and transmitting the feature information of the first image to a central processing unit; and obtaining feature information of the first image transmitted by the digital signal processor through the central processing unit. By means of the technical means, the problem that in the prior art, when unmanned equipment processes multiple tasks at the same time, the image feature extraction efficiency is affected is solved.

Description

Technical Field

[0001] The present application relates to the technical field of unmanned equipment, and in particular to an image feature acquisition method, device, unmanned equipment and storage medium. Background Art

[0002] The Scale Invariant Feature Transform (SIFT) algorithm can convert image data into scale-invariant coordinates relative to local image characteristics and perform feature description. The SIFT algorithm can effectively extract a large number of features from an image. These features are highly unique, which allows a single feature to be matched with a large feature database with a high probability, providing a data basis for the recognition of objects and scenes. Due to the stability, uniqueness, anti-interference ability and high efficiency of the features extracted by the SIFT algorithm, it is widely used in various operating scenarios in the field of unmanned equipment technology, such as three-dimensional reconstruction and dense point cloud reconstruction based on image features extracted by the SIFT algorithm.

[0003] In the prior art, unmanned equipment runs the SIFT algorithm through the central processor to extract image features. When the unmanned equipment processes multiple tasks at the same time, the efficiency of extracting image features will decrease, indirectly affecting the operating efficiency of the current unmanned equipment. Summary of the invention

[0004] The present application provides an image feature acquisition method, device, unmanned equipment and storage medium, which solves the problem in the prior art that the efficiency of image feature extraction will be affected when the unmanned equipment processes multiple tasks at the same time, improves the efficiency of image feature extraction, and further improves the operating efficiency of the unmanned equipment.

[0005] In a first aspect, the present application provides an image feature acquisition method, which is applied to an unmanned device, wherein the unmanned device includes an image acquisition device, a digital signal processor, and a central processing unit, wherein the digital signal processor is connected to the image acquisition device and the central processing unit, and the method includes:

[0006] Capturing a first image by means of the image acquisition device, and transmitting the first image to the digital signal processor;

[0007] Performing block processing on the first image by the digital signal processor to obtain a plurality of image blocks, extracting feature information of each of the image blocks, generating feature information of the first image according to the feature information of each of the image blocks, and transmitting the feature information of the first image to the central processor;

[0008] The characteristic information of the first image transmitted by the digital signal processor is acquired through the central processing unit.

[0009] In a second aspect, the present application provides an image feature acquisition device, which is applied to unmanned equipment, wherein the unmanned equipment includes an image acquisition device, a digital signal processor, and a central processing unit, wherein the digital signal processor is connected to the image acquisition device and the central processing unit, and the device includes:

[0010] an image acquisition module, configured to acquire a first image through the image acquisition device, and transmit the first image to the digital signal processor;

[0011] an image processing module, configured to perform block processing on the first image through the digital signal processor to obtain a plurality of image blocks, extract feature information of each of the image blocks, generate feature information of the first image according to the feature information of each of the image blocks, and transmit the feature information of the first image to the central processing unit;

[0012] The feature acquisition module is configured to acquire feature information of the first image transmitted by the digital signal processor through the central processing unit.

[0013] In a third aspect, the present application provides an unmanned device, including:

[0014] One or more processors; a memory storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the image feature acquisition method as described in the first aspect.

[0015] In a fourth aspect, the present application provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the image feature acquisition method as described in the first aspect.

[0016] In the present application, a first image is captured by an image acquisition device, and the first image is transmitted to a digital signal processor; the first image is processed by a digital signal processor in blocks to obtain a plurality of image blocks, feature information of each image block is extracted, feature information of the first image is generated according to the feature information of each image block, and the feature information of the first image is transmitted to a central processing unit; the feature information of the first image transmitted by the digital signal processor is obtained by the central processing unit. Through the above technical means, the SIFT algorithm is run on the digital signal processor to quickly obtain the feature point information of the image, thereby reducing the calculation pressure of the central processing unit. Moreover, the digital signal processor and the central processing unit operate independently, and the central processing unit will not affect the efficiency of the digital signal processor in running the SIFT algorithm when processing tasks, which solves the problem in the prior art that the extraction efficiency of image features will be affected when unmanned equipment processes multiple tasks at the same time, improves the extraction efficiency of image features, and thus improves the operation efficiency of unmanned equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a structural block diagram of an unmanned device provided in an embodiment of the present application;

[0018] Figure 2 is a flow chart of an image feature acquisition method provided in an embodiment of the present application;

[0019] Figure 3 is a schematic diagram of a process of extracting features from a first image by a digital signal processor provided in an embodiment of the present application;

[0020] Figure 4 is a flow chart of a digital signal processor provided in an embodiment of the present application extracting image features through multiple core processors;

[0021] Figure 5 is a flowchart of multiple core processors executing image processing algorithm tasks in parallel provided by an embodiment of the present application;

[0022] Figure 6 This is a flow chart of extracting features from an image block using a SIFT algorithm provided in an embodiment of the present application;

[0023] Figure 7 is a structural schematic diagram of an image feature acquisition device provided in an embodiment of the present application;

[0024] Figure 8 It is a structural schematic diagram of an unmanned device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0026] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0027] The image feature acquisition method provided in this embodiment can be executed by an unmanned device, which can be implemented by software and / or hardware, and can be composed of two or more physical entities or one physical entity. The unmanned device refers to a flying device or ground platform that operates according to remote control instructions or preset instructions, such as a drone and an unmanned vehicle.

[0028] For ease of understanding, this embodiment is described by taking an unmanned device as an example of the main body for executing the image feature acquisition method.

[0029] In one embodiment, the unmanned device includes a central processing unit (CPU) and an image acquisition device. When the unmanned device performs a three-dimensional reconstruction task or a dense point cloud reconstruction task of a surveying area, the image acquisition device shoots the surveying area to obtain an image containing the surveying area, and transmits the image to the central processing unit. The central processing unit runs the SIFT algorithm to extract the feature point information of the image, and constructs a three-dimensional map or a dense point cloud of the surveying area based on the feature point information. Due to the limited computing power resources of the central processing unit, when the unmanned device processes multiple tasks simultaneously through the central processing unit, the extraction efficiency of the image feature point information will be reduced. The feature point information is one of the foundations of three-dimensional reconstruction and dense point cloud reconstruction, so the extraction efficiency of the feature point information will indirectly affect the overall solution efficiency of a single flight. In addition, when the central processing unit extracts the feature point information, it needs to load the complete image of the original image resolution into the memory. This step will increase the input and output data volume from the disk to the memory. The read and write operations of a single thread will affect the calculation efficiency of other threads, thereby reducing the overall solution efficiency.

[0030] In order to solve the above problems existing in the prior art, this embodiment provides an image feature acquisition method to improve the efficiency of image feature extraction and thus improve the overall solution efficiency.

[0031] In one embodiment, Figure 1 is a structural block diagram of the unmanned equipment provided in the embodiment of the present application. Figure 1As shown, the unmanned equipment includes a central processing unit, a digital signal processor (DSP), an image acquisition device and a camera, and the digital signal processor connects the image acquisition device and the central processing unit. The image acquisition device is used to acquire images of the surveying and mapping area, the digital signal processor is used to extract feature point information in the image, and the central processing unit is used to perform three-dimensional reconstruction and dense point cloud reconstruction of the surveying and mapping area based on the feature point information. This embodiment realizes the image feature acquisition method through the mutual cooperation between the image acquisition device, the digital signal processor and the central processing unit.

[0032] Figure 2 A flowchart of an image feature acquisition method provided by an embodiment of the present application is given. Figure 2 , the image feature acquisition method specifically includes:

[0033] S110, capturing a first image through an image capturing device, and transmitting the first image to a digital signal processor.

[0034] Exemplarily, the unmanned device controls the unmanned device to fly above the surveying and mapping area based on a preset flight route, and when the unmanned device flies to a shooting point corresponding to the surveying and mapping area, the image acquisition device takes a picture of the surveying and mapping area to acquire a first image including the surveying and mapping area. The image acquisition device may be a camera or other imaging sensor.

[0035] After acquiring the first image, the image acquisition device transmits the first image to the digital signal processor through a communication connection with the digital signal processor, so that the digital signal processor runs a SIFT algorithm to extract feature point information of the first image.

[0036] S120, performing block processing on the first image through a digital signal processor to obtain a plurality of image blocks, extracting feature information of each image block, generating feature information of the first image according to the feature information of each image block, and transmitting the feature information of the first image to a central processing unit.

[0037] The feature information is the feature point information extracted from the image by the SIFT algorithm. Figure 3 FIG. 1 is a flow chart of a digital signal processor according to an embodiment of the present application performing feature extraction on a first image. Figure 3As shown, after receiving the first image 11 transmitted by the image acquisition device, the digital signal processor stores the first image 11 in the off-chip memory. The digital signal processor divides the first image 11 in the off-chip memory into multiple image blocks 12, and moves the image blocks 12 to the on-chip memory of the digital signal processor through DMA (Direct Memory Access, grouped data transmission mode). The digital signal processor extracts features from the image blocks 12 in the on-chip memory through the core processor to obtain feature point information 15 in the corresponding image blocks 12, and moves the feature point information to the off-chip memory through DMA. The core processor cyclically processes each image block 12 divided from the first image 11, and after the feature point information 15 of all image blocks 12 is moved to the off-chip memory, each feature point information 15 is synthesized into feature point information 16 of the first image according to the position information of the image blocks 12 corresponding to each feature point information 15 in the first image 11. Further, the digital core processor can transmit the feature point information 16 of the first image to the central processor through a communication connection with the central processor.

[0038] It should be noted that since the digital signal processor has a small amount of on-chip memory, most of which are in KB, and the data volume of the first image is large, the digital signal processor needs to perform block processing on the first image before extracting features from the first image, so as to divide the first image into multiple image blocks and store each image block in the on-chip memory. The digital signal processor can move the image blocks to the on-chip memory of the digital signal processor through DMA (Direct Memory Access, grouped data transfer mode). Since the first image is divided into multiple image blocks and the image blocks are moved to the on-chip memory through DMA, a large amount of input and output data from the disk to the memory can be reduced, and the impact of the read and write operations of a single thread on the computing efficiency of other threads can be reduced, which is conducive to improving the efficiency of feature extraction.

[0039] When the digital signal processor is a single-core, the SIFT algorithm can be run through a single thread to extract the feature point information in each image block. Exemplarily, the single core processor of the digital signal processor moves each image block to the on-chip memory through DMA, reads the image block in the on-chip memory and extracts the feature point information in the image block through the SIFT algorithm, and moves the extracted feature point information to the off-chip memory of the digital signal processor through DMA. After extracting the feature point information of all image blocks, the digital signal processor summarizes the feature point information in the off-chip memory to generate the feature point information of the first image, and transmits the feature point information of the first image to the central processing unit through the communication connection with the central processing unit.

[0040] When the digital signal processor is multi-core, that is, the digital signal processor includes multiple core processors, the SIFT algorithm can be run in parallel through multiple threads to extract feature point information in multiple image blocks in parallel, thereby effectively improving the efficiency of image feature extraction. Figure 4 FIG. 1 is a flow chart of a digital signal processor according to an embodiment of the present application extracting image features through multiple core processors. Figure 4 As shown, the steps of extracting image features by the digital signal processor through multiple core processors specifically include S1201-S1202:

[0041] S1201. Divide a first image into a plurality of image blocks of the same size, and evenly distribute the image blocks to the core processors according to the number of the image blocks.

[0042] Exemplarily, the digital signal processor may preset the size of the image blocks, evenly divide the first image into a plurality of image blocks of the same size, and determine the number of image blocks into which the first image can be divided according to the size of the image blocks and the size of the first image. In this embodiment, the digital signal processor may divide the first image into a plurality of image blocks of the same size by an edge filling algorithm, so that the edges of adjacent image blocks overlap. Figure 3 , the image block 12 is composed of a non-overlapping area 13 and an overlapping area 14. The non-overlapping area 13 is the area where the image block does not overlap with the adjacent image block, and the overlapping area 14 is the area where the image block overlaps with the adjacent image. For example, the preset image block size is W*H, and the edge filling size is X. Then, the pixel area formed by the pixels (0,0), (W,0), (0,H) and (W,H) in the first image can be used as an image block A, and the pixel area formed by the pixels (WX,0), (2W-X,0), (WX,H) and (2W-X,H) in the first image can be used as an image block B. The right side of the image block A is adjacent to the left side of the image block B and the edges overlap.

[0043] After determining the number of image blocks into which the first image is divided, the digital signal processor can evenly distribute the image blocks to each core processor according to the number of core processors so as to balance the computing pressure of each core processor and shorten the time difference between each core processor in completing the corresponding image block feature extraction task, thereby shortening the feature extraction time of the first image and improving the image feature extraction efficiency.

[0044] In this embodiment, the digital signal processor can determine the number of image blocks allocated to each core processor based on the number of each image block; and the corresponding number of image blocks are moved to the on-chip memory of the digital signal processor by the core processor based on the grouped data transmission method. This embodiment is described by taking the digital signal processor including two core processors as an example. If the first image is divided into 2N image blocks, the N image blocks are allocated to each core processor. The digital signal processor instantiates the image processing algorithm module into two objects, and the two objects are respectively run by the two core processors through threads so that the two core processors execute the image processing algorithm task of extracting feature point information of N image blocks in parallel. Among them, the image processing algorithm module includes a processing unit for moving image blocks to on-chip memory, extracting feature point information of image blocks, and moving feature point information to off-chip memory. Figure 5 1 is a flowchart of multiple core processors executing image processing algorithm tasks in parallel according to an embodiment of the present application. Figure 5 As shown, when the two core processors respectively run the instantiation objects of the image processing algorithm module through the corresponding threads, the two instantiation objects of the image processing algorithm module run simultaneously. The core processor can move an image block from the off-chip memory to the on-chip memory through DMA, and extract the feature point information of the image block through the SIFT algorithm, and move the extracted feature point information to the off-chip memory. If the core processor has not yet processed the corresponding N image blocks assigned, the new image block is moved from the off-chip memory to the on-chip memory and the feature point information is extracted until the core processor completes the feature extraction of the N image blocks. It can be understood that the core processor does not move all N image blocks to the on-chip memory at one time, but first moves an image block and extracts the feature point information of the image block, and then moves the next image block to the on-chip memory until the core processor moves N image blocks to the on-chip memory and extracts the feature point information of the N image blocks.

[0045] S1202: Extract feature information of corresponding allocated image blocks in parallel through each core processor.

[0046] refer to Figure 5 , the two core processors respectively run the instantiation objects of the image processing algorithm module in parallel through corresponding threads to extract the feature point information of the corresponding image blocks moved from the off-chip memory to the on-chip memory in parallel. It can be understood that the image blocks allocated to the core processors in this embodiment are the image blocks that the core processors actively move from the off-chip memory to the on-chip memory.

[0047] In this embodiment, after the core processor transfers the image blocks in the off-chip memory to the on-chip memory, the feature information of the image blocks in the on-chip memory can be extracted by the scale-invariant feature transformation algorithm; the feature information of the image blocks is transferred to the off-chip memory of the digital signal processor by group data transmission. Exemplarily, the core processor runs the SIFT algorithm to extract features from the image blocks in the on-chip memory, and the extracted feature point information is stored in the on-chip memory. In order to transfer the image blocks to be processed later to the on-chip memory, the space in the on-chip memory needs to be vacated. Therefore, when processing the next image block, the feature point information currently extracted can be transferred from the on-chip memory to the off-chip memory by DMA. The core processor can determine whether the number of image blocks transferred to the on-chip memory is equal to the number of corresponding allocated image blocks. If the number of image blocks transferred to the on-chip memory is less than the number of corresponding allocated image blocks, it indicates that the core processor has not yet completed the feature extraction of all allocated image blocks, and then continues to obtain image blocks from the off-chip memory and transfer them to the on-chip memory and runs the SIFT algorithm to extract features from the image blocks. If the number of image blocks transferred to the on-chip memory is equal to the number of corresponding allocated image blocks, it can be determined that the core processor has completed the feature extraction of all allocated image blocks, and then enters the buffer state to wait for other core processors to complete the feature extraction of all allocated image blocks.

[0048] When each core processor completes feature extraction of a corresponding number of image blocks, the feature information in the off-chip memory is read, and feature information of the first image is generated based on the read feature information. Exemplarily, when each core processor in the digital core processor completes feature extraction of a corresponding number of image blocks, it indicates that the feature point information currently stored in the off-chip memory is feature point information of all image blocks of the first image, and therefore, based on the position information of the image blocks corresponding to each feature point information in the current off-chip memory, each feature point information can be spliced ​​to obtain the feature point information of the first image.

[0049] In one embodiment, Figure 6 FIG. 1 is a flowchart of extracting features from an image block using the SIFT algorithm provided in an embodiment of the present application. Figure 6 As shown, the step of extracting features from the image block using the SIFT algorithm specifically includes S12021-S12025:

[0050] S12021. Construct a Gaussian pyramid model of the image block, and construct a Gaussian difference pyramid model based on the Gaussian pyramid model.

[0051] Exemplarily, the image block is converted from a color image to a grayscale image, and the grayscale image is converted from a uchar format to a short format to store more data information. After doubling the grayscale image, a Gaussian pyramid model is performed on the basis of the enlarged grayscale image. Gaussian blur is performed on the enlarged grayscale image multiple times, and the first octave of the Gaussian pyramid model is constructed by combining several Gaussian blurred grayscale images. The most blurred grayscale image in the octave is downsampled to reduce the image size, and the reduced grayscale image is the initial image of the second octave. Gaussian blur is performed on the initial image multiple times, and the second octave is constructed by combining several Gaussian blurred grayscale images, and so on, until all octaves of the Gaussian pyramid model are constructed.

[0052] Furthermore, a Gaussian difference pyramid model is obtained by performing a difference calculation on two adjacent grayscale images of the same octave in the Gaussian pyramid model.

[0053] It should be noted that in addition to first dividing the first image into blocks and then generating a Gaussian pyramid model based on the image blocks, the digital signal processor can also first generate a Gaussian pyramid model of the first image, then divide the Gaussian pyramid model of the first image into blocks, and move the divided Gaussian pyramid model to the on-chip memory for subsequent feature extraction processing.

[0054] S12022. Perform extreme point detection on the Gaussian difference pyramid model, and determine the detected extreme points as key points of the image block.

[0055] For example, in the Gaussian difference pyramid model, find the position p corresponding to the maximum or minimum Gaussian difference value within the 3*3*3 neighborhood. * =(x, y, σ), the position p * =(x, y, σ) is determined as the key point of the image block.

[0056] S12023. Position the key points to determine their scale and position, and assign angles to the key points to determine their directions.

[0057] Exemplarily, the discrete positions and scales of the key points are interpolated by a second-order Taylor formula expansion to accurately locate the key points. The second-order Taylor formula expansion of the Gaussian difference function D(x, y, σ) in the scale space is:

[0058]

[0059] Among them, D and its first-order partial derivative and second-order partial derivative are calculated at the sampling point. x = (x, y, σ) T is the offset from the sampling point. The offset of the true extreme value can be determined, thus achieving accurate positioning of key points.

[0060] Further, the Gaussian weighted-directional histogram of the local area around the key point position is calculated, where the histogram has 36 columns covering a 360-degree range, and each column represents a 10° range. The window radius of the local area is 3×1.5σ, that is, 4.5σ, the window size is (2*4.5σ+1)*(2*4.5σ+1), and the standard deviation of the Gaussian weight in the window of the local area is 1.5σ. Accordingly, the calculation formula for the angle and amplitude of each point in the window of the local area is as follows:

[0061]

[0062]

[0063] The peak of the histogram, that is, the highest column, represents the direction that is the main direction of the image gradient within the key point area, but the angle represented by the column is a range. The discrete histogram can be interpolated and fitted to obtain a more accurate direction angle value.

[0064] It should be noted that, in addition to first dividing the first image into blocks, or dividing the Gaussian pyramid model of the first image into blocks, and then performing subsequent operations of the SIFT algorithm, the digital signal processor can also first perform the operations of constructing the Gaussian pyramid model and determining the key point information in the SIFT algorithm on the first image, and then divide the key point information into blocks, and move the divided key point information to the on-chip memory for subsequent feature extraction operations. Among them, the key point information includes the position, scale, direction and response value of the key point.

[0065] S12024. Sort the key points in descending order according to their response values, and obtain a preset number of target key points that are ranked high.

[0066] For example, in the above process, each key point is assigned a response value, and the response values ​​of each key point are sorted in descending order, and the first K key points are selected as the input of the descriptor calculation module. The descriptor calculation module is used to determine the feature vector of the key point, and the target key point is the key point input to the descriptor calculation module.

[0067] S12025. Determine the feature vector of each target key point according to the position, scale and direction of each target key point, and generate feature information of the image block according to each target key point.

[0068] For example, the radius of the circular area around the key point is determined, and the angle and amplitude of the gradient of each point in the circular area are calculated. By trilinear interpolation, the amplitude and direction of the gradient of each point in the circular area before rotation are calculated, and then the d*d array direction histogram after rotation is interpolated by trilinear interpolation. p ), perform rotation, scaling and translation operations, and map it to a d*d array direction histogram. Each grid in the array generates an n-dimensional direction histogram, that is, the dimension of the feature vector of the key point is d*d*n.

[0069] After extracting the feature vector of each target key point, the target key point and the corresponding feature vector are determined as the feature point information of the corresponding image block.

[0070] S130. Acquire, by the central processing unit, feature information of the first image transmitted by the digital signal processor.

[0071] Exemplarily, the central processing unit obtains feature point information of the first image transmitted by the digital signal processor through a communication connection with the digital signal processor, and constructs a three-dimensional map or a dense point cloud of the surveying area based on the feature point information of the first image.

[0072] It should be noted that the unmanned equipment can not only use the feature point information of the first image to construct a three-dimensional map or a dense point cloud, but can also be used to process other work tasks. Therefore, this embodiment does not limit the use of the feature point information of the first image, but only limits its acquisition method.

[0073] In summary, the image feature acquisition method provided in the embodiment of the present application acquires a first image through an image acquisition device, and transmits the first image to a digital signal processor; the first image is processed into blocks by the digital signal processor to obtain multiple image blocks, and feature information of each image block is extracted, and feature information of the first image is generated according to the feature information of each image block, and the feature information of the first image is transmitted to the central processing unit; the feature information of the first image transmitted by the digital signal processor is acquired by the central processing unit. Through the above-mentioned technical means, the SIFT algorithm is run on the digital signal processor to quickly acquire the feature point information of the image, thereby reducing the calculation pressure of the central processing unit. Moreover, the digital signal processor and the central processing unit operate independently, and the central processing unit will not affect the efficiency of the digital signal processor in running the SIFT algorithm when processing tasks, which solves the problem in the prior art that the extraction efficiency of image features will be affected when unmanned equipment processes multiple tasks at the same time, improves the extraction efficiency of image features, and thus improves the operation efficiency of unmanned equipment.

[0074] Based on the above embodiments, Figure 7 This is a schematic diagram of the structure of an image feature acquisition device provided in an embodiment of the present application. Figure 7The image feature acquisition device provided in this embodiment specifically includes: an image acquisition module 21, an image processing module 22 and a feature acquisition module 23.

[0075] The image acquisition module 21 is configured to acquire a first image through the image acquisition device and transmit the first image to the digital signal processor;

[0076] The image processing module 22 is configured to perform block processing on the first image through the digital signal processor to obtain a plurality of image blocks, extract feature information of each of the image blocks, generate feature information of the first image according to the feature information of each of the image blocks, and transmit the feature information of the first image to the central processor;

[0077] The feature acquisition module 23 is configured to acquire feature information of the first image transmitted by the digital signal processor through the central processor.

[0078] Based on the above embodiment, the image processing module 22 includes: an allocation submodule, configured to divide the first image into multiple image blocks of the same size, and evenly allocate each image block to each core processor according to the number of the image blocks; a feature extraction submodule, configured to extract feature information of the corresponding allocated image blocks in parallel through each core processor.

[0079] Based on the above embodiment, the allocation submodule includes: an allocation quantity determination unit, configured to determine the number of image blocks allocated to each of the core processors according to the number of each of the image blocks; and a first transport unit, configured to transport the corresponding number of image blocks to the on-chip memory of the digital signal processor through the core processor based on a grouped data transmission method.

[0080] Based on the above embodiment, the feature extraction submodule includes: a feature extraction unit, configured to extract feature information of the image block in the on-chip memory through a scale-invariant feature transformation algorithm; and a second transport unit, configured to transport the feature information of the image block to the off-chip memory of the digital signal processor through a grouped data transmission method.

[0081] Based on the above embodiment, the image processing module 22 includes: a feature information generation submodule, which is configured to read the feature information in the off-chip memory when each of the core processors completes the feature extraction of a corresponding number of image blocks, and generate feature information of the first image based on the read feature information.

[0082] Based on the above embodiment, the allocation submodule includes: an image block division unit configured to divide the first image into a plurality of image blocks of the same size by an edge filling algorithm so that edges of adjacent image blocks overlap.

[0083] On the basis of the above embodiment, the feature extraction unit includes: a model construction subunit, configured to construct a Gaussian pyramid model of the image block, and construct a Gaussian difference pyramid model according to the Gaussian pyramid model; a key point detection unit, configured to perform extreme point detection on the Gaussian difference pyramid model, and determine the detected extreme points as key points of the image block; a key point information determination unit, configured to locate the key points to determine the scale and position of the key points, and to perform angle allocation on the key points to determine the direction of the key points; a target key point determination unit, configured to sort the key points in descending order according to the response values ​​of the key points, and obtain a preset number of target key points with top sorting; a feature information determination unit, configured to determine the feature vector of the target key point according to the position, scale and direction of each of the target key points, and generate feature information of the image block according to each of the target key points.

[0084] As mentioned above, the image feature acquisition device provided in the embodiment of the present application acquires a first image through an image acquisition device, and transmits the first image to a digital signal processor; the first image is processed into blocks by the digital signal processor to obtain multiple image blocks, and feature information of each image block is extracted, and feature information of the first image is generated according to the feature information of each image block, and the feature information of the first image is transmitted to the central processing unit; the feature information of the first image transmitted by the digital signal processor is acquired by the central processing unit. Through the above technical means, the SIFT algorithm is run on the digital signal processor to quickly acquire the feature point information of the image, thereby reducing the calculation pressure of the central processing unit. Moreover, the digital signal processor and the central processing unit operate independently, and the central processing unit will not affect the efficiency of the digital signal processor in running the SIFT algorithm when processing tasks, which solves the problem in the prior art that the extraction efficiency of image features will be affected when unmanned equipment processes multiple tasks at the same time, improves the extraction efficiency of image features, and thus improves the operation efficiency of unmanned equipment.

[0085] The image feature acquisition device provided in the embodiment of the present application can be used to execute the image feature acquisition method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0086] Figure 8 This is a schematic diagram of the structure of an unmanned device provided in an embodiment of the present application, with reference to Figure 8The unmanned device includes: a processor 31, a memory 32, a communication device 33, an input device 34 and an output device 35. The number of processors 31 in the unmanned device can be one or more, and the number of memories 32 in the unmanned device can be one or more. The processor 31, memory 32, communication device 33, input device 34 and output device 35 of the unmanned device can be connected through a bus or other methods.

[0087] The memory 32, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the image feature acquisition method of any embodiment of the present application (for example, the image acquisition module 21, the image processing module 22 and the feature acquisition module 23 in the image feature acquisition device). The memory 32 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the device, etc. In addition, the memory 32 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0088] The communication device 33 is used for data transmission.

[0089] The processor 31 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 32, that is, implements the above-mentioned image feature acquisition method.

[0090] The input device 34 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 35 may include a display device such as a display screen.

[0091] The unmanned equipment provided above can be used to execute the image feature acquisition method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0092] An embodiment of the present application also provides a storage medium containing computer executable instructions. When the computer executable instructions are executed by a computer processor, they are used to execute an image feature acquisition method. The image feature acquisition method includes: acquiring a first image through an image acquisition device, and transmitting the first image to a digital signal processor; performing block processing on the first image through the digital signal processor to obtain multiple image blocks, extracting feature information of each image block, generating feature information of the first image based on the feature information of each image block, and transmitting the feature information of the first image to a central processing unit; and acquiring, through the central processing unit, the feature information of the first image transmitted by the digital signal processor.

[0093] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROM, floppy disk or tape device; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc. Storage media may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system, which is connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media residing in different locations (for example, in different computer systems connected by a network). The storage medium may store program instructions (for example, embodied as a computer program) that can be executed by one or more processors.

[0094] Of course, the storage medium containing computer executable instructions provided in the embodiment of the present application, whose computer executable instructions are not limited to the above-mentioned image feature acquisition method, can also execute related operations in the image feature acquisition method provided in any embodiment of the present application.

[0095] The image feature acquisition device, storage medium and unmanned equipment provided in the above embodiments can execute the image feature acquisition method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the image feature acquisition method provided in any embodiment of the present application.

[0096] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the protection scope of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A method for acquiring image features, characterized in that: Applied to unmanned equipment, the unmanned equipment includes an image acquisition device, a digital signal processor and a central processing unit, the digital signal processor is connected to the image acquisition device and the central processing unit, and the method includes: Capturing a first image by means of the image acquisition device, and transmitting the first image to the digital signal processor; Performing block processing on the first image by the digital signal processor to obtain a plurality of image blocks, extracting feature information of each of the image blocks, generating feature information of the first image according to the feature information of each of the image blocks, and transmitting the feature information of the first image to the central processor; The characteristic information of the first image transmitted by the digital signal processor is acquired through the central processing unit.

2. The image feature acquisition method according to claim 1, characterized in that: The digital signal processor includes at least two core processors; accordingly, the first image is processed by the digital signal processor in blocks to obtain a plurality of image blocks, including: Dividing the first image into a plurality of image blocks of the same size, and evenly allocating each of the image blocks to each of the core processors according to the number of the image blocks; The core processors extract the feature information of the corresponding allocated image blocks in parallel.

3. The image feature acquisition method according to claim 2, characterized in that: The evenly distributing each of the image blocks to each of the core processors according to the number of the image blocks comprises: Determining the number of image blocks allocated to each of the core processors according to the number of each of the image blocks; The core processor transfers a corresponding number of image blocks to the on-chip memory of the digital signal processor based on a grouped data transmission method.

4. The image feature acquisition method according to claim 3, characterized in that: The extracting the feature information of the corresponding allocated image blocks in parallel by each of the core processors includes: Extracting feature information of the image block in the on-chip memory by using a scale-invariant feature transformation algorithm; The characteristic information of the image block is transferred to the off-chip memory of the digital signal processor by means of group data transmission.

5. The image feature acquisition method according to claim 4, characterized in that: The step of generating the feature information of the first image according to the feature information of each of the image blocks includes: When each of the core processors completes feature extraction of a corresponding number of image blocks, the feature information in the off-chip memory is read, and feature information of the first image is generated according to the read feature information.

6. The image feature acquisition method according to claim 2, characterized in that: The step of dividing the first image into a plurality of image blocks of the same size comprises: The first image is divided into a plurality of image blocks of the same size by using an edge filling algorithm, so that edges of adjacent image blocks overlap.

7. The image feature acquisition method according to claim 4, characterized in that: The extracting feature information of the image block by using a scale-invariant feature transformation algorithm includes: Constructing a Gaussian pyramid model of the image block, and constructing a Gaussian difference pyramid model based on the Gaussian pyramid model; Performing extreme point detection on the Gaussian difference pyramid model, and determining the detected extreme points as key points of the image block; Positioning the key points to determine the scale and position of the key points, and allocating angles to the key points to determine the directions of the key points; Sorting the key points in descending order according to the response values ​​of the key points, and obtaining a preset number of target key points with top order; According to the position, scale and direction of each of the target key points, a feature vector of the target key point is determined, and feature information of the image block is generated according to each of the target key points.

8. An image feature acquisition device, characterized in that: Applied to unmanned equipment, the unmanned equipment includes an image acquisition device, a digital signal processor and a central processing unit, the digital signal processor is connected to the image acquisition device and the central processing unit, and the device includes: an image acquisition module, configured to acquire a first image through the image acquisition device, and transmit the first image to the digital signal processor; an image processing module, configured to perform block processing on the first image through the digital signal processor to obtain a plurality of image blocks, extract feature information of each of the image blocks, generate feature information of the first image according to the feature information of each of the image blocks, and transmit the feature information of the first image to the central processing unit; The feature acquisition module is configured to acquire feature information of the first image transmitted by the digital signal processor through the central processing unit.

9. An unmanned device, characterized in that: include: one or more processors; A memory stores one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the image feature acquisition method as described in any one of claims 1-7.

10. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to execute the image feature acquisition method as described in any one of claims 1 to 7 when executed by a computer processor.

Images