Robot target detection method and device, computer equipment and readable storage medium
Through the combination of backbone network and area generation network, the problem that existing object detection algorithms cannot identify unlabeled object categories is solved, and the robot's ability to effectively detect obstacles in unknown environments is realized, and the detection flexibility and adaptability are improved.
Patent Information
- Application Number
- CN202411193183.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2024-12-20
AI Technical Summary
The existing object detection algorithm mainly relies on supervised learning and cannot identify unlabeled objects in the dataset, resulting in the robot being unable to effectively avoid obstacles when facing unknown environments.
Using a combination of backbone network and region generation network, the detection of unknown objects is achieved through feature extraction, anchor box generation, interchange ratio screening, far-reaching and near-degree value sorting and area of interest pooling. .
The robot's ability to effectively detect and respond to obstacles in unknown or complex environments is realized, and the flexibility and adaptability of target detection is improved.
Smart Images
Figure CN119169261A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of robot technology, and in particular to a robot target detection method, device, computer equipment and readable storage medium. Background Art
[0002] In recent years, with the rapid development of artificial intelligence technology, especially the breakthrough of deep learning in the field of computer vision, object detection has become one of the key technologies in robot-assisted obstacle avoidance. Object detection detects the target in front and feeds back the object category and position information to the robot, so that the robot can avoid obstacles.
[0003] However, most existing object detection algorithms are based on supervised learning, which relies on a large number of labeled data sets for training. Therefore, these algorithms can only identify object categories that have been labeled in the data set. For object categories that are not included in the data set, the algorithm often cannot effectively identify them, which will result in the robot being unable to effectively avoid obstacles. Summary of the invention
[0004] In view of this, the purpose of the present invention is to overcome the deficiencies in the prior art and to provide a robot target detection method, device, computer equipment and readable storage medium.
[0005] The present invention provides the following technical solutions:
[0006] In a first aspect, an embodiment of the present disclosure provides a robot target detection method, the method comprising:
[0007] Acquire an original image, input the original image into a backbone network, and perform feature extraction on the original image through the backbone network to obtain a feature map;
[0008] Input the feature map into a region generation network, generate multiple anchor boxes according to each pixel point of the feature map, and obtain multiple real boxes of the feature map;
[0009] According to the intersection-and-union ratio of each anchor point frame and its true frame, a plurality of positive sample anchor point frames are determined from each anchor point frame, wherein the intersection-and-union ratio is determined according to the intersection area and the union area of each anchor point frame and its true frame;
[0010] According to the distance between the center point of each positive sample anchor point frame and the boundary of its true frame, obtaining the distance value between the center point of each positive sample anchor point frame and the center point of its true frame;
[0011] Arrange the distance values in descending order, perform a region of interest pooling operation on the positive sample anchor point frames corresponding to a preset number of distance values in the front row, and obtain multiple candidate frames, each of which has a corresponding coordinate value;
[0012] According to the intersection area and the union area between each candidate frame and its real frame, the quality score of each candidate frame is obtained, and the target frame whose quality score is greater than the preset quality threshold is selected from each candidate frame, and the target detection is performed according to the coordinate values of each target frame.
[0013] Optionally, the backbone network includes a convolution layer and a pooling layer, and the extracting features of the original image through the backbone network to obtain a feature map includes:
[0014] Performing a convolution operation on the original image through the convolution layer to extract an initial feature map in the original image;
[0015] The pooling layer performs a maximum pooling operation and an average pooling operation on the initial feature map to obtain the feature map.
[0016] Optionally, generating a plurality of anchor boxes according to each pixel point of the feature map includes:
[0017] Determine the receptive field size corresponding to each pixel point of the feature map, and determine the basic size according to the receptive field size;
[0018] A preset scale and a preset aspect ratio are obtained, and a corresponding anchor point frame is generated on each pixel point of the feature map according to the basic size, the preset scale and the preset aspect ratio.
[0019] Optionally, determining a plurality of positive sample anchor frames from each of the anchor frames according to the intersection ratio of each of the anchor frames and its true frame includes:
[0020] If the intersection ratio of the current anchor box and its true box is greater than a preset positive threshold, the current anchor box is determined to be the positive sample anchor box;
[0021] If the intersection ratio of the current anchor box and its true box is less than the preset negative threshold, the current anchor box is determined to be a negative sample anchor box.
[0022] Optionally, obtaining the distance value between the center point of each positive sample anchor point frame and the center point of its true frame according to the distance between the center point of each positive sample anchor point frame and the boundary of its true frame includes:
[0023] Obtain a first distance between the center point of each positive sample anchor box and the left border of its true box, obtain a second distance between the center point of each positive sample anchor box and the right border of its true box, obtain a third distance between the center point of each positive sample anchor box and the upper border of its true box, and obtain a fourth distance between the center point of each positive sample anchor box and the lower border of the corresponding true box;
[0024] The distance value between the center point of each positive sample anchor point frame and the center point of its true frame is calculated according to the first distance, the second distance, the third distance and the fourth distance using a preset calculation formula, wherein the preset calculation formula is:
[0025]
[0026] In the formula, centerness * is the distance value, l * is the first distance, r * is the second distance, t * is the third distance, b * is the fourth distance.
[0027] Optionally, the region of interest pooling operation is performed on the positive sample anchor point frames corresponding to a preset number of distance values located in the front column to obtain multiple candidate frames, including:
[0028] Mapping the coordinates of the positive sample anchor point boxes corresponding to the preset number of distance values located in the front column to the feature area, and dividing the mapped feature area into a plurality of sub-areas of preset sizes;
[0029] A maximum pooling operation is performed on each of the sub-regions to obtain a corresponding plurality of regional features, and each of the regional features is spliced to obtain a corresponding plurality of candidate boxes.
[0030] Optionally, performing target detection according to the coordinate values of each target frame includes:
[0031] The coordinate values of each target frame are optimized by using an intersection-over-union loss function and a regression operation to obtain optimized coordinate values of each candidate frame;
[0032] Target detection is performed according to the optimized coordinate values of each target frame.
[0033] In a second aspect, an embodiment of the present disclosure provides a robot target detection device, the device comprising:
[0034] An extraction module is used to obtain an original image, input the original image into a backbone network, and perform feature extraction on the original image through the backbone network to obtain a feature map;
[0035] A generation module, used to input the feature map into a region generation network, generate multiple anchor boxes according to each pixel point of the feature map, and obtain multiple real boxes of the feature map;
[0036] A determination module, configured to determine a plurality of positive sample anchor frames from each of the anchor frames according to an intersection-and-union ratio of each of the anchor frames and its true frame, wherein the intersection-and-union ratio is determined according to an intersection area and a union area of each of the anchor frames and its true frame;
[0037] An acquisition module, used for acquiring a distance value between the center point of each positive sample anchor point frame and the center point of its true frame according to the distance between the center point of each positive sample anchor point frame and the boundary of its true frame;
[0038] A sorting module is used to arrange the distance values in descending order, perform a region of interest pooling operation on the positive sample anchor point frames corresponding to a preset number of distance values in the front row, and obtain multiple candidate frames, each of which has a corresponding coordinate value;
[0039] The detection module is used to obtain the quality score of each candidate frame based on the intersection area and the union area between each candidate frame and its real frame, select the target frame whose quality score is greater than a preset quality threshold from each candidate frame, and perform target detection according to the coordinate values of each target frame.
[0040] In a third aspect, a computer device is provided in an embodiment of the present disclosure, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the robot target detection method described in the first aspect when executing the computer program.
[0041] In a fourth aspect, a computer-readable storage medium is provided in an embodiment of the present disclosure, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the robot target detection method described in the first aspect are implemented.
[0042] Beneficial effects of this application:
[0043] The robot target detection method provided by the embodiment of the present application obtains an original image, inputs the original image into a backbone network, extracts features of the original image through the backbone network, and obtains a feature map; inputs the feature map into a region generation network, generates multiple anchor boxes according to each pixel point of the feature map, and obtains multiple real boxes of the feature map; determines multiple positive sample anchor boxes from each anchor box according to the intersection ratio of each anchor box and its real box, wherein the intersection ratio is determined according to the intersection area and the union area of each anchor box and its real box; the degree of distance is determined according to the center point of each positive sample anchor box. and the distance between the border of its real frame, obtain the distance value between the center point of each positive sample anchor frame and the center point of its real frame; arrange the distance values in order from large to small, perform a region of interest pooling operation on the positive sample anchor frames corresponding to a preset number of distance values in the front row, and obtain multiple candidate frames, each of which has a corresponding coordinate value; according to the intersection area and union area between each candidate frame and its real frame, obtain the quality score of each candidate frame, select the target frame whose quality score is greater than the preset quality threshold from each candidate frame, and perform target detection according to the coordinate value of each target frame. This application is based on a general target detection method, and only performs target detection based on the position and quality information of the object in front, and does not rely on the detection of objects of a specific category, so that the robot can effectively detect and respond to obstacles even when facing unknown or complex environments, and can be flexibly applied in different environments and tasks.
[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work. In each of the drawings, similar components are numbered similarly.
[0046] Figure 1 A flow chart of a robot target detection method provided by an embodiment of the present application is shown;
[0047] Figure 2 A flowchart of another robot target detection method provided by an embodiment of the present application is shown;
[0048] Figure 3 A flowchart of another robot target detection method provided by an embodiment of the present application is shown;
[0049] Figure 4 A schematic diagram of the structure of a robot target detection device provided in an embodiment of the present application is shown;
[0050] Figure 5 A schematic diagram of the structure of a computer device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0051] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.
[0052] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. In contrast, when an element is referred to as being "directly on" another element, there is no intermediate element. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.
[0053] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0054] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of the template are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.
[0056] Example 1
[0057] like Figure 1 , which is a flow chart of a robot target detection method in an embodiment of the present application, the robot target detection method provided in the embodiment of the present application comprises the following steps:
[0058] Step S110, obtaining an original image, inputting the original image into a backbone network, performing feature extraction on the original image through the backbone network, and obtaining a feature map.
[0059] In this embodiment, after the robot captures the original image (RGB image or grayscale image), the backbone network is used to extract features of the original image. The backbone network is the core part of the deep learning model, especially when processing image-related tasks. It is composed of multiple convolutional layers, pooling layers, activation functions, etc.
[0060] Specifically, the original image is convolved through the convolution layer to extract the initial feature map containing important features (such as edges, corners, etc.) in the original image, and then the initial feature map is subjected to maximum pooling and average pooling operations through the pooling layer after the convolution layer. While reducing the dimension and amount of calculation of the initial feature map, important feature information is also retained. Usually, an activation function (such as ReLU, Sigmoid, Tanh, etc.) is used to increase the nonlinear ability of the backbone network so that the backbone network can learn more complex feature representations. Finally, after a combination of multiple convolutional layers, pooling layers, and activation functions, the backbone network will output a series of feature maps. These feature maps contain high-level semantic information in the original image and can be used for subsequent object detection tasks.
[0061] Step S120, input the feature map into the region generation network, generate multiple anchor boxes according to each pixel point of the feature map, and obtain multiple real boxes of the feature map.
[0062] It should be noted that the main function of the RPN (Region Proposal Network) stage is to extract candidate boxes that may contain targets from the feature map.
[0063] In an optional embodiment, if Figure 2 As shown, step S120 includes:
[0064] Step S121, determining the receptive field size corresponding to each pixel point of the feature map, and determining the basic size according to the receptive field size;
[0065] Step S122, obtaining a preset scale and a preset aspect ratio, and generating a corresponding anchor point box at each pixel point of the feature map according to the basic size, the preset scale and the preset aspect ratio.
[0066] First, determine the receptive field size corresponding to each pixel on the feature map, and determine the basic size of the anchor box based on the receptive field size. For example, if each pixel on the feature map corresponds to a 16x16 area on the original image, the basic size may be set to 16. Then obtain the preset scale and preset aspect ratio. The scale and aspect ratio determine the diversity and coverage of the anchor box. For example, different scales (such as 8, 16, 32) and aspect ratios (such as 0.5, 1, 2) can be set to generate anchor boxes of different sizes and shapes. According to the basic size, preset scale and preset aspect ratio, a corresponding anchor box is generated at each pixel of the feature map. Anchor boxes of different sizes and ratios can cover targets of various shapes and sizes, thereby improving the robustness of detection.
[0067] Furthermore, multiple real boxes are obtained in the feature map. The real boxes represent the position and size of the target in the feature map in the real world. They are supervisory information in the target detection task and are usually obtained through some automatic labeling tools. This embodiment does not limit this.
[0068] Step S130, determining a plurality of positive sample anchor frames from each anchor frame according to the intersection-and-union ratio of each anchor frame and its true frame, wherein the intersection-and-union ratio is determined according to the intersection area and the union area of each anchor frame and its true frame.
[0069] Understandably, for each anchor box, the intersection ratio between it and the corresponding real box is calculated, that is, the ratio of the intersection area of the anchor box and its real box to the union area.
[0070] If the intersection ratio of an anchor box and its true box is greater than a preset positive threshold (such as 0.7), the anchor box is a positive sample anchor box (i.e., foreground). At the same time, in order to ensure that each true box corresponds to at least one positive sample anchor box, the anchor box with the largest intersection ratio with each true box will be selected as the positive sample anchor box.
[0071] If the intersection ratio of an anchor box and its true box is less than a preset negative threshold (such as 0.3), the anchor box is a negative sample anchor box (i.e., background).
[0072] It should be noted that the anchor boxes whose intersection ratio is between the preset positive threshold and the preset negative threshold are usually ignored and do not participate in the subsequent target detection task.
[0073] Step S140, obtaining the distance value between the center point of each positive sample anchor point frame and the center point of its true frame according to the distance between the center point of each positive sample anchor point frame and the boundary of its true frame.
[0074] Furthermore, the region generation network in this embodiment does not use a binary classification head, but uses positioning quality for representation, that is, the centerness value is used to represent the positioning quality of the anchor boxes whose sample categories are positive samples of the region generation network, and these anchor boxes are optimized. The centerness value is an indicator for measuring the distance from the center point of the anchor box to the center point of its corresponding real box.
[0075] Specifically, first obtain a first distance between the center point of each positive sample anchor box and the left boundary of its true frame, obtain a second distance between the center point of each positive sample anchor box and the right boundary of its true frame, obtain a third distance between the center point of each positive sample anchor box and the upper boundary of its true frame, obtain a fourth distance between the center point of each positive sample anchor box and the lower boundary of the corresponding true frame, and then use a preset calculation formula to calculate the distance value between the center point of each positive sample anchor box and the center point of its true frame according to the first distance, the second distance, the third distance and the fourth distance, wherein the preset calculation formula is as follows:
[0076]
[0077] In the formula, centerness * is the distance value, l * is the first distance, r * is the second distance, t * is the third distance, b * The fourth distance.
[0078] It can be understood that the closer the distance between the center point of the anchor box and the center point of its corresponding real box is, the closer the distance value is to 1, and vice versa, the farther the distance is, the closer the distance value is to 0.
[0079] It should be noted that, in this embodiment, the region generation network also includes a bounding box regression operation, which is used to correct the position and size of the anchor box to make it closer to the object boundary of the real box.
[0080] Step S150, arrange the distance values in descending order, perform a region of interest pooling operation on the positive sample anchor point frames corresponding to a preset number of distance values in the front row, and obtain multiple candidate frames, each of which has a corresponding coordinate value.
[0081] Furthermore, after calculating the distance value of each positive sample anchor point frame, the distance values are arranged in descending order, and the positive sample anchor point frames corresponding to the preset number (e.g., 100) of distance values in the front row are pooled into regions of interest, such as Figure 3 As shown, step S140 includes:
[0082] Step S141, mapping the coordinates of the positive sample anchor point boxes corresponding to a preset number of distance values in the front column to the feature region, and dividing the mapped feature region into a plurality of sub-regions of preset sizes;
[0083] Step S142, performing a maximum pooling operation on each sub-region to obtain a corresponding plurality of regional features, and concatenating the regional features to obtain a corresponding plurality of candidate boxes.
[0084] Specifically, the coordinates of the positive sample anchor point boxes corresponding to a preset number of distance values in the front column are mapped to the feature area, and the mapped feature area is divided into multiple sub-areas of preset sizes (usually grids, and the number of sub-areas must correspond to the size of the candidate boxes output subsequently); a maximum pooling operation is performed on each sub-area (i.e., the maximum value in each sub-area is taken as the representative value of the area) to obtain multiple corresponding regional features, and the regional features are spliced together to form multiple candidate boxes of fixed sizes, each of which has a corresponding coordinate value.
[0085] The above method can significantly speed up the target detection through the region of interest pooling operation, and has high flexibility and maintains high detection accuracy.
[0086] Step S160, obtaining the quality score of each candidate frame based on the intersection area and the union area between each candidate frame and its real frame, selecting a target frame whose quality score is greater than a preset quality threshold from each candidate frame, and performing target detection based on the coordinate values of each target frame.
[0087] The intersection-over-union ratio of each candidate box and its true box is calculated according to the calculation method of the intersection-over-union ratio in step S130, and the intersection-over-union ratio at this time is used as the quality score IoU Score of the candidate box in the target detection stage to represent the positioning quality.
[0088] The target frame whose quality score is greater than the preset quality threshold is selected from each candidate frame, and the coordinate values corresponding to each target frame are optimized using the intersection-over-union loss function IoU loss and regression operation to obtain the optimized coordinate values of each target frame, and then the target detection task is performed based on the optimized coordinate values of these target frames.
[0089] Compared with the traditional L1 or L2 loss function, the intersection-over-union loss function IoU loss can more directly reflect the degree of match between the candidate box and the real box, and has scale invariance, that is, for objects of different sizes, IoU loss can give a reasonable gradient to guide network learning. The position parameters of the candidate box (such as center point coordinates, width and height) can be adjusted according to the gradient of IoU loss to reduce the value of IoU loss, thereby improving the overlap between the candidate box and the real box, that is, improving the positioning quality.
[0090] Through the above method, only the quality score and coordinate value of the target box will be output in the end, and the category of the target box will not be output, so that the robot can detect the object in the foreground based on the quality score and coordinate value alone, without relying on the detection of objects of a specific category. By detecting all objects in the field of view, the robot can achieve autonomous obstacle avoidance. Experiments have shown that this method can effectively improve the recall of the target compared to ordinary target detection, because ordinary target detection will treat those foreground objects that are not in the category of the data set as background, which will damage the performance of the network.
[0091] The robot target detection method provided by the embodiment of the present application obtains an original image, inputs the original image into a backbone network, extracts features of the original image through the backbone network, and obtains a feature map; inputs the feature map into a region generation network, generates multiple anchor boxes according to each pixel point of the feature map, and obtains multiple real boxes of the feature map; determines multiple positive sample anchor boxes from each anchor box according to the intersection ratio of each anchor box and its real box, wherein the intersection ratio is determined according to the intersection area and the union area of each anchor box and its real box; and determines the positive sample anchor boxes according to the center point of each positive sample anchor box and its real box. , obtain the distance between the boundaries of each positive sample anchor box and the center point of its true box; arrange the distance values in order from large to small, perform region of interest pooling operations on the positive sample anchor boxes corresponding to the preset number of distance values in the front row, and obtain multiple candidate boxes, each candidate box corresponding to a coordinate value; according to the intersection area and union area between each candidate box and its true box, obtain the quality score of each candidate box, select the target box with a quality score greater than the preset quality threshold from each candidate box, and perform target detection according to the coordinate values of each target box. This application is based on a general target detection method, and only performs target detection based on the position and quality information of the object in front, and does not rely on the detection of objects of a specific category, so that the robot can effectively detect and respond to obstacles even when facing unknown or complex environments, and can be flexibly applied in different environments and tasks.
[0092] Example 2
[0093] like Figure 4FIG. 4 is a schematic diagram of a robot target detection device 400 according to an embodiment of the present application, wherein the device comprises:
[0094] The extraction module 410 is used to obtain the original image, input the original image into the backbone network, and perform feature extraction on the original image through the backbone network to obtain a feature map;
[0095] A generation module 420 is used to input the feature map into a region generation network, generate multiple anchor boxes according to each pixel of the feature map, and obtain multiple real boxes of the feature map;
[0096] A determination module 430 is used to determine a plurality of positive sample anchor frames from each anchor frame according to an intersection-and-union ratio of each anchor frame and its true frame, wherein the intersection-and-union ratio is determined according to an intersection area and a union area of each anchor frame and its true frame;
[0097] An acquisition module 440 is used to acquire a distance value between the center point of each positive sample anchor point frame and the center point of its true frame according to the distance between the center point of each positive sample anchor point frame and the boundary of its true frame;
[0098] A sorting module 450 is used to arrange the distance values in descending order, perform a region of interest pooling operation on the positive sample anchor point frames corresponding to a preset number of distance values in the front row, and obtain multiple candidate frames, each of which has a corresponding coordinate value;
[0099] The detection module 460 is used to obtain the quality score of each candidate frame based on the intersection area and the union area between each candidate frame and its real frame, select the target frame whose quality score is greater than the preset quality threshold from each candidate frame, and perform target detection according to the coordinate values of each target frame.
[0100] The robot target detection device provided in the embodiment of the present application performs target detection only based on the position and mass information of the object in front, and does not rely on the detection of objects of a specific category. This allows the robot to effectively detect and respond to obstacles even when facing unknown or complex environments, and can be flexibly used in different environments and tasks.
[0101] Example 3
[0102] The present application also provides a computer device. Figure 5 , Figure 5 This is a basic structural block diagram of the computer device in this embodiment.
[0103] The computer device 5 includes a memory 51, a processor 52, and a network interface 53 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 5 with a memory 51, a processor 52, and a network interface 53, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.
[0104] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.
[0105] The memory 51 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or D slot compatibility test memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 51 can be an internal storage unit of the computer device 5, such as a hard disk or memory of the computer device 5. In other embodiments, the memory 51 can also be an external storage device of the computer device 5, such as a plug-in hard disk equipped on the computer device 5, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 51 can also include both the internal storage unit of the computer device 5 and its external storage device. In this embodiment, the memory 51 is generally used to store the operating system and various application software installed on the computer device 5, such as computer-readable instructions of the slot compatibility test method, etc. In addition, the memory 51 can also be used to temporarily store various types of data that have been output or are to be output.
[0106] The processor 52 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other robot target detection chip in some embodiments. The processor 52 is generally used to control the overall operation of the computer device 5. In this embodiment, the processor 52 is used to run computer-readable instructions or process data stored in the memory 51, such as computer-readable instructions for running the slot compatibility test method.
[0107] The network interface 53 may include a wireless network interface or a wired network interface. The network interface 53 is generally used to establish a communication connection between the computer device 5 and other electronic devices.
[0108] The computer device provided in this embodiment can execute the above robot target detection method. Here, the robot target detection method can be the robot target detection method of each of the above embodiments.
[0109] Example 4
[0110] This embodiment further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the robot target detection method in the embodiment are implemented.
[0111] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium can be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium can also be an external storage device of a computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, referred to as SMC), a secure digital (Secure Digital, referred to as SD) card, a flash card, etc. Of course, the computer-readable storage medium can also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the computer-readable storage medium is generally used to store an operating system and various application software installed on the computer device. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or are to be output.
[0112] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow chart, and the combination of boxes in the structure diagram and / or the flow chart, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0113] In addition, the functional modules or units in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.
[0114] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or partly contributed to the prior art or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium can be a non-volatile storage medium or a volatile storage medium. For example, the storage medium can be: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes.
[0115] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. A robot target detection method, characterized in that: The method comprises: Acquire an original image, input the original image into a backbone network, and perform feature extraction on the original image through the backbone network to obtain a feature map; Input the feature map into a region generation network, generate multiple anchor boxes according to each pixel point of the feature map, and obtain multiple real boxes of the feature map; According to the intersection-and-union ratio of each anchor point frame and its true frame, a plurality of positive sample anchor point frames are determined from each anchor point frame, wherein the intersection-and-union ratio is determined according to the intersection area and the union area of each anchor point frame and its true frame; According to the distance between the center point of each positive sample anchor point frame and the boundary of its true frame, obtaining the distance value between the center point of each positive sample anchor point frame and the center point of its true frame; Arrange the distance values in descending order, perform a region of interest pooling operation on the positive sample anchor point frames corresponding to a preset number of distance values in the front row, and obtain multiple candidate frames, each of which has a corresponding coordinate value; According to the intersection area and the union area between each candidate frame and its real frame, the quality score of each candidate frame is obtained, and the target frame whose quality score is greater than the preset quality threshold is selected from each candidate frame, and the target detection is performed according to the coordinate values of each target frame.
2. The robot target detection method according to claim 1, characterized in that: The backbone network includes a convolution layer and a pooling layer, and the feature extraction of the original image through the backbone network to obtain a feature map includes: Performing a convolution operation on the original image through the convolution layer to extract an initial feature map in the original image; The pooling layer performs a maximum pooling operation and an average pooling operation on the initial feature map to obtain the feature map.
3. The robot target detection method according to claim 1, characterized in that: The generating a plurality of anchor boxes according to each pixel point of the feature map comprises: Determine the receptive field size corresponding to each pixel point of the feature map, and determine the basic size according to the receptive field size; A preset scale and a preset aspect ratio are obtained, and a corresponding anchor point frame is generated on each pixel point of the feature map according to the basic size, the preset scale and the preset aspect ratio.
4. The robot target detection method according to claim 1, characterized in that: The determining of a plurality of positive sample anchor frames from each of the anchor frames according to the intersection ratio of each of the anchor frames and its real frame includes: If the intersection ratio of the current anchor box and its true box is greater than a preset positive threshold, the current anchor box is determined to be the positive sample anchor box; If the intersection ratio of the current anchor box and its true box is less than the preset negative threshold, the current anchor box is determined to be a negative sample anchor box.
5. The robot target detection method according to claim 1, characterized in that: The obtaining, according to the distance between the center point of each positive sample anchor point frame and the boundary of its true frame, the distance value between the center point of each positive sample anchor point frame and the center point of its true frame comprises: Obtain a first distance between the center point of each positive sample anchor box and the left border of its true box, obtain a second distance between the center point of each positive sample anchor box and the right border of its true box, obtain a third distance between the center point of each positive sample anchor box and the upper border of its true box, and obtain a fourth distance between the center point of each positive sample anchor box and the lower border of the corresponding true box; The distance value between the center point of each positive sample anchor point frame and the center point of its true frame is calculated according to the first distance, the second distance, the third distance and the fourth distance using a preset calculation formula, wherein the preset calculation formula is: In the formula, centerness * is the distance value, l * is the first distance, r * is the second distance, t * is the third distance, b * is the fourth distance.
6. The robot target detection method according to claim 1, characterized in that: The region of interest pooling operation is performed on the positive sample anchor point frames corresponding to the preset number of distance values in the front row to obtain multiple candidate frames, including: Mapping the coordinates of the positive sample anchor point boxes corresponding to the preset number of distance values located in the front column to the feature area, and dividing the mapped feature area into a plurality of sub-areas of preset sizes; A maximum pooling operation is performed on each of the sub-regions to obtain a corresponding plurality of regional features, and each of the regional features is spliced to obtain a corresponding plurality of candidate boxes.
7. The robot target detection method according to claim 6, characterized in that: The performing target detection according to the coordinate values of each target frame includes: The coordinate values of each target frame are optimized by using an intersection-over-union loss function and a regression operation to obtain optimized coordinate values of each candidate frame; Target detection is performed according to the optimized coordinate values of each target frame.
8. A robot target detection device, characterized in that: The device comprises: An extraction module is used to obtain an original image, input the original image into a backbone network, and perform feature extraction on the original image through the backbone network to obtain a feature map; A generation module, used to input the feature map into a region generation network, generate multiple anchor boxes according to each pixel point of the feature map, and obtain multiple real boxes of the feature map; A determination module, configured to determine a plurality of positive sample anchor frames from each of the anchor frames according to an intersection-and-union ratio of each of the anchor frames and its true frame, wherein the intersection-and-union ratio is determined according to an intersection area and a union area of each of the anchor frames and its true frame; An acquisition module, used for acquiring a distance value between the center point of each positive sample anchor point frame and the center point of its true frame according to the distance between the center point of each positive sample anchor point frame and the boundary of its true frame; A sorting module is used to arrange the distance values in descending order, perform a region of interest pooling operation on the positive sample anchor point frames corresponding to a preset number of distance values in the front row, and obtain multiple candidate frames, each of which has a corresponding coordinate value; The detection module is used to obtain the quality score of each candidate frame based on the intersection area and the union area between each candidate frame and its real frame, select the target frame whose quality score is greater than a preset quality threshold from each candidate frame, and perform target detection according to the coordinate values of each target frame.
9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the robot target detection method according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the robot target detection method according to any one of claims 1 to 7 are implemented.